Sumber : http://www.usq.edu.au/library/help/postgrad/resmeth.htm
http://www.tardis.ed.ac.uk/~kate/qmcweb/q2.htm
A hypothetical question is one in which you are asking respondents to indicate what they think they would do under particular imaginary circumstances. These can't always be avoided in some attitudinal research, but they are difficult to administer and often give rise to unreliable answers
These are often included in poor questionnaires because the researcher feels strongly about a topic and assumes that everyone will be of the same opinion.
Problems which tax the respondent's memory too much are likely to lead to non-response or inaccurate replies. For example "What did you have for lunch each day last week?"
For example, "What is your National Insurance number?"
 Personal details / health / age

 Income
If you have to ask sensitive questions, the problem can be alleviated somewhat by the use of SHOW CARDS . Put all of the possible responses on a card, preferably mixed up, and ask the respondent to indicate which number relates to their own circumstances. For example,
Can you tell me the number on this card which corresponds to you income group?
SHOW CARD WITH……
1.
£7,000 - 12,000
2.
Over £60,000
3.
£18,000 - £30,000
4.
Under £7,000
5.
£40,000 - £60,000
6.
£12,000 - £18,000
7.
£30,000 - £40,000
In the show card above, you will note that somebody earning exactly £30,000 would perhaps wonder whether to give answer 3 or answer 7 on the show card. In practice, people are usually able to give their income as an approximation. You should, however, always watch out for questions where the multiple choice answers are not mutually exclusive and where a respondent will be uncertain about which category he/she falls under. It seems to be a particular problem with age brackets, and you can often see examples of mistakes here in even professionally produced surveys.
If your questions are too long and detailed, the respondent may get lost and the responses will relate only to the beginning or the end of the question. Where definitions and qualifications are necessary, use show cards.


Influenced by:
1. The number of variables being examined



Univariate analysis = one variable (e.g. gender, age) Methods: charts (e.g. bar chart, pie chart) or frequency table
Bivariate = 2 variables (e.g. gender + purchase of computers) Methods: crosstabulations; scatterplots; regression; comparison of means (but we will only be covering the first two in this module)
Multivariate = 3 or more variables (e.g. income influenced by both education level and gender) - Note: you'll be relieved to hear that you won't be covering any of these as part of this module)
2. Levels of measurement
 Nominal


Ordinal
 Interval
Ratio
3. Descriptive or inferential?
 Descriptive statistics : summarise patterns in responses (e.g. average age of respondents or number of respondents who buy a product)
 Inferential statistics : provide an idea about whether the patterns described in the sample are likely to apply in the population from which the sample is drawn
("tests of significance")
Should have separate columns for:


(a) number of people who gave each answer (N)
(b) percentage column
If (a) is excluded from the table, it is essential to include the total number of cases on whom percentages are calculated.
Example
Analysis of degree of agreement/disagreement with statement "Taking part in videoconferenced classes is a really exciting experience"
Answer N %

Strongly agree 254 34
Agree 201 27
Neutral
Disagree
119
97
16
13
Strongly Disagree 75 10
Total 746 100
 Crosstabulation = tabular representation of relationship between two (or can be more than two) variables


Also called CONTINGENCY TABLE
Table divided into cells with each cell representing the coincidence of a specific value from each variable
 The total for each row and each column is given at the end of the row and column.
These totals are called the ROW MARGINALS and COLUMN MARGINALS.
 It is normal (but not absolutely essential) for the dependent variable to appear in rows. This is because we may be crosstabulating one dependent variable by a series of possible explanatory variables.
A hypothesis can be described as "a tentative answer to a research question" or a
"provisional prediction". The plural is hypotheses .
Hypotheses should be:
 Stated clearly, using appropriate terminology
 Testable
 Statement of relationships between variables
 Limited in scope
Examples of hypotheses:
 Health education programmes influence the number of people who smoke


Newspapers affect people's voting patterns
 Attendance at lectures influences exam marks
Diet influences intelligence
In the above examples, "something" (e.g. diet, lecture attendance) affects "something else" (e.g. intelligence, exam marks). These are variables. A variable is anything which is

free to vary, and in order to describe them quantitatively, they have to be expressed in appropriate units (e.g. IQ scores, exam percentages)
The pairs of variables in the above examples have separate names. The variable we manipulate is called the independent variable (IV) . The variable we are hypothesising will alter as a result of our manipulations is called the dependent variable (DV) . The dependent variable alters as a consequence of the value of the independent variable - its value is dependent on this. The value of the independent variable is free to vary according to the whims of the experimenters.
Independent variable

Health education programmes

Newspaper

Attendance at lectures

Diet
Dependent variable

Number of people who smoke

Voting patterns

Exam marks

Intelligence
NB Many variables can be either dependent or independent, within the context of a particular study. For example, it could be argued that "Intelligence influences diet" or
"Exam marks influence attendance at lectures".
In the examples above, words like "influence" or "affects" are used without indicating direction. When a hypothesis states a predicted outcome (using words such as reduce, increase, lower, raise - it is called a directional or one-tailed hypothesis . Vaguer types of hypotheses (such as the ones given earlier) are known as non-directional or twotailed hypotheses .
Sumber : http://www.usq.edu.au/library/help/postgrad/resmeth.htm
http://www.tardis.ed.ac.uk/~kate/qmcweb/q2.htm
 Your respondents may refuse to co-operate if your survey begins with awkward or embarrassing questions.



People are more likely to give honest replies to personal questions if some rapport has been developed with the interviewer.
For the above reasons, it is generally best to keep all questions dealing with demographic information (such as age) at the end of the questionnaire.
 Print clearly (if computer printout is giving faint printouts, ask the IT staff whether a new print cartridge can be installed)
 Allow adequate space between questions so that you can write down any comments made (but don't waste too much paper!!)
 Write the questions themselves in lower case (i.e. like this writing) ,
INSTRUCTIONS IN UPPER CASE (i.e. capital letters)
A hypothetical question is one in which you are asking respondents to indicate what they think they would do under particular imaginary circumstances. These can't always be avoided in some attitudinal research, but they are difficult to administer and often give rise to unreliable answers
These are often included in poor questionnaires because the researcher feels strongly about a topic and assumes that everyone will be of the same opinion.
Problems which tax the respondent's memory too much are likely to lead to non-response or inaccurate replies. For example "What did you have for lunch each day last week?"

For example, "What is your National Insurance number?"


Personal details / health / age
Income
If you have to ask sensitive questions, the problem can be alleviated somewhat by the use of SHOW CARDS . Put all of the possible responses on a card, preferably mixed up, and ask the respondent to indicate which number relates to their own circumstances. For example,
Can you tell me the number on this card which corresponds to you income group?
SHOW CARD WITH……
8.
£7,000 - 12,000
9.
Over £60,000
10.
£18,000 - £30,000
11.
Under £7,000
12.
£40,000 - £60,000
13.
£12,000 - £18,000
14.
£30,000 - £40,000
In the show card above, you will note that somebody earning exactly £30,000 would perhaps wonder whether to give answer 3 or answer 7 on the show card. In practice, people are usually able to give their income as an approximation. You should, however, always watch out for questions where the multiple choice answers are not mutually exclusive and where a respondent will be uncertain about which category he/she falls under. It seems to be a particular problem with age brackets, and you can often see examples of mistakes here in even professionally produced surveys.
If your questions are too long and detailed, the respondent may get lost and the responses will relate only to the beginning or the end of the question. Where definitions and qualifications are necessary, use show cards.

Influenced by:
1. The number of variables being examined
 Univariate analysis = one variable (e.g. gender, age) Methods: charts (e.g. bar chart, pie chart) or frequency table
 Bivariate = 2 variables (e.g. gender + purchase of computers) Methods: crosstabulations; scatterplots; regression; comparison of means (but we will only
 be covering the first two in this module)
Multivariate = 3 or more variables (e.g. income influenced by both education level and gender) - Note: you'll be relieved to hear that you won't be covering any of these as part of this module)
2. Levels of measurement
 Nominal
 Ordinal
 Interval
 Ratio
3. Descriptive or inferential?
 Descriptive statistics : summarise patterns in responses (e.g. average age of
 respondents or number of respondents who buy a product)
Inferential statistics : provide an idea about whether the patterns described in the sample are likely to apply in the population from which the sample is drawn
("tests of significance")
Should have separate columns for:


(a) number of people who gave each answer (N)
(b) percentage column
If (a) is excluded from the table, it is essential to include the total number of cases on whom percentages are calculated.
Example

Analysis of degree of agreement/disagreement with statement "Taking part in videoconferenced classes is a really exciting experience"
Answer N %
Strongly agree 254 34
Agree 201 27
Neutral
Disagree
119 16
97 13
Strongly Disagree 75 10
Total 746 100
 Crosstabulation = tabular representation of relationship between two (or can be more than two) variables
 Also called CONTINGENCY TABLE
 Table divided into cells with each cell representing the coincidence of a specific

 value from each variable
The total for each row and each column is given at the end of the row and column.
These totals are called the ROW MARGINALS and COLUMN MARGINALS.
It is normal (but not absolutely essential) for the dependent variable to appear in rows. This is because we may be crosstabulating one dependent variable by a series of possible explanatory variables.
A hypothesis can be described as "a tentative answer to a research question" or a
"provisional prediction". The plural is hypotheses .
Hypotheses should be:
 Stated clearly, using appropriate terminology
 Testable
 Statement of relationships between variables
 Limited in scope
Examples of hypotheses:
 Health education programmes influence the number of people who smoke
 Newspapers affect people's voting patterns



Attendance at lectures influences exam marks
Diet influences intelligence
In the above examples, "something" (e.g. diet, lecture attendance) affects "something else" (e.g. intelligence, exam marks). These are variables. A variable is anything which is free to vary, and in order to describe them quantitatively, they have to be expressed in appropriate units (e.g. IQ scores, exam percentages)
The pairs of variables in the above examples have separate names. The variable we manipulate is called the independent variable (IV) . The variable we are hypothesising will alter as a result of our manipulations is called the dependent variable (DV) . The dependent variable alters as a consequence of the value of the independent variable - its value is dependent on this. The value of the independent variable is free to vary according to the whims of the experimenters.
Independent variable

Health education programmes

Newspaper

Attendance at lectures

Diet
Dependent variable

Number of people who smoke

Voting patterns

Exam marks

Intelligence
NB Many variables can be either dependent or independent, within the context of a particular study. For example, it could be argued that "Intelligence influences diet" or
"Exam marks influence attendance at lectures".
In the examples above, words like "influence" or "affects" are used without indicating direction. When a hypothesis states a predicted outcome (using words such as reduce, increase, lower, raise - it is called a directional or one-tailed hypothesis . Vaguer types of hypotheses (such as the ones given earlier) are known as non-directional or twotailed hypotheses .
Sumber : http://www.researchmethods.com/
I. The problem with random error
A. Plays havoc with individual scores

B. Like static or unwanted noise, hides the message that the data are sending you--but doesn't bias the results.
C. Sources of random error
1. Observers
2. Testing situations vary
3. Individuals vary (weight, mood, etc. fluctuate)
II. Assessing random error to see if it is a problem
A. The technique for assessing random error is based on one assumptions about the nature of people: underlying traits and behavior are basically stable and consistent.
B. Test-retest reliability coefficient ranges from 0-1, with higher numbers indicating higher reliability
1. What to do if it's over .90
2. What to do if it's under .60
III. Assessing random error due to the observer
A. Interjudge agreement
B. Inter-observer reliability
1. ranges from 0-1
2. puts a ceiling (but not a floor) on test-retest reliability
IV. What to do if there is too much random error due to the observer
A. Train and motivate raters
B. Use instruments
V. Dealing with random error due to non-observer sources
A. Reduce random error due to the testing situation by standardizing how the measure is administered
B. Reduce random error due to the participant by

1. Interviewing participants to find out if certain questions are confusing or hard to answer. Since participants may be guessing on those items, that guessing will lead to random error. Consequently, deleting those items may increase reliability.
2. Using more items or larger samples of behavior so that random error will have more chances to balance out. That is, scores on a multiple-choice test made up of 1 item will be more influenced by random error than a 100-item test.
VI. Summary:
A. Reliability does not guarantee validity; it is only a prerequisite for validity. A reliable measure can be consistently measuring (or being influenced by) the wrong thing, such as:
1. Another construct
2. Observer bias
B. Low reliability waters down, but does not poison, validity.
Sumber : http://www.researchmethods.com/
I. The problem with random error
A. Plays havoc with individual scores
B. Like static or unwanted noise, hides the message that the data are sending you--but doesn't bias the results.
C. Sources of random error
1. Observers
2. Testing situations vary
3. Individuals vary (weight, mood, etc. fluctuate)
II. Assessing random error to see if it is a problem
A. The technique for assessing random error is based on one assumptions about the nature of people: underlying traits and behavior are basically stable and consistent.

B. Test-retest reliability coefficient ranges from 0-1, with higher numbers indicating higher reliability
1. What to do if it's over .90
2. What to do if it's under .60
III. Assessing random error due to the observer
A. Interjudge agreement
B. Inter-observer reliability
1. ranges from 0-1
2. puts a ceiling (but not a floor) on test-retest reliability
IV. What to do if there is too much random error due to the observer
A. Train and motivate raters
B. Use instruments
V. Dealing with random error due to non-observer sources
A. Reduce random error due to the testing situation by standardizing how the measure is administered
B. Reduce random error due to the participant by
1. Interviewing participants to find out if certain questions are confusing or hard to answer. Since participants may be guessing on those items, that guessing will lead to random error. Consequently, deleting those items may increase reliability.
2. Using more items or larger samples of behavior so that random error will have more chances to balance out. That is, scores on a multiple-choice test made up of 1 item will be more influenced by random error than a 100-item test.
VI. Summary:
A. Reliability does not guarantee validity; it is only a prerequisite for validity. A reliable measure can be consistently measuring (or being influenced by) the wrong thing, such as:
1. Another construct
2. Observer bias

B. Low reliability waters down, but does not poison, validity.