Conducting and Analyzing Surveys

When conducting a survey, care must be taken so that bias is not introduced through choice of sample or through the survey questions. If a survey is used to make inferences, it is necessary to estimate how accurate the results are.

Concept

Bias in Survey Questions

Data in a survey is collected by having participants answer questions. If the questions are written so that they affect the answers, the questions are biased. To avoid this, questions should be written so that

there is no implication of there being a certain answer that is favorable or correct,
the person who answers has the necessary knowledge to answer the question, and
the participants answer truthfully even if the subject is sensitive.

The answers to survey questions can also be affected by survey fatigue, which can occur if completing the questionnaire takes a lot of time.

Correct the bias in the survey questions

fullscreen

Exercise

Identify and correct the bias in the following two survey questions.

Question in a health study. "Research shows that it is beneficiary to your health to exercise regularly. How many times a week do you exercise?"
Question in a survey about attitudes, conducted by a math teacher in a class he teaches. "Do you like mathematics?"

Show Solution

Solution

Example

Health Study

Type of bias: The question is biased because, by stating that it is a good thing to exercise, it is implied that it is better to answer with a high number than a low. Therefore, some people might answer that they exercise more than they actually do.
Correct questioning: The question can be improved by not including the reference to research. The question could then be stated like this. "How many times a week do you exercise?"

Example

Attitude Study

Type of bias: If a math teacher asks a student if he or she likes mathematics, some students might find it uncomfortable to admit that they do not.
Correct questioning: It would be better if the survey was conducted by someone who is not teaching the class. Also the question could be written so that it gives the students a way to rank different subjects against each other. The question could then look as follows.
"Rank the following school subjects by writing the numbers 1 to 4 on the line. 1 indicating your favorite subject and 4 your least favorite.
___ English
___ Mathematics
___ Chemistry
___ History"

Concept

Population Mean

When the mean value can be determined for a specific characteristic of a population, this is referred to as a population mean. For example, it is possible to measure the height of all citizens of North Dakota. The population mean would be the average height of all North Dakotans.

Concept

Population Proportions

A population is often not uniform. Some characteristics include

age,
gender,
income,
marital status, and/or
religion.

The proportion of members in the population that possess a certain characteristic is called a population proportion. When selecting a sample, it is important that the ratio of members in the sample with each relevant characteristic, the sample proportion, is the same as in the population proportion. This is accomplished by making a stratified sample based on the relevant population proportion.

Rule

Accuracy and Margin of Error

A survey only targets a sample of the population. Therefore, when using survey data to make inferences, there is always uncertainty in the results. The larger the sample is, the more reliable the results are. Margin of error is an estimate of how much the responses from the sample at most will differ from those of the population. With the sample size n, the margin of error can be approximated by this formula.

$Margin of error = \pm \frac{1}{n}$

For example, if the ratio p of the test subjects answer "yes" to a question in a survey, then a value between

p - \frac{1}{n}

and

p + \frac{1}{n}

would answer "yes" on the same question.

Calculate and interpret the margin of error

fullscreen

Exercise

The director Selma Omm has just finished the production of her latest movie, Galactic War IV. A test screening was held for high school students, the typical audience for the Galactic War movies. When asked what they thought of the character Tom Ent, 72 of 360 students said they liked him. Determine and interpret the margin of error in this survey.

Show Solution

Solution

Out of the 360 who watched the screening, 72 liked Tom Ent. We can calculate the percent this represents.

Thus, 20 % of the audience in the screening liked him. The margin of error depends on the population size, n. In this case it is the total audience in the screening, 360.

Margin of error = \pm \frac{1}{n} = \pm \frac{1}{3 6 0} \approx \pm 0.05

It can be expected that the character Tom Ent will be liked by somewhere in the interval 0.2−0.05=0.15=15 % and 0.2+0.05=0.25=25 % of the total audience of the movie.