Goals of this unit:

1)  Students will read, understand, and interpret graphical displays of data in published studies.
2)  Students will understand the concepts of statistical and practical significances.
3)  Students will determine whether results in published studies are statistically significant.
4)  Students will determine whether results in published studies are practically significant.

Unit 4 Course Notes

---

XIII.  Types of Data

A researcher will collect data based on his/her research questions or hypotheses.  The type(s) of data collected depends entirely on the purpose and nature of the research study.  When conducting a study, a researcher may collect several different types of data.

Consider this example:  I collected data at Paulding County High School while working on a project about writing achievement.  Students were asked to complete a survey about their reading and writing habits, were given the MBTI to measure psychological traits, wrote an essay, and were given the Woodcock-Johnson (Revised) test of fluid reasoning ability.

On the survey, students provided responses to these types of questions:
 

Responses to the questions above are examples of categorical data.  Categorical data include any data that fit into discrete categories.  Another type of data is quantitative (also called continuous) data.  Quantitative data include data that are numerical.  If I had asked the question, "How many books did you read last month?"  the response would be classified as quantitative data.

There are two subcategories of categorial data and two subcategories of quantitative data:
 
 

Nominal:  Data exist in discrete categories.  Examples are gender (a person is either in the category of male or female), employment (a person is classified as employed or unemployed), voting preference (a person is classified as either democrat, republican, liberal, independent, green, etc.). Question 5 on the example survey above provides nominal data.  Classification of students on the MBTI (are individuals extroverted or introverted?) is an example of nominal data.

Ordinal:  Data exist in categories that can be ordered.  For example, if students are classified by ability, ability is an exmaple of ordinal data.  We measure student ability, then label them high, average, or low ability (see question 6 on the example survey above).  These are three categories of ability, but they have order (high to low).  Other examples of ordinal data are Likert scale response and rubric scores.  When individuals complete a Likert scale, like the one used on the end-of-course evaluation, they are typically asked their level of agreement/disagreement with a particular statement (or they may be asked how often they exhibit a certain behavior--frequently, sometimes, never).  Questions 1 and 3 on the survey above are examples of Likert scale items.  Rubric scores are another example of ordinal data.  When scoring the essay in my writing study, students received a score of 1 to 10 based on how well they met certain criteria that were deemed important (flow of the essay, proper sentence structure, writing for the appropriate audience, etc.).  Scores on the essay did not stand for an amount of correct responses or percentage correct.  Instead the scores represented the quality of a response.

Click here for a GREAT website on rubrics.  This is a great teacher source!
 

Interval:  Data are numerical but exist on a scale where zero does not mean an absence of something.  Test score is an example of interval data.  A score of zero on a test does not mean an absence of any knowledge in the area being tested (it would be virtually impossible to come to the conclusion that an individual knows NOTHING about a certain topic...imagine how many questions we'd have to ask to be able to make that kind of assessment!), it simply means the individual did not get any questions correct.  On the interval scale, zero does not mean an absence of something.  In my writing study, score on the Woodcock Johnson was an example of interval level data.

Ratio:  Data are numerical and zero means an absence of whatever is being measured.  Number of tardies is an example of ratio data.  Zero would mean there were NO tardies.  If I walked into a bar and asked for a show of hands of all those who were designated drivers that night, this would be an example of ratio data (zero means that there would be no designated drivers that night).  It is easy to get confused when you are thinking about ratio data.  Sometimes students say things like, "But it can't be ratio data because three people said they were designated drivers!"  Doesn't matter how many people we place into the category of "designated driver."  IF (notice that 'if' is capitalized) we asked for a show of hands and nobody said they were designated drivers, we would have zero designated drivers, meaning an absence of them.  Here's another example:  heart rate is an example of ratio data.  Yes, an individual would have to be dead to have zero heart beats per minute, but zero has a meaning on the heart rate scale.  It means NO heart rate.  In the example survey above, if I had asked students how many books they read in a month, this would be an example of ratio data.

It is important to be able to determine what types of data are being collected and analyzed in a study.  Different types of data require different types of analysis.

Once data have been collected, a researcher may choose to graphically display the results.  This is especially true in studies where collected data are categorical.  A researcher may choose to explain what was indicated by the data using paragraph form, but graphical displays can be very helpful to the reader.  Providing a visual representation makes understanding the results much easier.

Here are some examples of how data might be graphically displayed:
 

Percentage of students at or above the writing achievement levels: 1998

SOURCE: U.S. Department of Education, National Center for Education Statistics,
National Assessment of Educational Progress (NAEP) 1998Writing Assessment.
(Previously published on p. 10 of the NAEP1998 Writing Report Card Highlights.)

Notice here that categorical data are being displayed.  Here we are looking at the percentage of students in grades 4, 8, and 12 that are in different ORDINAL categories of writing achievement (below basic, at or above basic, at or above proficient, or advanced).
--------------------------------------------------------------------------------------------------------

Status dropout rates and number and distribution of 16- through 24-year-olds who were dropouts,
by background characteristics: October 1998

¹Due to relatively small sample sizes, American Indians/Alaska Natives are included
  in the total but are not shown separately.

²Individuals defined as "first generation" were born in the 50 states or the District of
 Columbia, and one or both of their parents were born outside the 50 states and the
 District of Columbia.

³Individuals defined as "second generation or more" were born in the 50 states or the
District of Columbia, as were both of their parents.

NOTE: Because of rounding, detail may not add to totals.

SOURCE: U.S. Department of Commerce, Bureau of the Census, Current Population
Survey (CPS), October 1998. (Originally published as table 3 on p. 13 of the complete report from which this article is excerpted.)

Once again we are looking at categorical data.  This time, all data are nominal (gender, ethnicity, region) except for age.
--------------------------------------------------------------------------------------------------------

Here is a graphical display from USAToday.  This represents nominal data.  Think about the survey or questionnaire that was used to collect this information.  It's likely that smokers were asked these questions:

Do you think smoking is addictive?  Yes/No
Do you think that cigarette makers know that smoking causes cancer?  Yes/No
Do you think that smoking causes cancer?  Yes/No
Do you think that cigarette makers aim some ads at teens?  Yes/No

--------------------------------------------------------------------------------------------------------

This pie chart shows students passing (green) and failing (red) a graduation test.

This is an example of a frequency polygon.  It shows how test scores are distributed for a group of students.  Notice that almost 50 students scored an 80.  Only about 4 students scored a 50.
 
 
 
 
 
 
 
 
 
 
 

This is an example of a contingency table.  This table shows the number of individuals in each of two groups (85 people were in the experimental group and 82 people were in the treatment group) who graduated or failed to graduate.  You can see that more students in the experimental group (73) graduated.  Only 43 students in the control group graduated.
 
 
 
 

This is a scatterplot, which is used with correlational research.  This scatterplot displays the relationship between spatial ability and intelligence.  Each point represents an individual.  The circled point represents a child with a score of 10 on the spatial ability test and a 28 on the intelligence test.
 
 
 
 
 
 
 
 

This is a bar graph showing differences in scores on various tests among three ethnic groups.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

This is a histogram.  It shows the salaries for graduates with special education majors (bachelor's degree only).
 
 
 
 
 
 
 
 
 
 
 
 
 
 

In the journal articles that you read, you may see quantitative data displayed in these ways:
 
 

Group	M	SD	N	Sig.
Treatment	92.3	12.24	30
Control	76.3	14.61	32	p = .002

This table gives means, standard deviations, and number of participants for a treatment group and a control group.  The column labeled Sig. indicates whether the difference between the groups is significant (more about that when we get to inferential statistics).

This is an interaction graph.  It shows the way in which one independent variable interacts with another indepedent variable.
 
 
 
 
 
 
 
 
 
 
 
 

If, for example, the purposes of a study were:

1)  To describe the reading interests of a group of high school students; and

2)  To compare writing achievement of students labled avid readers, readers, and nonreaders.

We might begin by creating some graphical displays based on the responses to a survey intended to determine reading interests.  One of those graphical displays might look like this:

Next, we would want some general information about the writing achievement scores of the participants.  We could do that using a table like this:

Table 1

Group	M	SD	Sig.
Avid Reader	88.25	12.55
Reader	79.00	14.61
Nonreader	61.25	22.38	p = .002

This table provides means and standard deviations on the writing achievement assessment for student in the three groups.  These are descriptive statistics.

• central tendency
• variation
• distributional shape

• Mean is the average of a group of scores.  It is calculated by adding all the scores for a group and dividing by the number of people in the group.  For example, here are test scores for a group of students:

To get the mean, I add all the scores: 88+91+65+100+50+78+63+51+91+83+97+60+72+80 = 1069.
Next, I divide by the number of scores (14) =  1069/14 = 76.35714286.  Always round to two decimal places.
The mean (M) = 76.36.  Sometimes the mean will be noted using the symbol x-bar (this is an x with a bar over it).
 
 

• Median is the score that is physically in the middle of a group of ordered scores.  To get the median of the scores listed above, first we need to put them in order, then we find the middle point:

There isn't a number that's in the middle of these numbers because there is an even number of scores.  The middle point is between 78 and 80.  The midway point between 78 and 80 is 79.
The median (Md) = 79.

Click here for more information on median.
 

• Mode is the most frequently occuring score.  In our example data, the mode is 91 (it occurs twice).

A researcher must decide whether to describe data using either the mean or median (mode is almost never used to describe data).  The mean can be greatly affected by outliers (extreme scores), so if they are present, the median should be used.  Look at these two data sets:
 
 

Data Set 1	Data Set 2
65	20
68	68
70	70
72	72
75	75
76	76
89	89
90	90
M = 75.63	M = 70.00

These two data sets differ by only one score.  Notice how the mean of the second data set is lowered by 5 points.  The median is not affected however.  The mean for Data Set 1 = 73.5.  The mean for Data Set 2 is also 73.5.  The median is not affected by the extreme score.

Variation can be described in terms of range and standard deviation.
 

• Range.  Range is calculated by subracting the lowest score from the highest score.  In the table above, the range of scores in Data Set 1 = 25 (90 - 65 = 25).  The range for Data Set 2 = 70 (90 - 20 = 70).  Notice that range, like mean, can be greatly influenced by extreme scores.

• Standard Deviation.  Standard deviation is less affected by extreme scores than range.  Standard deviation is the average distance from the mean that a group of scores are.  If standard deviation is 14 points, it means that on average, students are scoring about 14 points from the mean.

John, who scored a 70, tells you his score is 10 points different.
Samala, who scored an 80, tells you her score is not different from the average.
Lanna, who scored a 100, tells you her score is 20 points different.

...and so on.

After all students told you how different their scores were from the mean, you average these differences.  This average is the standard deviation.  The actual calculation for standard deviation is a bit different than this simple explanation, but this is how standard deviation works.

Click here to learn how to calculate standard deviation.

This histogram represents scores that are positively skewed.  Eight students scored a 60, 6 students scored a 66, 2 students scored a 70, 2 students scored 75, and 1 student earned an 80.  Most scores are at the low end of the distribution, which makes this a positively scored distribution.
 
 
 
 
 
 
 
 
 
 
 

This histogram represents scores that are negatively skewed.  Two students scored 60, 2 students scored 70, but most students scored 80 or 90.  Most scores are at the high end of the distribution, which makes this a negatively skewed distribution.
 
 
 
 
 
 
 
 
 
 
 
 
 

Scores are normally distributed here.  Most scores are in the middle, with just a few extremely low and high scores on the ends.  When a distribution is symettrical (you can fold it in the middle and it is equal on both sides), it is normal.  Also, when the mean, median, and mode are the same number (notice here the mean, median, and mode are all 80), the distribution is normal.
 
 
 
 
 
 
 
 
 
 
 
 

When distributions are skewed, as in the first two examples, the median is a better estimate of central tendency than the mean.  Notice that the mode is closer to the true center of the distribution than the mean.

Descriptive statistics are used to describe data in our sample, but so far all we know how to do is display data in graph or table form and give measures of central tendency, variablity, and distributional shape.  We still do not know how to determine if groups are different from one another.

Inferential statistics allow us to make determinations about whether groups are significantly different from each other.  Consider this hypothesis:

Students who receive phonics instruction will have higher reading comprehension scores than students who receive whole language instruction.

Let's say that we have looked at reading comprehension scores for students in both instructional groups (following 18 weeks of instruction).  Students in the phonics group have a mean score of 87 and students in the whole language group have a mean score of 85.  Is this two point difference significant?  What if the difference had been 20 points?  Is that large difference significant?

There's no way to simply look at scores and determine whether they are significantly different.  But using inferential statistics can help us make this determination.  Although we might be tempted to say "I believe a 20 point difference is significant because it's big!", we cannot determine statistical significance until we've used inferential statistics.

Inferential statistics is based on a strange and mystical concept called falsification.  Although you might think the process is simple--Write a hypothesis, test it, hope to prove it--inferential statistics works this way:
 

Write a hypothesis you believe to be true.
Write the OPPOSITE of this hypothesis, which is called the null hypothesis.
Test the null, hoping to reject it.
If the null is rejected, you have evidence that the hypothesis you believe to be true may be true.
If the null is not rejected, reach no conclusion.

Inferential statistics is not a system of magic and trick mirrors.  Inferential statistics are based on the concepts of probability (what is likely to occur) and the idea that data distribute normally.

XVII.  Practical Significance