What is Statistical Significance? Date 01/29/2019
What is statistical significance? If we randomly assignsubjects to two groups we would expect there to be some differencein the groups just by chance. If a difference is statisticallysignificant then it is large enough that we would not expect it tohappen just by chance. When we compare the results of anexperiment, if there is a statistically significant difference thenwe can conclude that there is a cause-effect relationship betweenthe explanatory variable and the response variable. In thisactivity we will explore what size difference that is due tochance. This will help us determine statistically significantdifferences.
The Experiment
Have you ever used music at work to jack up productivity orchange your mood? Interestingly some rhythms such as​ ​baroque​,induce enzymes in the brain and add amazing well being and focus.Other tunes leave you punchy ... and unable to focus. Classical​​music, such as Haydn and Mozart, often improves concentration andmemory when played in the background.
I would like to determine if listening to classical musicwhile studying improves exam scores. Twenty students volunteer toparticipate in my experiment. Outline the experiment. Use theexample shown below as a guide.
What is the explanatory variable?________________________________________________
What is the response variable?___________________________________________________
Now it is time to randomly assign the students to the twotreatments: to study while listening to classical music or to studyin a quiet environment. In order to determine the magnitude ofdifferences that happen by chance, we will assume that listening toclassical music while studying does not improve exam scores.
1. We must randomly assign the 20 students to a treatmentgroup. Use the random number table (last page) to make thisassignment. Generate 20 one-digit random numbers and indicate theline you used.
Line ​134 20 one-digit numbers ​2, 7, 8, 1, 6, 7, 8, 4, 1, 6,1, 8, 3, 2, 9, 2, 1, 3, 3, 7
If the number is odd (1, 3, 5, 7, 9), assign the student tostudy with music. If the number is even (0, 2, 4, 6, 8), assign thestudent to study with no music. As soon as you have 10 in onetreatment group, put remaining students in other group so thatthere are 10 students in each treatment group.
Student
Random Number
Group (Circle)
Student
Random Number
Group (Circle)
1
​2
Study with music
Study with no music
11
​1
S​tudy with music Study with no music
2
​7
Study with music
Study with no music
12
​8
Study with music
Study with no music
3
​8
Study with music
Study with no music
13
​3
Study with music
Study with no music
4
​1
Study with music
Study with no music
14
​2
Study with music
Study with no music
5
​6
Study with music
Study with no music
15
​9
Study with music
Study with no music
6
​7
Study with music
Study with no music
16
​2
Study with music
Study with no music
7
​8
Study with music
Study with no music
17
​1
Study with music
Study with no music
8
​4
Study with music
Study with no music
18
​3
Study with music
Study with no music
9
​1
Study with music
Study with no music
19
​3
Study with music
Study with no music
10
​6
Study with music
Study with no music
20
​7
Study with music
Study with no music
2. The students take the exam and the grades they earned arerecorded below:
3. Record the average score for the subjects in each of thetwo groups. Also calculate the
difference between these group averages (‘study with music’average minus ‘study without music’ average). Be sure to indicateif your difference is negative or positive.
Study with music: ​70.7​ Study without music: ​62.9Difference: ​7.8
4. Below I have recorded the differences determined bystudents in previous classes. Add
your difference to the results given below.
Match the comparison with the correct interpretation:
___​A​____ What does a positive difference mean? A. Thestudents who studied without music did better than the students whostudied with music.
____​B​___ What does a negative difference mean? B. Thestudents who studied with music did better than the students whostudied without music.
5. Examine the differences reported above.
Student
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Score
97
61
83
57
79
63
75
67
83
82
77
87
62
81
78
77
52
77
95
73
Study with music – study with no music
- 4.2
2.2
- 2.6
3.4
0.2
3.6
- 3.8
- 4.6
5.2
 Â
What is the biggest difference that you observe?
What is the smallest difference you observe?
What is the typical difference that you observe? Explain howyou determined this value.
6. Suppose I found that there was a difference of 3 points inthe average score of the two groups in my experiment. Do you feelthis difference is likely to happen just by chance? Explain yourreasoning.
7. Suppose I found that there was a difference of 10 points inthe average score of the two groups in my experiment. Do you feelthis difference is likely to happen just by chance? Explain yourreasoning.
8. Which of the differences discussed above (3 points and 10points) is a significance difference? Explain.
