17|RELATED-SAMPLES ANOVA

Overview

• Logic
• Calculations
• Post-hoc tests
• Learning checks

Logic

Independent-samples

• Independent-measures ANOVA
  • “Between-participants”
  • Different participants in each treatment
  • Between-groups variability could be due to treatment effect and/or individual differences

Related samples

• Repeated-measures (within-participants)
  • Same participants in each treatment
• Or matched-samples
  • Matched participants in each treatment

Related samples

• Advantages
  • Eliminates individual differences as a source of variability between treatments
  • Smaller number of participants needed
• Disadvantages
  • Something other than the treatment may cause participant’s scores to change
  • E.g. practice, world events, natural improvement

Partitioning variance

• Independent samples

Total variance

Variance between treatments
\(MS_{between}\)

• Treatment effect
• Sampling error
• Individual differences

Variance within groups
\(MS_{within}\)

• Sampling error
• Individual differences

\(F = \dfrac{{MS_{between}}}{{MS_{within}}} = \dfrac{treatment \cdot error \cdot ind. \ diff.}{error \cdot ind. \ diff.}\)

Partitioning variance

• Related samples: problem

Total variance

Variance between treatments
\(MS_{between \ treatments}\)

• Treatment effect
• Sampling error

Variance within groups
\(MS_{within}\)

• Sampling error
• Individual differences

\(F = \dfrac{{MS_{between \ treatments}}}{{MS_{within}}} = \dfrac{treatment \cdot error}{error \cdot ind. \ diff.}\)

Partitioning variance

• Related samples: solution

Total variance

Variance between treatments
\(MS_{between \ treatments}\)

• Treatment effect
• Sampling error

Variance within groups
\(MS_{within}\)

Error
\(MS_{error}\)

Between subjects
\(MS_{between \ S's}\)

• Sampling error

• Individual differences

\(F = \dfrac{{MS_{between \ treatments}}}{{MS_{error}}} = \dfrac{treatment \cdot error}{error}\)

Calculations

Calculations: \(SS\) and \(df\)

\(SS_{total}\)

\(SS_{btwn \ T's}\) \(SS_{within}\)

\(SS_{error}\) \(SS_{btwn \ S's}\)

\(df_{total}\)

\(df_{btwn \ T's}\) \(df_{within}\)

\(df_{error}\) \(df_{btwn \ S's}\)

\(SS_{total} = \Sigma X^2 - \dfrac{G^2}{N}\)

\(SS_{within} = \Sigma SS_{within \ each \ treatment}\)

\(SS_{between \ treatments} = \Sigma \dfrac{T^2}{n} - \dfrac{G^2}{N}\)

\(SS_{between \ subjects} = \Sigma \dfrac{P^2}{k} - \dfrac{G^2}{N}\)

\(SS_{error} = SS_{within}-SS_{between \ subjects}\)

\(df_{total} = N-1\)

\(df_{within} = N-k\)

\(df_{between \ treatments} = k-1\)

\(df_{between \ subjects} = n-1\)

\(df_{error} = df_{within}-df_{between \ subjects}\)

Calculations: \(MS\) and \(F\)

• Step 2. \(MS\) values

\[\begin{align} MS_{between \ treatments} &= \dfrac{SS_{between \ treatments}}{df_{between \ treatments}} \\ \ \\ MS_{error} &= \dfrac{SS_{error}}{df_{error}} \end{align}\]

• Step 3. \(F\)-ratio

\(F = \dfrac{MS_{between \ treatments}}{MS_{error}}\)

Summary table

Source	\(SS\)	\(df\)	\(MS\)	\(F\)
Between treatments
Within treatments
Between subjects
Error
Total

Hypothesis test

Step 1. State hypotheses

	Manipulation
Person	🍌 Banana	🍬 Candy	😐 Control
A	9	3	5
B	11	5	6
C	13	4	7

• \(H_0: \mu_1 = \mu_2 = \mu_3\)
• \(H_1\) : At least one population mean differs from another

Step 2. Critical region

• Numerator: \(df_{between \ treatments} = k-1\)
• Denominator: \(df_{error} = df_{within}-df_{between \ S's} = (N-k)-(n-1)\)

\(\alpha = .05\)	\(df_{numerator}\)
\(df_{denominator}\)	1	2	3	4	5	6	7	8	9	10
1	161.45	199.50	215.71	224.58	230.16	233.99	236.77	238.88	240.54	241.88
2	18.51	19.00	19.16	19.25	19.30	19.33	19.35	19.37	19.39	19.40
3	10.13	9.55	9.28	9.12	9.01	8.94	8.89	8.85	8.81	8.79
4	7.71	6.94	6.59	6.39	6.26	6.16	6.09	6.04	6.00	5.96
5	6.61	5.79	5.41	5.19	5.05	4.95	4.88	4.82	4.77	4.74
6	5.99	5.14	4.76	4.53	4.39	4.28	4.21	4.15	4.10	4.06
7	5.59	4.74	4.35	4.12	3.97	3.87	3.79	3.73	3.68	3.64
8	5.32	4.46	4.07	3.84	3.69	3.58	3.50	3.44	3.39	3.35
9	5.12	4.26	3.86	3.63	3.48	3.37	3.29	3.23	3.18	3.14
10	4.96	4.10	3.71	3.48	3.33	3.22	3.13	3.07	3.02	2.98

3. Calculate \(F\)-ratio

	Manipulation
Person	🍌 Banana	🍬 Candy	😐 Control	P
A	9	3	5	17
B	11	5	6	22
C	13	4	7	24
\(M\)	11	4	6
\(T\)	33	12	18
\(SS\)	8	2	2

\(n = 3\)
\(k = 3\)
\(N = 9\)
\(G = 63\)
\(\Sigma X^2 = 531\)

\[\begin{align} df_{total} &= N-1 = 8 \\ df_{within} &= N-k = 6 \\ df_{between \ treatments} &= k-1 = 2 \\ df_{between \ subjects} &= n-1 = 2\\ df_{error} &= df_{within}-df_{between \ subjects} = 4 \end{align}\]

3. Calculate \(F\)-ratio

	Manipulation
Person	🍌 Banana	🍬 Candy	😐 Control	P
A	9	3	5	17
B	11	5	6	22
C	13	4	7	24
\(M\)	11	4	6
\(T\)	33	12	18
\(SS\)	8	2	2

\(n = 3\)
\(k = 3\)
\(N = 9\)
\(G = 63\)
\(\Sigma X^2 = 531\)

\(SS_{total} = \Sigma X^2 - \dfrac{G^2}{N} = 90\)

\(SS_{within} = \Sigma SS_{within \ each \ treatment} = 12\)

\(SS_{between \ treatments} = \Sigma \dfrac{T^2}{n} - \dfrac{G^2}{N} = 78\)

\(SS_{between \ subjects} = \Sigma \dfrac{P^2}{k} - \dfrac{G^2}{N} = 8.67\)

\(SS_{error} = \Sigma SS_{within}-SS_{between \ subjects} = 3.33\)

3. Calculate \(F\)-ratio

	Manipulation
Person	🍌 Banana	🍬 Candy	😐 Control	P
A	9	3	5	17
B	11	5	6	22
C	13	4	7	24
\(M\)	11	4	6
\(T\)	33	12	18
\(SS\)	8	2	2

\(n = 3\)
\(k = 3\)
\(N = 9\)
\(G = 63\)
\(\Sigma X^2 = 531\)

\(MS_{between \ treatments} = \dfrac{SS_{between \ treatments}}{df_{between \ treatments}} = 39\)

\(MS_{error} = \dfrac{SS_{error}}{df_{error}} = 0.83\)

3. Calculate \(F\)-ratio

	Manipulation
Person	🍌 Banana	🍬 Candy	😐 Control	P
A	9	3	5	17
B	11	5	6	22
C	13	4	7	24
\(M\)	11	4	6
\(T\)	33	12	18
\(SS\)	8	2	2

\(n = 3\)
\(k = 3\)
\(N = 9\)
\(G = 63\)
\(\Sigma X^2 = 531\)

\(F = \dfrac{MS_{between \ treatments}}{MS_{error}} = 46.8\)

Step 4. Decision

• Step 4a. Significance
  • \(F > F_{critical}\)?
• Step 4b. Effect size

\[\begin{align} \eta^2_{partial} &= \dfrac{SS_{between \ treatments}}{SS_{total} - SS_{between \ subjects}} \\ &= \dfrac{SS_{between \ treatments}}{SS_{between \ treatments} + SS_{error}} \\ &= 0.96 \end{align}\]

Post-hoc tests

• Step 4c: Post-hoc tests
  • Tukey’s Honestly Significant Difference
  • Minimum difference between pairs of treatment means so that \(p < \alpha_{experimentwise}\)

\[\begin{align} HSD &= q \sqrt{\dfrac{MS_{denominator}}{n}} \\ &=5.04 \sqrt{\dfrac{0.83}{3}} \\ &= 2.66 \end{align}\]

	Number of conditions
\(df\)	2	3	4	5	6	7
2	6.08	8.33	9.80	10.88	11.73	12.44
3	4.50	5.91	6.83	7.50	8.04	8.48
4	3.93	5.04	5.76	6.29	6.71	7.05
5	3.63	4.60	5.22	5.67	6.03	6.33
6	3.46	4.34	4.90	5.30	5.63	5.89
7	3.34	4.17	4.68	5.06	5.36	5.61
8	3.26	4.04	4.53	4.89	5.17	5.40

Report results

A single-factor, related-samples ANOVA revealed a significant difference in people’s test scores when the test was preceded by consumption of a banana (\(M = 11.00\); \(SD = 2.00\)), a candy bar (\(M = 4.00\); \(SD = 1.00\)), and no snack (\(M = 6.00\); \(SD = 1.00\)); \(F(2,4) = 46.8\), \(p < .05\), \(\eta^2 = 0.96\). Post-hoc tests using Tukey’s HSD revealed that test scores were significantly better following banana consumption than following no snack or candy consumption; the candy did not differ significantly from the control condition.

Learning checks

1. A researcher obtains an \(F\)-ratio with \(df = 2, 12\) in an independent-samples study ANOVA
   • How many levels of the IV were there?
   • How many subjects participated in the study?
     
     
2. A researcher obtains an \(F\)-ratio with \(df = 2, 12\) in a repeated-measures study ANOVA
   • How many levels of the IV were there?
   • How many subjects participated in the study?

w = 500
h = 550
r = 20
fill = d3.scaleOrdinal(d3.schemeSet3)

makeSample = function () {
    var arr = [];
    for (var i = 0; i < 10; i++) {
      arr.push(Math.floor(Math.random()*10))
    }
    return arr;
}

data = ["A","B","C"]
  
independentSamplesFigure = {

  const svg = d3.select("#independent-samples-container")
    .append("svg")
    .attr("viewBox", "0 0 " + w + " " + h)
    .attr("preserveAspectRatio", "xMinYMin meet")

  const labels = svg.append("g")
    .style("font-size", "20px")
    .style("font-weight", "bold")
    .attr("class", "invertable")
    
  labels.selectAll("text")
    .data(data).enter().append("text")
    .attr("x", (d,i) => w/4 + i * w/data.length).attr("y", 20)
    .text(d => "SAMPLE " + d).attr("text-anchor", "middle")

  const sampleDots = svg.append("g")
  
  const g = sampleDots.selectAll("g").data(data).enter().append("g").attr("transform", (d,i) => `translate(${w/4 + i * w/data.length}, 0)`).style("stroke", "black")
  
  g.each(function(d,i){
    d3.select(this)
    .selectAll("circle")
      .data(makeSample()).enter().append("circle")
        .attr("cy", (d,i) => 60 + i*r*2.5)
        .attr("r", r)
        .style("fill", d => fill(d))
  })
      
}


relatedSamplesFigure = {

  const svg = d3.selectAll("#related-samples-container")
    .append("svg")
    .attr("viewBox", "0 0 " + w + " " + h)
    .attr("preserveAspectRatio", "xMinYMin meet")

  const labels = svg.append("g")
    .style("font-size", "20px")
    .style("font-weight", "bold")
    .attr("class", "invertable")
    
  labels.selectAll("text")
    .data(data).enter().append("text")
    .attr("x", (d,i) => w/4 + i * w/data.length).attr("y", 20)
    .text(d => "SAMPLE " + d).attr("text-anchor", "middle")
    
  const lines = svg.append("g").attr("transform", `translate(0, 60)`).style("stroke", "black")
  .attr("class", "invertable")
    .selectAll("line").data([0,1,2,3,4,5,6,7,8,9]).enter()
    .append("line")
      .attr("x1", w/4 + r * 1.5)
      .attr("x2", w/4 + 2 * w/data.length - r*1.5)
      .attr("y1", (d,i) => i * r * 2.5)
      .attr("y2", (d,i) => i * r * 2.5)

  const sampleDots = svg.append("g")
  const sample = makeSample()
  
  const g = sampleDots.selectAll("g").data(data).enter().append("g").attr("transform", (d,i) => `translate(${w/4 + i * w/data.length}, 0)`).style("stroke", "black")
  
  g.each(function(d,i){
    d3.select(this)
    .selectAll("circle")
      .data(sample).enter().append("circle")
        .attr("cy", (d,i) => 60 + i*r*2.5)
        .attr("r", r)
        .style("fill", d => fill(d))
  })
      
}

matchedSamplesFigure = {

   function makeSample () {
    var arr = [];
    for (var i = 0; i < 10; i++) {
      arr.push(Math.floor(Math.random()*10))
    }
    return arr;
  }

  const svg = d3.selectAll("#matched-samples-container")
    .append("svg")
    .attr("viewBox", "0 0 " + w + " " + h)
    .attr("preserveAspectRatio", "xMinYMin meet")

  const labels = svg.append("g")
    .style("font-size", "20px")
    .style("font-weight", "bold")
    .attr("class", "invertable")
    
  labels.selectAll("text")
    .data(["A","B"]).enter().append("text")
    .attr("x", (d,i) => w/4 + i * w/2).attr("y", 20)
    .text(d => "SAMPLE " + d).attr("text-anchor", "middle")
    
  const lines = svg.append("g").attr("transform", `translate(${w/4 + r * 1.5}, 60)`)
    .style("stroke", "black").style("stroke-dasharray", [10,10]).attr("class", "invertable")
    .selectAll("line").data([0,1,2,3,4,5,6,7,8,9]).enter()
    .append("line")
      .attr("x2", w/2 - 2 * r * 1.5)
      .attr("y1", (d,i) => i * r * 2.5)
      .attr("y2", (d,i) => i * r * 2.5)

  const sampleDots = svg.append("g")
  const sample = makeSample()
  
  const g = sampleDots.selectAll("g").data([0,1]).enter().append("g").attr("transform", (d,i) => `translate(${w/4 + i * w/2}, 0)`).style("stroke", "black")
  
  g.each(function(d,i){
    d3.select(this)
    .selectAll("circle")
      .data(sample).enter().append("circle")
        .attr("cy", (d,i) => 60 + i*r*2.5)
        .attr("r", r)
        .style("fill", d => fill(d))
  })

}