Nathen's Miraculous Escape

Part One:  Brief Response

The main, and most important part of Chapter 13 was how to talk and think about factorial ANOVAs; the terminology, notation, calculations, and underlying logic. There were several other major points, but I thought the most important might be how to deal with unequal sample sizes—the problems created and ways to resolve them.

I found section 13.8 (new edition) to be the most confusing. I get most of the individual concepts, like sampling fractions and fixed variables, and I’m a bit confused about how random sampling can generate levels, but I didn’t put it all together: I don’t understand expected mean squares.

Part Two: Problem Set

1a. Reported level of distraction had a significant effect on number of errors made (F(1, 131) = 65.696, p < .001). However, after controlling for level of distraction, there was still a significant effect of task type on number of errors (F(2, 131) = 139.638, p < .001). That is, although level of distraction had a significant effect on the number of errors made, the type of task significantly predicted number of errors above and beyond level of distraction.

1b. The full model from last week, not including distractibility, had F(2, 132) = 113.474, p < .001 and R² = .632. When distractibility was accounted for, both the F-value and the R² increased. This is because the extra factor, distractibility, accounts for some of the variation that had previously been considered error, decreasing the SS of error from 16670.400 to 11102.535. Decreasing the error increases the power of the test.

1c. Pattern recognition had a raw mean of 9.644 and a slightly larger adjusted ean of 9.709. The cognitive task had a raw mean of 38.778 and a smaller adjusted mean of 37.236. The driving simulation had a raw mean of 6.356 and a larger adjusted mean of 7.833. The means have changed because we have taken out the effect contributed by distractibility. The raw means are just the mean number of errors per task. The adjusted means are calculated with the effect of the covariate removed, holding it at its mean, 112.54. Figure 4 shows a plot of these means.

Figure 1. Means are raw means.

Figure 2. Showing a significant p-value for distractibility and for

the omnibus test for type of task.

Figure 3. These are the adjusted means.

Figure 4. Estimated marginal means for number of errors by

task type.

2a. Distractibility significantly predicted number of errors ( = .438, t (133) = 6.355, p < .001), accounting for just over 23% of the variance in number of errors, R² = .233. That is, the more distractible participants rated themselves, the more errors they tended to make, and about 23% of the errors made can be accounted for by this relationship.

The reported above (.483) represents the change in errors made, in terms of standard deviations, associated with a one standard deviation change in distractibility. The unstandardized coefficient for distractibility, b = .418, is the change in number of errors associated with a one raw unit change in distractibility. Both of these are statistically significantly different from zero, as indicated by p < .001 (See Figure 6), which means that the variation accounted for by distractibility is significant.

Figure5.

Figure 6.

3. An augmented model including Distractibility (b = .309, t(131) = 8.105, p < .001), and the orthogonal contrasts show in Figure 7, the Pattern Recognition vs. Driving Simulation contrast (b = 9.488, t(131) = 16.695, p < .001), and the Cognitive Task vs. the average of the Pattern Recognition and Driving Simulation contrast (b = –.938, (131) = .962, p = .338), provided a reasonably good fit (R² = .755), significantly predicting variance in errors made (F(3, 131) = 134.625, p < .001). In other words, this model predicted over 75% of the variation in number of errors, which was statistically significant. Note that on the correlation table, Figure 9, the contrasts are correlated at 0, showing that they are indeed orthogonal.

3a. The unstandardized coefficient for Distractibility (.309) means that for every increase of one unit in Distractibility, participants tended to make .309 more errors. The unstandardized coefficient for the Pattern Recognition vs. Driving Simulation contrast (.938) equals half the number of errors that can be attributed to the shift between the pattern recognition task and the driving simulation task. That is, it is half of the difference between the mean number of errors in the pattern recognition task and the driving simulation task. The unstandardized coefficient for the Cognitive Task vs. the average of the Pattern Recognition and Driving Simulation contrast (9.488) is one third the number of errors that can be attributed to the shift between the cognitive task and the average of the pattern recognition and driving simulation tasks. The value under B for the constant in the Coefficients table, Figure 12, is not interpretable in the same way as that coefficient from homework 5; it can only be used to calculate the adjusted means.

3b. The number of errors in the Cognitive Task (adjusted mean = 37.24) was significantly higher than number of errors in the Pattern Recognition and Driving Simulation tasks, even after accounting for the effects of distractibility (b = 9.488, t(131) = 16.696, p < .001). Number of errors in the Pattern Recognition task (adjusted mean = 9.71) was not significantly different from number of errors in the Driving Simulation task (adjusted mean = 7.83) after controlling for the effect of distractibility (b = -.938, t(131) = .962, p = .338). That is, after controlling for distractibility, the number of errors in the Cognitive Task is significantly higher than the Pattern Recognition and Driving Simulation tasks, and the Pattern Recognition and Driving Simulation tasks are not significantly different from each other.

The p-value for pattern vs. driving, p < .338, means that the coefficient for that contrast is not significantly different from zero. The p-value for cognitive vs. pattern and driving, p < .001, means that the coefficient for that contrast, 9.488, is significantly different from zero, which tells us if the comparison coded into that contrast is significant or not.

3c. These results are similar to what the ANCOVA gave us except that they give us more insight into what’s going on between the tasks. The adjusted means for the tasks are the same, R² is the same, and the F values for the whole models are the same (134.625)  the current t-value for distractibility (squared) equals the F from the ANCOVA; 8.105² = 65.696. The differences are that the ANCOVA is an omnibus test when it comes to the differences between the task types, so we didn’t tell exactly where the differences are until we do the regression. The degrees of freedom are also different.

Figure 7.    Figure 8.

Figure 9.

Figure 10.

Figure 11.

Figure 12.

4a. Testing the reduced model (just distractibility, from question 2, or see model 1 in Figure 12) against the full model (distractibility plus the Cognitive Task vs. the average of the Pattern Recognition and the Driving Simulation contrast, and the Pattern Recognition vs. Driving Simulation contrast (model 2 in Figure 12)), we can conclude that the two contrasts explain a significant amount of variance that is not accounted for by distractibility, ΔR² = .522, ΔF(2, 131) = 139.638, p < .001. That is, adding the two contrasts to our model explains a significantly larger amount of the variation in number of errors made, above and beyond that accounted for by distractibility.

5a. The Homogeneity of Regression assumption is not met in this data set. The model summary table above shows a significant change in F (p < .001) when the covariate interactions are added to the model. The groups who did each task differed significantly on their levels of distractibility, so distractibility cannot be legitimately covaried out; it would be unwise to use ANCOVA here.

5b. The tolerance and VIF statistics for model 2 (shown in Figure 16) are all problematic, except for the statistics for distractibility. Those for model 1 look fine. The correlation table, Figure 13, shows only two problematic correlations; each contrast is highly correlated with its respective covariate interaction. This is to be expected, since the contrasts are factors of the interactions. Looking at the data set, two of the distance values are over three, but all of the Cook’s D and leverage values look fine. Overall I would say that collinearity is not much of a problem here, except that groups are not supposed to differ on the covariate, and some of the adjusted means were higher, some lower than their raw means, suggesting that the groups were different on the covariate.

5c. The results of 5a would probably have kept me from running an ANCOVA in the first place and, having done so, I would not be confident in the results. I would also be somewhat cautious interpreting the results based on 5b, at least until I had talked to someone who knew more than I do about collinearity. (See pages 11-13 for Figures 13-17.)

Figure 13.

Figure 14.

Figure 15.

Figure 16.

Figure 17.