Chapter 9 Mixing categorical and metric predictors (ANCOVA)
In this chapter, we will consider combining categorical and metric independent variables to predict a dependent variable. Such an analysis is generally called an Analysis of Covariance (ANCOVA). There are a number of reasons why you might want to include metric predictors (or “covariates”) in an ANOVA-type analysis: to statistically control for pre-existing differences between conditions of potential confounders, to reduce prediction error and increase the power of statistical tests, and because they are theoretically interesting as for instance potential mediators of the effect of experimental manipulations. In the course of this chapter, we will see examples of each of these. We will also discuss the interpretation of contrasts in ANCOVA models as reflecting differences between group means which are adjusted for the effect of the covariate.
9.1 Subjective feelings of power and priming
To illustrate the models, we will consider the data from Gilder & Heerey (2018) again. As part of the experiment, the researchers measured participants’ subjective feeling of power, both before and after providing them with the power priming task. In the priming task, participants were presented with randomly scrambled sentences, and they were asked to reorder the words into grammatically correct sentences. In the high-power prime condition, half of the sentences included words associated with high power (e.g., “dominates,” “commands”) and the other half were neutral words, while in the low-power prime condition, half the sentences contained words associated with low power (e.g., “subordinate,” “obeys”). If the priming task had the desired effect, then participants in the high-power condition should report feeling more power after the priming task than participants in the low-power condition. Before the administration of the power priming task, we would expect no difference in subjective feeling of power. Figure 9.1 shows the means of the subjective feeling of power before and after the power priming manipulation. As can be seen there, the task indeed appears to have the desired effect. Before the priming task, differences between the groups are relatively small and there appears to be no real difference between the priming conditions. After the priming task, the averages in the low-power prime conditions are lower and those in the high-power prime conditions higher than before the priming task.
Figure 9.1: Means and standard errors of subjective feeling of power before (pre-test) and after (post-test) power prime.
To determine whether the differences between the conditions after the power prime manipulation are significant, we can use the by now hopefully familiar procedure of constructing appropriate contrast codes for priming condition and experimenter belief. To code for prime condition, it makes sense to assign a value of \(\tfrac{1}{2}\) to the high-power prime condition and a value of \(-\tfrac{1}{2}\) to the lower-power prime condition. As one would expect the high-power prime to increase the subjective feeling of power, one would then expect a positive slope for the associated contrast-coded predictor. Similarly, it makes sense to assign a value of \(\tfrac{1}{2}\) to the “experimenter believes high-power” condition, and a value of \(-\tfrac{1}{2}\) to the “experimenter believes low-power” condition. If experimenter belief has an effect on feelings of power, one would expect participants’ subjective feeling of power to be higher when the experimenter believes the participant was given a high-power prime. This form of contrast coding turns that into an expectation of a positive slope of the corresponding contrast-coded predictor. The contrast code for the interaction is constructed as usual by multiplying the values of these two contrast codes. So our full set of (orthogonal) contrast codes is:
| \(\texttt{P}\) | \(\texttt{B}\) | \(\texttt{P}\times\texttt{B}\) | |
|---|---|---|---|
| PL,EL | \(-\tfrac{1}{2}\) | \(-\tfrac{1}{2}\) | \(\tfrac{1}{4}\) |
| PL,EH | \(-\tfrac{1}{2}\) | \(\tfrac{1}{2}\) | \(-\tfrac{1}{4}\) |
| PH,EL | \(\tfrac{1}{2}\) | \(-\tfrac{1}{2}\) | \(-\tfrac{1}{4}\) |
| PH,EH | \(\tfrac{1}{2}\) | \(\tfrac{1}{2}\) | \(\tfrac{1}{4}\) |
We can then estimate a linear model predicting \(\texttt{post-power}\), the subjective feeling of power after the priming manipulation, as: \[\begin{equation} \texttt{post-power}_i = \beta_0 + \beta_\text{P} \times \texttt{P}_i + \beta_\text{B} \times \texttt{B}_i + \beta_{\text{P}\times \text{B}} \times (\texttt{P} \times \texttt{B})_i + \epsilon_i \tag{9.1} \end{equation}\] The estimates and hypothesis tests of the parameters of this model are given in Table 9.1.
| \(\hat{\beta}\) | \(\text{SS}\) | \(\text{df}\) | \(F\) | \(p(\geq \lvert F \rvert)\) | |
|---|---|---|---|---|---|
| Intercept | 57.32 | 1.31e+06 | 1 | 5431.58 | 0.000 |
| \(\texttt{P}\) | 3.87 | 1.50e+03 | 1 | 6.21 | 0.013 |
| \(\texttt{B}\) | -0.01 | 1.00e-02 | 1 | 0.00 | 0.995 |
| \(\texttt{P} \times \texttt{B}\) | 3.80 | 3.61e+02 | 1 | 1.49 | 0.222 |
| Error | 9.58e+04 | 396 |
The results show a significant effect of power prime (\(\texttt{P}\)). The slope is estimated as \(\hat{\beta}_\text{P} = 3.87\). Remember that the slope of a contrast-coded predictor represents a difference between marginal means; in this case, we are comparing the marginal mean of conditions with a high-power prime to conditions with a low-power prime (averaging over the levels of experimenter Belief): \[ \begin{aligned} \hat{\beta}_\text{P} &= \hat{\mu}_{\text{PH},\cdot} - \hat{\mu}_{\text{PL},\cdot} \\ &= \frac{\overline{Y}_\text{PH,EL} + \overline{Y}_\text{PH,EH}}{2} - \frac{\overline{Y}_\text{PL,EL} + \overline{Y}_\text{PL,EH}}{2} \\ &= 3.87 \end{aligned} \] Hence, the power prime manipulation appears to result in a difference of 3.87 points in subjective feeling of power. Neither the main effect of experimenter belief, nor the interaction between Power Prime and Experimenter Belief are significant. We may thus conclude that experimenter belief has no effect on participants’ subjective feeling of power, neither directly nor by moderating the effect of power prime.
While the results are indicative of a successful manipulation of participants’ subjective feeling of power, it could be the case that these differences between the conditions were already present before the priming manipulation. While random assignment of participants to the conditions makes such pre-existing differences unlikely, they cannot be ruled out a priori. As we also have a measure of participants’ feeling of power before the priming task, we can test whether there is evidence for such pre-existing differences. This can be done with a similar model as before, but now using \(\texttt{pre-power}\) as the dependent variable. The results are provided in Table 9.2. As only the intercept is significant (which tells us, rather uninterestingly, that the grand mean of subjectively felt power differs from 0), we have no evidence for pre-existing differences, which strengthens our belief that the power manipulation had indeed a causal effect on feelings of power.
| \(\hat{\beta}\) | \(\text{SS}\) | \(\text{df}\) | \(F\) | \(p(\geq \lvert F \rvert)\) | |
|---|---|---|---|---|---|
| Intercept | 57.287 | 1.31e+06 | 1 | 5742.575 | 0.000 |
| \(\texttt{P}\) | -0.700 | 4.90e+01 | 1 | 0.214 | 0.644 |
| \(\texttt{B}\) | 0.095 | 9.04e-01 | 1 | 0.004 | 0.950 |
| \(\texttt{P} \times \texttt{B}\) | 3.429 | 2.94e+02 | 1 | 1.286 | 0.257 |
| Error | 9.05e+04 | 396 |
9.2 Acounting for pre-existing differences
We have just used two separate analyses to (1) assess differences between the priming conditions in feelings of power after the priming task, and (2) rule out that these were due to pre-existing differences. Relying on patterns of significance over multiple analyses is not ideal, particularly when one of these involves an expectation of a null effect. The problem is that a non-significant test result is not direct evidence for the absence of an effect. Non-significant results can always be due to a lack of power. So again, to help you remember: a non-significant test result is not direct evidence for the absence of an effect.
There is a better way to reach both objectives: by including \(\texttt{pre-power}\) as a predictor in our model of \(\texttt{post-power}\), we can determine the effect of prime and experimenter belief, whilst controlling for the effect of \(\texttt{pre-power}\). This is the general idea of ANCOVA: to test for group differences whilst controlling for the effect of covariates (metric predictors). Because in a linear model, effects (i.e slopes) of predictors represent unique effects, we always control for the effects of the other predictors in the model when assessing the effect of a focal predictor.
When we add \(\texttt{pre-power}\) to the model, the model can be written as: \[\begin{equation} \texttt{post-power}_i = \beta_0 + \beta_\text{P} \times \texttt{P}_i + \beta_\text{B} \times \texttt{B}_i + \beta_{\text{P}\times \text{B}} \times (\texttt{P} \times \texttt{B})_i + \beta_\text{pre} \times \texttt{pre-power}_i + \epsilon_i \tag{9.2} \end{equation}\] Parameter estimates and tests are provided in Table 9.3. The results show a significant effect of prime (\(\texttt{P}\)), as well as the pre-test power score (\(\texttt{pre-power}\)). The latter effect is expected, and can be interpreted as usual: the positive slope indicates that participants who scored relatively high in the pre-test also score relatively high in the post-test. Comparing the effect of power prime to that obtained earlier (Table 9.1), we can notice two things: the slope is somewhat higher, and the value of the \(F\) statistic is substantially higher. We will consider the interpretation of the slope in the next section, and the reason for the higher \(F\) statistic in Section 9.5.
| \(\hat{\beta}\) | \(\text{SS}\) | \(\text{df}\) | \(F\) | \(p(\geq \lvert F \rvert)\) | |
|---|---|---|---|---|---|
| Intercept | 16.071 | 6664 | 1 | 53.86 | 0.000 |
| \(\texttt{P}\) | 4.379 | 1916 | 1 | 15.49 | 0.000 |
| \(\texttt{B}\) | -0.079 | 1 | 1 | 0.00 | 0.944 |
| \(\texttt{pre-power}\) | 0.720 | 46920 | 1 | 379.19 | 0.000 |
| \(\texttt{P} \times \texttt{B}\) | 1.334 | 44 | 1 | 0.36 | 0.550 |
| Error | 48876 | 395 |
9.3 Slopes of contrast-coded predictors in ANCOVA models
Although a metric predictor in an ANCOVA model is really just like any other predictor in the GLM, it is useful for clarity to (momentarily) denote such a covariate with a different symbol, e.g. \(Z\), instead of the usual \(X\) we used for predictors. We can then write the model of Equation (9.2) more abstractly as: \[\begin{equation} Y_i = \beta_0 + \beta_1 \times X_{1,i} + \beta_2 \times X_{2,i} + \beta_3 \times X_{3,i} + \beta_\text{z} \times Z_i + \epsilon_i \tag{9.3} \end{equation}\] Here, the \(X\) variables are the contrast-coded predictors (e.g. \(\texttt{P}\), \(\texttt{B}\), and \(\texttt{P}\times\texttt{B}\)), and \(Z\) is a single covariate (e.g. \(\texttt{pre-power}\)).
As in any General Linear Model, inclusion of additional predictors will likely change the slopes of already included predictors. The slopes of the latter predictors will remain the same only when the additional predictors are completely independent from the already-included predictors. Such complete non-redundancy is in practice extremely unlikely. If there is redundancy (i.e. multicollinearity) in a model like that of Equation (9.3), we can no longer use Equation (7.3) to compute the estimated slopes of contrast-coded predictors which are based on a set of orthogonal contrast codes. Fortunately, that does not mean those slopes no longer have a useful interpretation. In fact, the estimated slopes can be expressed as an adjusted version of Equation (7.3): \[\begin{equation} \hat{\beta}_j = \frac{\sum_{k=1}^{g} c_{j,k} \overline{Y}_k}{\sum_{k=1} c_{j,k}^2} - \hat{\beta}_z \frac{\sum_{k=1}^{g} c_{j,k} \overline{Z}_k}{\sum_{k=1} c_{j,k}^2} \tag{9.4} \end{equation}\] where \(\hat{\beta}_z\) is the slope of the covariate in the full model (Equation (9.3)). You can think of this as follows: An orthogonal contrast code reflects differences between marginal means. In the presence of a covariate, these differences are adjusted for the same difference in the marginal means of the covariate, weighted by its effect on the dependent variable. If there are differences in the covariate between the groups, and the covariate has an effect on the dependent variable, then at least some part of these differences can be explained by differences in the covariate between the groups. The adjustment removes the effect of differences in covariate values between the conditions from the values of the dependent variable. After the adjustment, the slope of the contrast code then reflects differences between the groups that are not explained by the covariate.
Let’s illustrate how this works. The means of \(\texttt{pre-power}\) and \(\texttt{post-power}\) are:
| Prime | Belief | pre-power | post-power |
|---|---|---|---|
| PL | EL | 58.5 | 56.3 |
| PL | EH | 56.8 | 54.4 |
| PH | EL | 56.0 | 58.3 |
| PH | EH | 57.8 | 60.2 |
The estimated slope of \(\texttt{pre-power}\) was \(\hat{\beta}_\text{pre} = 0.72\). Using the contrast code for Prime as we used before (i.e. \(-\tfrac{1}{2}\) for the low-power prime conditions and \(-\tfrac{1}{2}\) for the high-power prime conditions), we can determine the estimated slope of Prime as on post-power
\[\begin{aligned} \hat{\beta}_\texttt{P} &=&& \frac{\tfrac{1}{2} \times 58.31 + \tfrac{1}{2} \times 60.2 - \tfrac{1}{2} \times 56.33 - \tfrac{1}{2} \times 54.42}{ (\tfrac{1}{2})^2 + (\tfrac{1}{2})^2 + (-\tfrac{1}{2})^2 + (-\tfrac{1}{2})^2} - \\ &&& 0.72 \times \frac{\tfrac{1}{2} \times 56.03 + \tfrac{1}{2} \times 57.84 - \tfrac{1}{2} \times 58.45 - \tfrac{1}{2} \times 56.83}{ (\tfrac{1}{2})^2 + (\tfrac{1}{2})^2 + (-\tfrac{1}{2})^2 + (-\tfrac{1}{2})^2} \\ & = && \frac{3.875}{1} - 0.72 \times \frac{-0.7}{1} \\ & = && 4.379 \end{aligned}\]
An alternative way to define the slope of Equation (9.4) is to first adjust the means of \(Y\) (i.e. post-power here) and then enter the adjusted group means in the usual formula for the slope of a contrast-coded predictor. These adjusted means are computed using a centered version of the covariate (i.e. \(Z'_i = Z_i - \overline{Z}\)):
\[\overline{Y}_k' = \overline{Y}_k - \hat{\beta}_z \times (\overline{Z}_k - \overline{Z})\]
where \(\overline{Z}\) is the average of the covariate over all observations (i.e. it is not an “average of averages”). The slope can then be expressed as a function of these adjusted means:
\[\hat{\beta}_j = \frac{\sum_{k=1}^{g} c_{j,k} \overline{Y}_k'}{\sum_{k=1} c_{j,k}^2}\]
The fact that these two formulations are equivalent again illustrates the main idea of an ANCOVA model, which is to assess group differences that can not be attributed to the covariate.
9.4 Homogeneity of slopes
The model of Equation (9.1) does not include any interactions between the covariate and the contrast-coding differences. As such, the model assumes the effect of the covariate on the dependent variable is the same in each condition. This assumption is usuallty referred to as the assumption of homogeneity of regression slopes. The resulting regression lines for the relation between \(\texttt{power-pre}\) and \(\texttt{power-post}\) in the different conditions are depicted in Figure 9.2. The assumption that the slopes are identical means that the the regression lines are parallel to each other. Note that the height of each regression line reflects the combined effect of power priming and experimenter belief. Within each condition, these effects are a constant, and hence the combined effect of these factors effectively is equal to the intercept of each regression line.
As discussed above, you can think of an ANCOVA model as statistically correcting the conditions for any differences in the covariate. After this correction, the conditions effectively have the same value on the covariate. In the context of Figure 9.2, you can pick any value of the covariate you like, and then think of the contrasts between the conditions as comparisons between the model predictions at that value of the covariate. Because the regression lines for both high-power prime conditions, and for both low-power prime conditions, are almost the same, it is clear that experimenter belief has little effect on feelings of power. The high-power and low-power conditions have more clearly separated regression lines, which is corroborated by the significant effect of power prime in the model.
Figure 9.2: Regression lines reflecting the relation between \(\texttt{power-pre}\) and \(\texttt{power-post}\) in the four conditions of the experimenter belief and power priming experiment.
While the assumption of parallel regression lines (i.e. homogeneity of regression slopes) allows for a straightforward interpretation of the other effects as reflecting differences in adjusted means, as for any assumption we make, there is always uncertainty about whether it can reasonably be assumed to hold. Due to the symmetry of moderation, not only does the lack of interactions between the covariate and contrast-coded predictors imply that the effect of the covariate on the dependent variable is identical in each condition, it also means that the effect of the conditions is the same regardless of the value on the covariate. There may be good reasons to suspect that the effect of the experimental manipulations differs for people who score differently on the covariate. For instance, you might think that a high-power prime is effective in lifting the subjective feeling of power for those who have a relatively low feeling of power to start off with, while it doesn’t do much for those who feel powerful anyway. In a similar vain, the low-power prime might lower the subjective feeling of power for those who have a relatively low feeling of power to start off with, while those with a relatively high feeling of power might be immune to it. If this were true, then the effects of experimental manipulations are moderated by the covariate. This indicates we should include interactions between the covariate and the contrast-coded predictors. Allowing for such interactions is straightforward: we just construct new product-predictors and include them in the model.
Allowing for moderation of the effect of conditions by the covariate in the model of Equation (9.1), we would need add three product predictors: \((\texttt{P} \times \texttt{power-pre})_i\), \((\texttt{B} \times \texttt{power-pre})_i\), and \((\texttt{P} \times \texttt{B} \times \texttt{power-pre})_i\). The results of the expanded model are given in Table 9.4.
| \(\hat{\beta}\) | \(\text{SS}\) | \(\text{df}\) | \(F\) | \(p(\geq \lvert F \rvert)\) | |
|---|---|---|---|---|---|
| Intercept | 15.945 | 6519.6 | 1 | 52.440 | 0.000 |
| \(\texttt{P}\) | 6.358 | 259.1 | 1 | 2.084 | 0.150 |
| \(\texttt{B}\) | -2.766 | 49.1 | 1 | 0.395 | 0.530 |
| \(\texttt{pre-power}\) | 0.722 | 46882.4 | 1 | 377.091 | 0.000 |
| \(\texttt{P} \times \texttt{B}\) | -4.101 | 27.0 | 1 | 0.217 | 0.642 |
| \(\texttt{P} \times \texttt{pre-power}\) | -0.035 | 28.0 | 1 | 0.225 | 0.635 |
| \(\texttt{B} \times \texttt{pre-power}\) | 0.048 | 51.3 | 1 | 0.412 | 0.521 |
| \(\texttt{P} \times \texttt{B} \times \texttt{pre-power}\) | 0.095 | 51.2 | 1 | 0.412 | 0.521 |
| Error | 48736.0 | 392 |
We can see that none of the interactions between \(\texttt{pre-power}\) and the contrast-coded predictors are significant. As such, there is no strong evidence of an interaction between the covariate and the experimental manipulations. Rather than inspecting each interaction separately, it makes sense to perform an omnibus test, comparing the expanded model to the one of Equation (9.1). This comparison tests the hypothesis that all the slopes of the additional product-predictors equal 0. If there is no overly strong reason to suspect a moderation for particular conditions, the omnibus test is a reasonable test that the covariate interacts with any of the experimental manipulations.22 This omnibus test is a straightforward application of the model comparison approach. We need the Sum of Squared Errors of each model, which are \(\text{SSE}(G) = 48736\) and \(\text{SSE}(R) = 48876\) for the expanded model and the one of Equation (9.1), respectively. The \(F\) statistic is then computed as usual as
\[\begin{aligned} F &= \frac{\frac{\text{SSE}(R) - \text{SSE}(G)}{\text{npar}(G) - \text{npar}(R)}}{\frac{\text{SSE}(G)}{n-\text{npar}(G)}} \\ &= \frac{\frac{48876 - 48736}{8 - 5}}{\frac{48736}{400-8}} \\ &= 0.375 \end{aligned}\] With \(\text{df}_1 = 3\) and \(\text{df}_2 = 392\), the critical value is \(F_{3,392, .05} = 2.63\). Hence, the result is non-significant, and we do not reject the null hypothesis that there are no interactions between the covariate and the contrast-coded predictors. In other words, we have no evidence that the assumption of homogeneous regression slopes is violated.
If the assumption of homogeneous regression slopes is violated, then the model should really include the appropriate interaction terms. In that case, it is important to remember that the “simple slopes” of the contrast-coded predictors then reflect tests of group differences at particular values of the covariate. Group differences at one value of the covariate may be substantial, but absent or even reversed at another. It might make sense to center the covariate, so that you would test for group differences for an average value of the covariate. You could also consider not only centering at the mean, but at a few informative other values. For instance, you could consider testing the slopes at three values of the covariate: the minimum value, mean, and maximum value in the data. You could do this by creating three different “centered” covariates: \(Z'_i = Z_i - \overline{Z}\), \(Z''_i = Z_i - \text{min}(Z)\), and \(Z'''_i = Z_i - \text{max}(Z)\), and entering each in a different version of the same model. If the estimates of the group differences have the same direction and the tests significant in each, then you would conclude that there is evidence of group differences in the whole range of the covariate observed in the data. Other strategies are possible, of course. The main thing to realise is that in a model with interactions between the covariate and contrast-codes, group differences depend on the value of the covariate, and statements about group differences should be qualified by for which value of the covariate they hold.
9.5 Power considerations in ANCOVA
A primary reason for including covariates in models with categorical independent variables is to assess group differences in the dependent variable whilst controlling for any possible differences in the covariates. But there is another reason why the inclusion of covariates can be a good idea: it may increase the power of the tests of group differences. We saw evidence of this when we included \(\texttt{pre-power}\) in the model for \(\texttt{post-power}\). Comparing the results in Table 9.3 to those in Table 9.1, you can see that the test results for the main effect of priming are stronger after inclusion of the covariate. The reason for this is that the covariate can explain differences in \(\texttt{post-power}\) within each condition. Accounting for this within-condition variance reduces the overall error variance of the model, which can subsequently increase the power of the tests. This is particularly the case when the covariate is strongly related to the dependent variable, but independent of the contrast-coded predictors. Such independence implies that the groups do not differ in the average value of the covariate.
We can illustrate this with the Venn diagrams in Figure 9.3. Let’s consider first the situation of no redundancy between the covariate \(Z\) and the contrast-coded predictors \(X\), which is depicted in the left-hand plot. The SSR associated to the contrast-coded predictors is region \(B\), and the SSR associated to the covariate is region \(C\). The test of group differences (i.e. the test of the contrast-coded predictors) would involve a comparison of the SSR of region \(B\) against the unexplained error of region \(A\). In a model without the covariate, the unexplained error would be the sum of region \(A\) and \(B\). As this error is larger, the \(F\) statistic will be smaller, even though the SSR term for \(X\) is the same. As such, inclusion of the covariate increases the power of the test of \(X\), and the increase in power is larger the stronger the relation between the covariate \(Z\) and the dependent variable \(Y\) (i.e. the larger region \(B\)). The situation is more complicated when there is redundancy between the covariate and the contrast-coded predictors. This is the situation depicted in the right-hand plot. Whilst inclusion of the covariate again reduces the error – which would be the sum \(A+B\) for a model without the covariate, but only \(A\) for a model with the covariate – it also reduces the unique SSR that can be associated to the contrast-coded predictors. In a model without the covariate, \(\text{SSR}(X) = B + D\), and in a model with the covariate, \(\text{SSR}(X) = B\). The ANCOVA model is now no longer guaranteed to increase the power of the test of \(X\), as this depends on how the reduction in error compares to the reduction of the SSR.
In conclusion, for purposes of increasing power, the covariate is ideally strongly related to the dependent variable, but unrelated to the other predictors in the model.
Figure 9.3: Partitioning the variance in an ANCOVA model. The circle labelled as \(Y\) represents the variance of the dependent variable, circle \(X\) represents a (set of) contrast-coded predictor(s), and circle \(Z\) the covariate. Overlapping regions represent shared variability (e.g. covariance) between variables.
9.6 Models with multiple covariates
The analyses reported above indicate that the power priming task had the desired result of changing participants’ subjective feeling of power. The significant effect of prime on \(\texttt{post-power}\) reflects differences between the means of the priming conditions. It is likely that within those conditions, there is variability in the subjective feeling of power. According to the social priming hypothesis, we might expect the approach advantage scores to be higher for those participants with a relatively strong subjective feeling of power, and lower for those with a relatively weak subjective feeling of power.
Given the significant differences in \(\texttt{post-power}\) between the priming conditions, but the lack of a significant effect of priming condition on approach advantage scores, a relation between \(\texttt{post-power}\) and \(\texttt{ApproachAdvantage}\) seems unlikely, but we can include \(\texttt{post-power}\) as a predictor of \(\texttt{ApproachAdvantage}\) and see. Rather than the final subjective feeling of power, it might also be the case that the approach advantage is related to how much the priming manipulation increased or decreased the feeling of power. To investigate this, we might include the difference \(\texttt{post-power} - \texttt{pre-power}\) as a predictor. But actually, we can obtain a similar effect by just including \(\texttt{pre-power}\) as a predictor in the model; when the slope of \(\texttt{pre-power}\) is negative and the slope of \(\texttt{post-power}\) positive, the combined effect on \(\texttt{ApproachAdvantage}\) would be similar. The model thus becomes: \[\begin{aligned} \texttt{ApproachAdvantage}_i =& \beta_0 + \beta_\text{P} \times \texttt{P}_i + \beta_\text{B} \times \texttt{B}_i + \beta_{\text{P}\times \text{B}} \times (\texttt{P} \times \texttt{B})_i + \\ & \beta_\text{pre} \times \texttt{pre-power}_i + \beta_\text{post} \times \texttt{post-power}_i + \epsilon_i \end{aligned}\] Parameter estimates and tests are provided in Table 9.5. As in the earlier ANOVA (Table 8.1), we obtain a significant effect of experimenter belief. None of the other effects are significant. Neither the subjective feeling of power before or after the priming manipulation appears to affect the approach advantage scores. Interestingly, whilst not significant, the slope of \(\texttt{post-power}\) is positive, and the slope of \(\texttt{pre-power}\) negative, which is what one would expect when the difference between the two is related to the dependent variable.
| \(\hat{\beta}\) | \(\text{SS}\) | \(\text{df}\) | \(F\) | \(p(\geq \lvert F \rvert)\) | |
|---|---|---|---|---|---|
| Intercept | 28.396 | 18309 | 1 | 0.397 | 0.529 |
| \(\texttt{P}\) | 9.823 | 9279 | 1 | 0.201 | 0.654 |
| \(\texttt{B}\) | 90.569 | 820210 | 1 | 17.765 | 0.000 |
| \(\texttt{pre-power}\) | -0.573 | 15190 | 1 | 0.329 | 0.567 |
| \(\texttt{post-power}\) | 0.452 | 9998 | 1 | 0.217 | 0.642 |
| \(\texttt{P} \times \texttt{B}\) | 37.883 | 35727 | 1 | 0.774 | 0.380 |
| Error | 18190490 | 394 |
Note that the slopes of the contrast-coded predictors are somewhat different from those in Table 8.1. As before, this is because the slopes in the ANCOVA model represent differences between adjusted means. Analogous to Equation (9.4), when there are a total of \(L\) covariates included in the model, which we can denote as \(Z_1, Z_2, \ldots, Z_L\), the estimate of the slope of the contrast-coded predictors can be written as: \[\begin{equation} \hat{\beta}_j = \frac{\sum_{k=1}^{g} c_{j,k} \overline{Y}_k}{\sum_{k=1} c_{j,k}^2} - \sum_{l=1}^L \hat{\beta}_{z_l} \frac{\sum_{k=1}^{g} c_{j,k} \overline{Z}_{l,k}}{\sum_{k=1} c_{j,k}^2} \tag{9.5} \end{equation}\] i.e. the total adjustment of the slope consists of the sum of adjustments for each covariate.
9.7 Mediation with categorical independent variables
The finding that experimenter belief has an effect on participants’ approach advantage indicates that experimenter expectations affect participants behaviour, even though the experimenters in the study of Gilder & Heerey (2018) asserted that their knowledge of the condition had not affected their behaviour towards the participants. The question is then how experimenter belief changed their interaction with the participants to change their approach advantage scores. To attempt to answer this question, we assess whether experimenter belief changed participants’ perception of the experimenters, and whether such changes resulted in the difference in approach advantage. Thus, as another example of mixing categorical and metric independent variables, we can consider assessing whether the effect of experimenter belief on approach advantage is mediated by participants’ perceptions of the experimenters.
At the end of the experiment, participants rated how attractive, competent, friendly, and trustworthy they found their experimenter. Preliminary analysis (not shown here for brevity) indicates that when experimenters believed participants were assigned to the high-power condition, they were rated as more attractive, friendly, and trustworthy. If the effect of experimenter belief is mediated by these changes in perception, then after including the ratings of attractiveness, friendliness, and trustworthiness, we would expect the effect of experimenter belief to be reduced. However, the results of the analysis (see Table 9.6) are only slighly suggestive of this. Controlling for attractiveness, friendliness, and trustworthiness, we still obtain a highly significant effect of experimenter belief, with a slope which is a little lower than in a model without these covariates. But none of the covariates appears to be related to the approach advantage.
| \(\hat{\beta}\) | \(\text{SS}\) | \(\text{df}\) | \(F\) | \(p(\geq \lvert F \rvert)\) | |
|---|---|---|---|---|---|
| Intercept | -29.611 | 12777 | 1 | 0.277 | 0.599 |
| \(\texttt{P}\) | 10.895 | 11704 | 1 | 0.253 | 0.615 |
| \(\texttt{B}\) | 85.603 | 698114 | 1 | 15.120 | 0.000 |
| \(\texttt{attractive}\) | -0.792 | 561 | 1 | 0.012 | 0.912 |
| \(\texttt{friendly}\) | 10.378 | 39004 | 1 | 0.845 | 0.359 |
| \(\texttt{trustworthy}\) | -0.889 | 253 | 1 | 0.005 | 0.941 |
| \(\texttt{P} \times \texttt{B}\) | 39.049 | 37861 | 1 | 0.820 | 0.366 |
| Error | 18144858 | 393 |
9.8 ANCOVA vs difference scores
To end the chapter, I want to discuss an alternative method to assess whether the power prime manipulation was effective. Note that the main effect of power prime reported in Table 9.1 just reflects a difference between high-power and low-power priming conditions. That there is a difference between the conditions does not really tell us exactly what the priming task did to participants’ feeling of power. It could be that unscrambling high-power sentences increases feelings of power, and unscrambling low-power sentences decreases feelings of power. But it could also be that the task increased feelings of power in both conditions, but more so in the high-power conditions. Or it could be that the task decreased feelings of power in both conditions, but less so in the high-power conditions. In all these cases, the mean would be higher in the high-power conditions than in the low-power conditions, but the effect of the priming task is rather different. To more directly assess whether the high-power prime increased, and the low-power prime decreased participants’ subjective feeling of power, we can consider using the difference \(\texttt{diff-power}_i = \texttt{post-power} - \texttt{pre-power}\) as dependent variable in the model \[\texttt{diff-power}_i = \beta_0 + \beta_\text{P} \times \texttt{P}_i + \beta_\text{B} \times \texttt{B}_i + \beta_{\text{P}\times \text{B}} \times (\texttt{P} \times \texttt{B})_i + \epsilon_i\] The model results are given in Table 9.7. We again find a significant and positive effect of power prime. We also find that the intercept is close to 0 and non-significant. That indicates that averaged over all conditions, the subjective feeling of power seems to neither increase or decrease. However, the significant positive slope of power prime then shows that the high-power prime increased, and the low-power prime decreased, subjective feeling of power.
| \(\hat{\beta}\) | \(\text{SS}\) | \(\text{df}\) | \(F\) | \(p(\geq \lvert F \rvert)\) | |
|---|---|---|---|---|---|
| Intercept | 0.028 | 3.21e-01 | 1 | 0.002 | 0.962 |
| \(\texttt{P}\) | 4.575 | 2.09e+03 | 1 | 14.806 | 0.000 |
| \(\texttt{B}\) | -0.105 | 1.11e+00 | 1 | 0.008 | 0.930 |
| \(\texttt{P} \times \texttt{B}\) | 0.373 | 3.48e+00 | 1 | 0.025 | 0.875 |
| Error | 5.60e+04 | 396 |
Although a model with a difference score as dependent variable is straightforward to interpret when we are mainly interested in determining group differences (i.e. the effects of the contrast-coded predictors), there are reasons to prefer an ANCOVA model (e.g. Equation (9.2)) to this analysis. If we write out the difference score model as \[\texttt{post-power}_i - \texttt{pre-power}_i = \beta_0 + \beta_\text{P} \times \texttt{P}_i + \beta_\text{B} \times \texttt{B}_i + \beta_{\text{P}\times \text{B}} \times (\texttt{P} \times \texttt{B})_i + \epsilon_i\] we can add \(\texttt{pre-power}\) to both sides of the equation to obtain \[\texttt{post-power}_i = \beta_0 + \beta_\text{P} \times \texttt{P}_i + \beta_\text{B} \times \texttt{B}_i + \beta_{\text{P}\times \text{B}} \times (\texttt{P} \times \texttt{B})_i + \texttt{pre-power}_i + \epsilon_i\] Note that this model is the same model as in Equation (9.2) if we fix \(\beta_\text{pre} = 1\). In other words, the difference model is a special case of the ANCOVA model, where we assume we know the slope of the covariate equals 1. If this assumption is (approximately) true, then the difference model can be preferred, because it has one parameter less to estimate. If, however, the true slope of the covariate is different from 1, then the difference model is less accurate than the ANCOVA model. If the slope of the covariate is sufficiently different from 1, then the ANCOVA model will provide more powerful tests of the contrasts than the difference model (Charles M. Judd et al., 2011). In addition, if one is interested in modelling changes within participants, then approaches such as repeated-measures ANOVA and linear mixed-effects models are a better choice.
References
Although my reasoning why the power-prime manipulation might have a different effect for those participants with an initial high or low feeling of power might suggest otherwise, this was to illustrate why you might suspect an interaction, it is not a hypothesis I would stand by personally. I’m by no means an expert in the scientific study of social power.↩︎