Lecture 30—Monday, March 6, 2006

What was covered?

• Testing linear combinations of parameters
• Orthogonal contrasts

Terminology defined

• linear combination
• orthogonal contrast

Testing linear combinations of regression parameters

• I revisit the echolocation data set that we began discussing last time. By default R uses the smallest numerical value of the type variable to define the baseline level of the factor variable we create.

type.f<-factor(type,label=c('nonecho bats','birds','echo bats'))
contrasts(type.f)

             birds echo bats
nonecho bats     0         0
birds            1         0
echo bats        0         1

• I fit the no interaction (parallel lines) model by including the dummy regressors as additive terms.

new.model<-lm(log(energy)~log(mass)+type.f, data=bats)

This fits the model shown below.

• From the coding scheme for the dummy regressors we see that the intercepts for the three groups are given by the following.

  nonecho bats: β₀
  birds: β₀ + β₂
  echo bats: β₀ + β₃

• Using the output produced by the summary function in R we can test the hypotheses H₀: β₂ = 0 and H₀: β₃ = 0.

> summary(new.model,corr=F)

Call:
lm(formula = log(energy) ~ log(mass) + type.f)

Residuals:
     Min       1Q   Median      3Q     Max
-0.23224 -0.12199 -0.03637 0.12574 0.34457

Coefficients:
                Estimate Std. Error t value  Pr(>|t|)
(Intercept)     -1.57636    0.28724  -5.488 4.96e-05 ***
log(mass)        0.81496    0.04454  18.297 3.76e-12 ***
type.fbirds      0.10226    0.11418   0.896 0.384
type.fecho bats  0.07866    0.20268   0.388 0.703
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.186 on 16 degrees of freedom
Multiple R-Squared: 0.9815, Adjusted R-squared: 0.9781
F-statistic: 283.6 on 3 and 16 DF, p-value: 4.464e-14

• The first hypothesis, H₀: β₂ = 0 , corresponds to a test of whether nonecho bats and birds have the same intercept. This test appears in the line labeled type.fbirds in the summary output.
• The second hypothesis, H₀: β₃ = 0, tests whether echo bats and nbats have the same intercept. This corresponds to the line labeled type.fecho bats in the summary output.

• What if on the other hand we wanted to test whether birds and echo bats have the same intercept? From the equations for the intercept this is a test of the hypothesis H₀: β₂ = β₃ = 0, or equivalently, H₀: β₂ – β₃ = 0. Do we need to refit the model with a different dummy coding scheme?
• The answer is no. This hypothesis can be tested using the above model. We just need to extract some additional information beyond what is displayed in the summary table.
• The t-values that appear in the summary output of the model are calculated by taking a parameter's estimate and dividing the estimate by the standard error of that estimate. This is the basic form of a Wald statistic.

We can apply this basic formula to test H₀: β₂ – β₃ = 0, but to do so we need an estimate of β₂ – β₃ and its standard error.

• The estimate is easy. It's just where each of the individual estimates appears in the summary table above.
• The standard error can be obtained by using the general formula for the variance of a linear combination of parameters that is shown below.

Here the c_i are just numbers while the β_i are the parameters of interest.

• Now where c₂ = 1 and c₃ = –1. Thus applying the above formula we obtain

• All the terms we need to carry out this calculation can be found in what's called the variance-covariance matrix of the parameter estimates. The basic form of this matrix for the model that was fit above is shown below.

Thus we see that the terms we need define the submatrix occupying rows 3 through 4 and column 3 through 4 of this matrix.

• While typically not part of the standard output, the variance-covariance matrix of the parameter estimates is generated as part of fitting the model and can easily be extracted. While we could just extract these individual values and plug them into the formula given above, there is a more elegant way to carry out the calculation.
• It turns out that the variance of a linear combination is an example of a quadratic form, which are a special kind of matrix product. If we create a vector containing the coefficients c₂ = 1 and c₃ = –1 of the linear combination, then the variance we desire is obtained as follows.

• More simply since , we have

• Taking the square root of this quantity yields the standard error required in the denominator of our Wald statistic.

Using a dummy coding scheme to test specific research hypotheses

• Thus far in our analysis we have simply accepted the default coding scheme for the dummy regressors provided by the software. In practice it's far more useful to construct our own hypotheses and hardwire them into the coding scheme. What hypotheses might those be? Some obvious choices would be the following.
  1. Do echolocating animals have different energy usages than non-echolocating animals? This would contrast group 3 (echo bats) with groups 1 and 2 (nonecho bats and birds).
  2. Do birds have a different energy usage from bats? This would contrast group 2 (birds) with groups 1 and 3 (the two kinds of bats).
  3. Do echolocating bats differ in their energy usage from non-echolocating bats? This would compare group 1 vs. group 3 and ignore group 2 altogether.
• As we saw in lecture, figuring out what contrasts are being tested by parameters in various coding schemes typically requires writing out the equations for the groups individually and then figuring out some way to isolate the parameter of interest. This is both bothersome and backwards. We want the hypothesis to define the parameter not vice versa.
• There is one kind of coding scheme in which constructing sensible hypotheses is easy. It's when the contrasts embodied in the coding scheme are constructed so that they are orthogonal to each other. There are two conditions that define orthogonal contrasts.
1. The coefficients contained in a single dummy regressor must sum to zero. Thus if we let c_i denote the individual coefficients of the dummy regressor, then the condition required here is , where m is the number of groups.
2. When the corresponding coefficients of two different dummy regressors are multiplied together and the resultant products are summed, they should sum to zero. Thus if we let c_i denote the coefficients of one dummy variable and d_i denote the coefficients of a second dummy variable then the condition required here is , where m is the number of groups. If you know a little linear algebra you'll recognize that the operation being described here is what's called the dot product or scalar product between two vectors. Note: This last condition needs to be modified if the groups being contrasted have unequal sample sizes.
• In an ANOVA setting orthogonal contrasts can be used to exactly partition the overall significance test into tests of independent information. For this to happen all groups must have the same number of observations.
• As an example here are two dummy regressors whose components test hypotheses 2 and 3 listed above. Their components will also define orthogonal contrasts if it is also the case that each group has the same number of observations.

• Observe that they satisfy both of the orthogonality conditions listed above.
  1. The coefficients in each column sum to 1.
  2. If we multiply corresponding coefficients and add we obtain: (1)(1) + (–2)(0) + (1)(–1) = 0
• The orthogonality conditions are a simple way to guarantee that to understand what a specific regressor tests we need only look at that regressor. The coding scheme of other regressors will have no effect on its interpretation.
• To see this let's fit a model in which the two dummy regressors in the table above comprise the only predictor. Denote the regressors by x₁ and x₂. The general model is

• Remembering that regression equations yield information about the conditional mean of the response, we can write separate equations for mean energy use of nonecho bats, birds, and echo bats using the definitions of x₁ and x₂.

  nonecho bats:

  birds:

  echo bats:

• Consider the hypothesis that is supposedly tested by x₁: are birds different from bats? To test this we should compare the bird mean to the average of the bat means and see whether the difference is different from zero.

• So we see that H₀: β₁ = 0 does test this hypothesis. The exact same approach will show that H₀: β₂ = 0 tests whether the two kinds of bats have the same energy usage.
• As was noted above the conditions given there for orthogonal contrasts only hold if the groups in question all have the same number of observations (balanced case). When this is not the case the second condition for orthogonality is more complicated, , where n_i is the sample size of group i. The hypothesis being tested by these coefficients is less interpretable than in the balanced case.
• I tend to use the original definition of orthogonality in constructing the contrasts regardless of the sample sizes, while recognizing that the contrasts themselves are not truly orthogonal and hence not all of the benefits from orthogonality will accrue.

Cited Reference

• Speakman, J. R. and P. A. Racey. 1991. No cost for echolocation for bats in flight. Nature 350: 421–423.

Course Home Page