Introduction

The recently completed report on teaching in America released by the National Commission on Teaching and America's Future offers a general indictment of the teaching profession. The commission cites a number of statistics that purport to show many newly hired teachers are unqualified for the job. In particular, the commission reports that one fourth of high school teachers lack college training in their primary classroom subject and that teacher recruiting and hiring practices nationwide are `distressingly ad hoc' (Washington Post, 9/13/96). Underlying the concern about out-of-field teaching is the assumption that teachers with degrees in their primary classroom subject are more effective. Although this may seem a common sense proposition, previous work on the relationship between educational outcomes and teacher characteristics is far from conclusive.

There have been literally hundreds of studies, by economists, sociologists and others, on the impact that schools and teachers have on students. Most have modeled standardized test scores across students, schools, or school districts, as a function of individual and family background characteristics and schooling variables such as expenditures per pupil and class size. Most of these conclude that individual and family background traits explain the vast majority of variation in student test scores. The effects of educational inputs such as per pupil spending, teacher experience, and teacher degree level have been shown to be relatively unimportant predictors of outcomes, and the impact of any particular input to be inconsistent across studies (Hanushek 1986).

These results are puzzling, particularly with regard to teachers. Teaching is the largest profession in the United States, employing over three million adults (NCES 1994, 71). An elaborate system of teacher education and certification is geared toward the preparation of those entering teaching, and there are significant professional development opportunities for those in the profession. More than 40 percent of teachers have at least a master's degree and more than 25 percent have at least 20 years full-time teaching experience (NCES 1994, 77). Over 60 percent of all schooling expenditures at the K-12 level are devoted to instructional costs which consist overwhelmingly of teacher salaries and benefits. Further, teacher salary incentives reward years of experience and degree levels, traits that do not appear to have a relationship to student achievement. What can explain the inconsistent findings of the educational productivity literature with respect to educational resources, particularly teachers? In this paper we shed some light on the relationship between student achievement and teacher degree levels. We begin, in the next section, by reviewing the educational productivity literature.

Background: Previous Literature on Educational Productivity

"Educational productivity" studies typically regress student outcomes, such as performance on standardized tests, on factors such as individual and family background variables, and measures of school inputs such as class size, teacher experience and education, and expenditures per pupil.^\1\  A number of studies using this methodology have yielded inconclusive findings. Eric Hanushek notes that these studies as a whole show that "differences in [school] quality do not seem to reflect variations in expenditures, class sizes, or other commonly measured attributes of schools and teachers" (Hanushek 1986, 1142). He concludes that there is "no strong evidence that teacher-student ratios, teacher education, or teacher experience have an expected positive effect on student achievement" and that "there appears to be no strong or systematic relationship between school expenditures and student performance" (Hanushek 1986, 1162).

These findings raise the question of whether it makes sense, from an efficiency standpoint, for schools to spend large sums of money hiring teachers with advanced degrees. However, it may be premature to reach such strong conclusions about the impact of teacher training on student outcomes based on the previous research. For example, a recent "meta-analysis" by Hedges, Laine, and Greenwald (1994), using the same set of studies reviewed by Hanushek, found that the pattern of estimated coefficients reveals a positive relationship between observable teacher characteristics and student outcomes. One may also reject many of the studies reviewed by Hanushek on the basis of poor data. For instance, many early studies were unable to control for prior achievement using "pre-test" scores to net out individual ability, as is now generally accepted to be important (Boardman and Murnane 1979; Hanushek 1979; Hedges, Laine, and Greenwald 1994).

Another problem with many of the studies reviewed by Hanushek is that variables representing school and teacher "quality" are typically very crude. For instance, degree level alone does not distinguish between colleges of differing quality, nor when the degree was granted, nor does it convey any information about college major, certification requirements fulfilled, or subsequent professional development.

Production function studies which have used more refined measures of teacher inputs have found more consistently positive results. Monk and King (1994) report that teacher subject matter preparation in mathematics and science does have some positive impact on student achievement in those subjects. Measures of the selectivity of teachers' colleges have also been shown to be positively related to student achievement (Ehrenberg and Brewer 1994). The latter result most likely reflects the fact that the selectivity measure captures teacher ability. Also, the few studies which have had measures of teacher (verbal) ability, for example in the form of a teacher test score, have found a more positive relationship to student achievement (Coleman et al. 1966; Ehrenberg and Brewer 1995; Ferguson 1991) than those using other teacher characteristics. Additionally, teacher motivation, enthusiasm, and skill at presenting class material are likely to influence students' achievement, but are difficult traits to accurately measure and are thus omitted from standard regression analyses (Goldhaber and Brewer 1997).

Data deficiencies in previous studies may also have led to significant measurement error problems. Many studies that include teacher and class characteristics use variables that have been aggregated to the school level. There is considerable variation in teacher and class characteristics within schools; hence these aggregate level variables are measured with error and may not accurately reflect the true student-teacher relationships. This can lead to dramatically different estimates of the effects of school resources on achievement. Akerhielm (1995) finds this result in the case of class size. Here we focus primarily on teachers, emphasizing how subtle differences in model specification can influence the results and interpretation of the relationship between teacher qualifications and student outcomes.

Econometric Methodology and Data

Following the conventional educational production function methodology, we model the achievement of student i at school j, Y_ij as a function of a vector of individual and family background variables (including some measure of prior ability or achievement), X_ij, and a vector of schooling resources, S_j, which do not vary across students, and a random error term:

S_j may consist of school, teacher, or class specific variables. ß is the return to individual and family background characteristics and is the return to schooling resources. The dependent variable, Y_ij, is individual student achievement (in the 10th grade) on separate standardized tests in each of the four subject areas: mathematics, science, English, and history. The assumption of the model is that the included individual and family background variables and included schooling resources are uncorrelated with the error term.^\2\

We start by including only school-level variables in S_J, then sequentially include general teacher characteristic variables, class-level variables, and finally specific teacher degree variables. If (1) is correctly specified, Ordinary Least Squares (OLS) estimation will yield consistent estimates of ß and . The overall importance of schooling factors S_j can be ascertained by performing an F-test of the hypothesis that the coefficients of the schooling variables are jointly equal to zero. The addition of subject-specific teacher degree information to the model allows us to determine whether these variables affect student outcomes, and how the omission of these variables can influence the general interpretation of teachers' impact on students.

The data used here are derived from the first two waves of the National Educational Longitudinal Study of 1988 (NELS:88). NELS:88 is a nationally representative survey of about 24,000 eighth-grade students conducted in the spring of 1988. About 18,000 of these students were resurveyed and re-tested in the 10th grade (spring 1990). At the time of each survey students took one or more subject based tests in four subject areas: mathematics, science, English, and history. The tests were carefully designed to avoid "floor" and "ceiling" testing effects and were put on a common scale using Item Response Theory.^\3\

The NELS:88 data set is particularly well suited for our analysis since it is nationally representative, contains a comprehensive set of educational variables, and unlike most other data, links students to specific classes and teachers. This is an important characteristic of the survey since it eliminates problems that may arise from using data aggregated to the school-level. Further, this linkage allows us to investigate in detail the effect of subject-specific teacher degree levels on student achievement since the characteristics of each 10th-grade teacher (race/ethnicity, degree level, experience, certification, etc.) who taught students taking the 10th-grade subject tests are known. The teacher and class data in NELS:88 are organized by school subject, such that separate information is available about the teachers in each of the four subject areas sampled. As a result, the sample here is also classified by subject area and all regressions are estimated separately by subject on students who have complete school and family background information. We confine our attention to public school students to avoid potential problems arising from the non-random assignment of students to private schools (Goldhaber 1996). The sample consists of 5,113 students in mathematics; 4,357 students in science; 6,196 students in English; and 2,943 students in history.

Virtually all teachers in public schools have at least an undergraduate degree. However, as illustrated in table 1, which shows descriptive statistics broken down by subject area, far fewer teachers have degrees specific to the subject in which they teach. Consistent with the findings of the National Commission on Teaching and America's Future, in our sample only 68 to 76 percent (depending on class subject) of teachers have at least a BA in their subject area. A lower proportion of mathematics and science teachers have BA degrees in their subject area than English and history teachers. And although about half of all teachers have at least an MA degree, less than a quarter have advanced degrees in their subject area. Finally, it is interesting to note that there is considerable variation by subject in the proportion of teachers who are female, with a much higher proportion of female teachers in English.^\4\

Results

General Educational Production Function Models^\5\

Table 2 shows the OLS estimates of the 10th-grade educational achievement in each of four subject areas. Included in the model are four sets of explanatory variables: individual and family background variables, school-level variables, teacher variables, and class variables. The individual and family background variables include sex, race/ethnicity, parental education, family structure, family income, and 8th-grade test score. School variables include urbanicity, regional dummies, school size, the percentage of students at the school who are white, the percentage of students at the school who are from single parent families, and the percentage of teachers at the school with at least an MA degree. Teacher variables include sex, race/ethnicity, years of experience at the secondary level, whether the teacher is certified, and the teacher's degree level. Class-level variables include class size and percentage of minority students in the class.

Although we do not show the coefficients of individual and family background variables, they are included in each model. For each subject area these variables alone account for the majority of the variation that we are able to explain with our full models. Most of the estimated coefficients of these variables are statistically significant in the expected direction. For instance, years of parental education is significant and positively related to test scores in all four subjects.

We estimate the models sequentially, first including only individual and family background variables, then adding school, teacher, and class variables, respectively. There are interesting differences between subjects in terms of what is explained by each set of variables. Separate F-tests for the school, teacher, and class variables, of the hypotheses that the coefficients at each level are jointly equal to zero, are rejected at the 5 percent level for mathematics and science subjects. However, in English and history, the null hypotheses of joint significance is only rejected in two cases: for the class-level variables in English and the school-level variables in history. It is also worth noting that we explain a much larger portion of the overall variation in mathematics and English test scores, than we do in science and history.

A closer examination of the results reveals that few of the school, teacher, or class coefficients are statistically significant in the expected direction. For instance, we find the counter-intuitive result that class size is positively associated with student achievement in three of the four subject areas (with history being the exception).^\6\ We also find the percentage of teachers with at least an MA degree is statistically insignificant in all four subject areas (this is true in both the model estimated with only school-level variables and the models shown in table 2 which include school, teacher, and class variables). Although this finding may simply indicate that there is little relationship between school-level variables and individual student achievement, it is certainly consistent with previous findings which have helped to shape the impression that teachers' qualifications don't matter.

Other results from these general models tell a similar story. The years of teaching experience variable is not statistically significant in any subject area, nor is it statistically significant whether the teacher has an MA degree.^\7\ This implies that teachers with an MA degree are no more (or less) effective than those without advanced degrees, clearly a counter-intuitive finding. The results for teacher certification are similar in that we find the coefficient on teacher certification to be statistically insignificant (except in English, where teacher certification is significant and negative). In the next section we discuss the impact of adding subject-specific teacher characteristics to the model.

Subject-specific Teacher Models

Traditional education production functions do not include subject-specific teacher degree and certification information. The results in the previous section would lead one to the conclusion that teacher degree and certification have no impact on student achievement, which is in line with much of the previous literature. However, at least in our sample, the use of teacher subject-specific information is critical in interpreting the effects of these teacher characteristics on student achievement.

Table 3 shows the results when we add subject-specific teacher characteristics to our model (whether the teacher is certified in their subject area, and whether the teacher has a BA or MA degree in his or her subject area). These variables allow us to distinguish between teachers who are teaching specific classes and who have a major in that subject (BA or MA), teaching specific classes and are certified in that subject, and those who are teaching but do not have subject-specific training. Columns (1), (3), (5), and (7) of the table are the estimated teacher coefficients when only general teacher variables are included in the model (reproduced from columns 1-4 of table 1), while columns (2), (4), (6), and (8) show the results when we include the more refined subject-specific teacher characteristics.

In mathematics and science, teacher subject-specific training has a significant impact on student test scores in those subjects (see columns (2) and (4)). A teacher with a BA in mathematics, or an MA in mathematics, has a statistically significant positive impact on students' achievement relative to teachers with no advanced degrees or degrees in non-mathematics subjects. We find similar results with teacher certification as illustrated by comparing the certification results in columns (1) and (2). We also see that teachers with BA degrees in science have a positive impact relative to those who teach science but have either no degree or a BA in another subject. These results are confirmed by performing F-tests of the hypotheses that the coefficients of the subject-specific variables are jointly equal to zero. The F-tests are rejected for mathematics and science (at the one percent level). By contrast, we find no evidence that subject-specific degrees or certification have an effect on student achievement in English or history, where the subject-specific variables were statistically insignificant. In these subjects we could not reject the null hypothesis that the coefficients of the subject-specific variables are jointly equal to zero. It is possible that the positive findings for teachers degrees in mathematics and science do not reflect the training that they have in those subjects but simply that mathematics and science degrees serve as proxies for teacher ability. To test this hypothesis we re-estimated all models, including whether a teacher has a mathematics or science degree in the English and history regressions. If mathematics and science degrees serve as proxies for teacher quality, we would expect the coefficients on these variables to be significant and positive in all of the subject areas, including English and history. This is not the case. Neither the mathematics nor the science degree level variables are statistically significant in the English and history regressions. This result clearly suggests that, in mathematics and science, it is the teacher subject-specific knowledge that is the important factor in determining 10th-grade achievement.

We can infer the magnitude of the effect of teacher training on student achievement by examining the estimated coefficients in the models that include subject-specific information. For example, the total effect of a teacher having an MA degree in any subject in the model with only general teacher variables is simply the coefficient on the MA variable. However, in the models with subject-specific information we are able to calculate more refined measures of the impact of teacher degrees. Here, the effect of a teacher having an MA in mathematics is the sum of the coefficients of MA and MA major in mathematics. Table 4 shows the estimated effects of model specification on predicted 10th-grade achievement scores in mathematics and science (we do not show English and history because none of the subject-specific variables were statistically significant). All other variables are measured at their mean value.

We see the impact of model specification in mathematics and science by comparing columns (1) and (2) for mathematics, and columns (3) and (4) for science. The science results do not differ much when subject-specific variables are used; however, there are important differences in the mathematics findings. In the model with general teacher variables we predict students (with average characteristics) who have a teacher certified in mathematics and has both a BA and an MA in mathematics to have a 10th-grade mathematics score of 44.06. However, these same students are predicted to have a 10th-grade mathematics score of 44.69 when the subject-specific specification of the model is used. The difference between these predicted scores, .63, is about 5 percent of the 10th-grade mathematics test standard deviation, a relatively small difference.

Conclusion

Most traditional educational production function studies have used somewhat crude teacher characteristics. For example, in many cases only school-level teacher variables (e.g. percentage of teachers in a school with an MA degree) are included in statistical models of student achievement. In this paper we assess the impact of educational resources in explaining student achievement using more refined measures of teacher skill. We are able to do this using data drawn from the NELS:88 which includes subject-specific teacher degree information and allows us to link students particular teachers and classes. This link enables us to avoid problems with aggregation that may have plagued earlier studies.

We find that subtle differences in model specification can result in very different interpretations of whether teachers affect student outcomes. Although school-level variables do not, in general, seem to have an affect on student achievement level, some teacher characteristics do. Teachers who are certified in mathematics and have BA and MA degrees in mathematics are associated with higher student mathematics test scores. Likewise, teachers with BA degrees in science are associated with higher student science test scores. Because mathematics and science degrees were not found to influence student outcomes in English and history, we believe that these results suggest that it is the subject-specific training rather than teacher ability that leads to these findings. This is important because it suggests that student achievement in technical subjects can be improved by requiring in subject teaching.

Footnotes

References

[The Productivity Collapse in Schools]