Commissioner, National Center for Education Statistics
National Assessment of Educational Progress
Mapping 2005 State Proficiency Standards onto the NAEP Scales
June 7, 2007
Today I am releasing the results of a study that uses NAEP as a basis for comparing state-level proficiency standards. To do so, NCES developed a methodology that uses NAEP as a common yardstick.
This methodology is based on locating the proficiency standard each state has developed for its own state test on the NAEP scale. There are many nuances in estimation, weighting and error estimation, but the crux of the comparison is quite simple.
First, we determine what percent of a stateís students have reached that stateís proficiency standard. We then place that standard at the location on the NAEP scale above which the same percentage of students performed on NAEP.
As an example (see slide 1), assume that 86 percent of state Aís students reach the proficiency standard on its own state assessment. We then observe that 86 percent of students in state A scored at or above 186 on NAEP, making 186 the NAEP-equivalent score of that stateís proficiency standard.
In the other example on slide 1, assume that 66 percent of state Bís students scored at or above that stateís own proficiency standard. Using the same procedure, we identify the top 66 percent in state Bís NAEP results and find that 212 is state Bís NAEP-equivalent score.
The NAEP-equivalent score, then, is the basis for making comparisons from state to state.
We applied this methodology to all states with available data in fourth- and eighth-grade reading and mathematics. Some states were not included because we were unable to obtain complete data, or because the degree of alignment between the state assessment and NAEP was not close enough. We were able to find the NAEP-equivalent scores for between 32 and 36 states depending on the subject-grade combination.
Results for 2005
The 2005 results for fourth-grade reading (slide 2) indicate that there was a wide variation in state proficiency standards. The range in NAEP-equivalent scores for state proficiency standards was 73 points, from Massachusetts, the state with the highest NAEP-equivalent score, to Mississippi, the state with the lowest NAEP-equivalent score. To put this in context, the variation in student performance on NAEP between the highest and lowest scoring states was 27 points.
This study provides information about how state proficiency standards relate to NAEP achievement levels. When plotted against the NAEP achievement levels of Basic, Proficient, and Advanced, most statesí standards fall below NAEP Basic for fourth-grade reading.
The position of a stateís standard relative to that of other states does not necessarily reflect its studentsí performance on NAEP. A state with a high standard for its own test can have low average performance on NAEP, and vice versa. Even though most states have a proficiency standard in NAEPís below Basic range, the majority of state average NAEP scores are at or above Basic for fourth-grade reading.
For eighth-grade reading (slide 3), there was a 61-point range on the NAEP scale in state proficiency standards between the highest and lowest states.
Most statesí NAEP-equivalent standards fell in the NAEP Basic range for eighth-grade reading.
For fourth-grade mathematics (slide 4), there was a 55-point range between the highest and lowest state standard. The highest stateís standard was at the NAEP-equivalent score of 255 and the lowest state at 200.
In terms of the NAEP achievement levels for mathematics, most states fall in the NAEP Basic range, with two states in the Proficient range.
A similar pattern was observed for eighth-grade mathematics (slide 5). The range of state proficiency standards in terms of their NAEP-equivalent was 81 points.
Again, most state standards fell in the NAEP Basic range, with three states in the NAEP Proficient range.
There are several things we must take into account when interpreting the findings in this report. First, state assessments differ among themselves in content and design, and also vary from NAEP.
Second, states and NAEP may define the term "proficient" differently. Some states may consider proficiency as minimum competency, while others may set their proficient level as the standard for promotion to the next grade or as grade-level performance. NAEP defines proficiency as competency over challenging subject matter. Further, states and NAEP vary in their approach to establishing cut scores for the proficient level.
Recent similar studies by others, notably the Education Trust, have reached similar conclusions. The NAEP proficiency standard is higher than that of most states in their own tests, and there is a wide variation between state standards. The present study offers a more useful basis for making comparisons among statesí proficiency standards by linking them to a single score scale (NAEP). In addition, this report gives states more information about where they stand across the full range of performance.
The report, individual state profiles of proficiency, and additional data are available on the Web. We will be releasing other reports that address this issue in the coming months. I would like to thank the principal researcher of this work, Dr. Henry Braun, as well as Dr. Donald McLaughlin, upon whose earlier work this report is based.
Commissioner Mark Schneider's Powerpoint Presentation:
Comparing State Proficiency Standards (383 KB)