Background and Purpose of the Study
Data and Sample
Mapping of State Standards Onto the NAEP Scale
NAEP reports student data in two ways: scale scores and achievement levels.
Each state designs its own assessments, sets its state-level proficiency standards, and reports the percentages of students who meet these standards in reading and mathematics. These standards are the points on the performance scale at which students reach proficient on the state assessment. The percentage of students reaching the proficient level is used by states to report adequate yearly progress (AYP) to the U.S. Department of Education under the 2001 reauthorization of the Elementary and Secondary Education Act (ESEA).
The main purpose is to provide a common metric for comparing the stringency of different state criteria for academic proficiency.
Each state designs its own assessment and sets its own standards for fourth- and eighth-grade reading and mathematics proficiency. Because the states use different standards for “proficiency,” state standards cannot be compared directly; they need to be placed on a common metric. NAEP provides the common metric.
States’ standards are compared using the NAEP scale equivalent score corresponding to a state’s standard.
Using the percentages of students reaching proficiency of standards of states who also participated in NAEP, a score point on NAEP scale for each state is identified. Mapping states’ proficiency standards onto a single measurement scale (in this case, NAEP) provides a basis for making comparisons of the states.
The 2013 study used the following data sources to acquire NAEP and state assessment results for grades 4 and 8:
No data for grade 8 mathematics were available for California and Virginia because the states do not test general mathematics at this grade level.
In 2009, Nebraska was not included because the state did not administer a statewide assessment. No data for grade 8 mathematics were available for California and Virginia because the states do not test general mathematics at this grade level.
A state’s NAEP scale equivalent is the score on the NAEP scale at which the percentage of the NAEP sample in the state scoring at or above that value matches the percentage of students in the state scoring proficient or higher on the state assessment. For example, if 45 percent of students in a state score at or above proficient on the state assessment, the state’s NAEP scale equivalent will be the NAEP score at which exactly 45 percent of the state’s NAEP sample scores higher.
After mapping the state proficiency standards to the NAEP scale, it was found that most states’ standards for proficiency were at or below the NAEP Basic achievement level, meaning partial mastery of fundamental skills.
The standard for proficiency may be different among states. In some states, for example, proficient may mean that a student demonstrates minimum competency. Other states may set the level for proficiency as the standard for promotion to the next grade, or as the grade-level standard. By comparison, the NAEP Proficient achievement level is defined as competency over challenging subject matter.
No, student scores cannot be linked to the NAEP scale because NAEP does not generate reliable scores for individual students, only average scores for groups of students (e.g., males, females) within a jurisdiction (e.g., state, district).
No. A more rigorous standard for proficiency is not necessarily better than a less rigorous criterion, because a standard that is too high might not serve to motivate teachers and students as well as a standard that is within reach. Statistical analysis cannot directly tell us about the quality of states’ standards, in part because quality cannot be determined quantitatively. The setting and evaluation of standards must incorporate expertise both in subject matter and cognitive development, and even then value judgments must be made. By deliberately omitting substantive policy conclusions, the report allows others in education policy circles to draw their own conclusions.
NAEP’s standards indicate what students should know and be able to do according to the National Assessment Governing Board and are generally considered aspirational. Standards may differ for different tests because they have different purposes. States may construct their standards for the same purpose as NAEP, or they may use them for defining minimum competency, determining promotion to the next grade, or addressing other purposes. Under NAEP’s legislation, NAEP’s achievement levels remain in a ‘trial’ status, to be used and interpreted with caution, until the Commissioner of Education Statistics (on the basis of a congressionally mandated, independent evaluation of NAEP) determines that NAEP’s achievement levels are reasonable, valid, and informative to the public.