## ​​​​​​​​​​​Frequently Asked Questions about Mapping State Proficiency Standards Onto NAEP Scales: 2005-2013

Background and Purpose of the Study

Data and Sample

Mapping of State Standards Onto the NAEP Scale

## Background and Purpose of the Study

### How does NAEP report on student performance?

NAEP reports student data in two ways: scale scores and achievement levels.

• Scale scores show what students know and can do. Scores are reported on a 0-500 point scale for reading and mathematics at grades 4 and 8. Since these scales are calculated independently from each other, we cannot compare scale scores across subjects and grades.
• Achievement levels are performance standards for what students should know and be able to do. The NAEP achievement levels Basic, Proficient, and Advanced are indicators of student performance. NAEP results are reported as percentages of students performing at or above these levels. Basic denotes partial mastery of the knowledge and skills that are fundamental to proficient work at a given grade. Proficient represents solid academic performance. Students reaching this level have demonstrated competency on challenging subject matter: Proficient is not synonymous with grade-level performance. Advanced signifies superior performance. These achievement levels are set independently for each subject and grade by the National Assessment Governing Board, which sets policy for NAEP.

### What are the state proficiency standards?

Each state designs its own assessments, sets its state-level proficiency standards, and reports the percentages of students who meet these standards in reading and mathematics. These standards are the points on the performance scale at which students reach proficient on the state assessment. The percentage of students reaching the proficient level is used by states to report adequate yearly progress (AYP) to the U.S. Department of Education under the 2001 reauthorization of the Elementary and Secondary Education Act (ESEA).

### What are the main purposes of the mapping study?

The main purpose is to provide a common metric for comparing the stringency of different state criteria for academic proficiency.

### Why is NAEP used as the basis for comparison of state proficiency standards?

Each state designs its own assessment and sets its own standards for fourth- and eighth-grade reading and mathematics proficiency. Because the states use different standards for “proficiency,” state standards cannot be compared directly; they need to be placed on a common metric. NAEP provides the common metric.

### How were states compared?

States’ standards are compared using the NAEP scale equivalent score corresponding to a state’s standard.

Using the percentages of students reaching proficiency of standards of states who also participated in NAEP, a score point on NAEP scale for each state is identified. Mapping states’ proficiency standards onto a single measurement scale (in this case, NAEP) provides a basis for making comparisons of the states.

## Data and Sample

### What are the data sources used for these reports?

The 2013 study used the following data sources to acquire NAEP and state assessment results for grades 4 and 8:

• NAEP data files for the 2011 and 2013 assessments in reading and mathematics for participating public schools;
• School-level state assessment results directly provided by the states.

### Are any states excluded from the mapping of 2013 state standards?

No data for grade 8 mathematics were available for California and Virginia because the states do not test general mathematics at this grade level.

### Why were some states not represented in some analyses?

In 2009, Nebraska was not included because the state did not administer a statewide assessment. No data for grade 8 mathematics were available for California and Virginia because the states do not test general mathematics at this grade level.

## Mapping of State Standards Onto the NAEP Scale

### How should we interpret a state’s NAEP scale equivalent?

A state’s NAEP scale equivalent is the score on the NAEP scale at which the percentage of the NAEP sample in the state scoring at or above that value matches the percentage of students in the state scoring proficient or higher on the state assessment. For example, if 45 percent of students in a state score at or above proficient on the state assessment, the state’s NAEP scale equivalent will be the NAEP score at which exactly 45 percent of the state’s NAEP sample scores higher.

### How do state proficiency standards relate to NAEP achievement levels?

After mapping the state proficiency standards to the NAEP scale, it was found that most states’ standards for proficiency were at or below the NAEP Basic achievement level, meaning partial mastery of fundamental skills.

The standard for proficiency may be different among states. In some states, for example, proficient may mean that a student demonstrates minimum competency. Other states may set the level for proficiency as the standard for promotion to the next grade, or as the grade-level standard. By comparison, the NAEP Proficient achievement level is defined as competency over challenging subject matter.

### Does this mapping method allow us to link individual student scores received on the state test to the NAEP scale? If not, why not?

No, student scores cannot be linked to the NAEP scale because NAEP does not generate reliable scores for individual students, only average scores for groups of students (e.g., males, females) within a jurisdiction (e.g., state, district).

### Can the mapping provide a value judgment about the state standards?

No. A more rigorous standard for proficiency is not necessarily better than a less rigorous criterion, because a standard that is too high might not serve to motivate teachers and students as well as a standard that is within reach. Statistical analysis cannot directly tell us about the quality of states’ standards, in part because quality cannot be determined quantitatively. The setting and evaluation of standards must incorporate expertise both in subject matter and cognitive development, and even then value judgments must be made. By deliberately omitting substantive policy conclusions, the report allows others in education policy circles to draw their own conclusions.

### Are the NAEP standards too high?

​NAEP’s standards indicate what students should know and be able to do according to the National Assessment Governing Board and are generally considered aspirational. Standards may differ for different tests because they have different purposes. States may construct their standards for the same purpose as NAEP, or they may use them for defining minimum competency, determining promotion to the next grade, or addressing other purposes. Under NAEP’s legislation, NAEP’s achievement levels remain in a ‘trial’ status, to be used and interpreted with caution, until the Commissioner of Education Statistics (on the basis of a congressionally mandated, independent evaluation of NAEP) determines that NAEP’s achievement levels are reasonable, valid, and informative to the public.

Last updated 01 July 2015 (DS)