Skip Navigation
small NCES header image

Interpreting NAEP Long-Term Trend Results

Overview of the Assessment
Reporting the Assessment—Scale Scores and Performance Levels
Results Are Estimates
NAEP Reporting Groups
Statistical Significance
Exclusion Rates and Assessment Results
Cautions in Interpretations

Overview of the Assessment

One of the primary objectives of NAEP is to track trends in student performance over time. The NAEP long-term trend assessments in reading and mathematics were administered throughout the nation in the 2011–2012 school year to students aged 9, 13, and 17. Because the long-term trend program uses substantially the same assessments decade after decade, it has been possible to chart educational progress since 1971 in reading and 1973 in mathematics. Measuring trends in student achievement, or change over time, requires the precise replication of past procedures. Therefore, the long-term trend instruments do not evolve based on changes in curricula or in educational practices; in this way, the long-term trend assessments differ from main NAEP. It is not possible to compare results from national or state main NAEP with those of the long-term trend assessment.

Beginning with the 2004 administration of the long-term trend assessments in reading and mathematics, several changes were made to the assessment design. Any time changes are made in a long-term trend assessment, studies are required to ensure that the results can continue to be reported on the same trend line—that is, that they are validly comparable to earlier results. Analyses were needed to ensure that the 2004 results under the new design were comparable to the results from 1971 through 1999, under the design that existed earlier. Therefore, two assessments were conducted in 2004. The revised assessment used the new design, and the original assessment replicated the former design. Comparisons of the results could then detect any shifts in results due to changes in test design. The original assessment linked the old assessments to the new one.

NAEP assesses student performance by administering samples that are representative of the nation's students. The content of the NAEP long-term trend assessments is determined by a set of objectives incorporating expert perspectives about the measurement of reading and mathematics. Read how the long-term trend assessment was developed and how the assessment was administered. Read more about what the long-term trend reading and long-term trend mathematics assessments measure.

The results presented on the website are based on representative samples of students for the nation. In 2012, approximately 26,300 students took the reading assessment, while about 26,200 took the mathematics assessment. To ensure unbiased samples, NAEP statistical standards require that participation rates for original school samples be 70 percent or higher to report national results separately for public and private school students. At all three ages, the school participation rates met the standards for reporting results separately for public schools in 2012 but not for private schools. Catholic school participation rates, however, did meet the standards in 2012 for reporting results separately; therefore, results for Catholic schools are included on the website.

The target population for the 2012 NAEP long-term trend assessments consisted of 9-, 13-, and 17-year-old students enrolled in public and private schools nationwide. Eligibility for the age 9 and age 13 samples was based on the calendar year. Students in the age 9 sample were 9 years old on January 1, 2012, with birth months January 2002 through December 2002. Students in the age 13 sample were 13 years old on January 1, 2012, with birth months January 1998 through December 1998. Students eligible for the age 17 sample had to be 17 years old on October 1, 2012, with birth months October 1994 through September 1995.

Reporting the Assessment—Scale Scores and Performance Levels

The results of student performance on the long-term trend assessment are presented on this website in three ways: as average scores, scores at selected percentiles, and percentages of students performing at or above each performance level. Student performance in each subject area is summarized as an average score on a 0-to-500 scale. For each year in which the assessments were administered, achievement in a particular subject area is described for a group of students by their average scale score. Trends in student achievement are determined by examining the average scale scores attained by students in the current assessment year and comparing them to the average scale scores in other assessment years. While the score ranges in both subjects are identical, the scale was derived independently for each subject. Therefore, average scale scores between subjects cannot be compared.

Student performance is also described in terms of the percentages of students attaining specific levels of performance. These performance levels correspond to five points on the reading and mathematics scales: 150, 200, 250, 300, and 350. For each subject area, the performance levels from lowest to highest are associated with increasingly advanced skills and knowledge. Examining the percentages of students in each year who attained each performance level provides additional insights into student achievement. Read more about the long-term trend performance levels.

Results are Estimates

The average scores and percentages presented on this website are estimates because they are based on representative samples of students rather than on the entire population of students. Moreover, the collection of subject-area questions used at each grade level is but a sample of the many questions that could have been asked. As such, NAEP results are subject to a measure of uncertainty, reflected in the standard error of the estimates. The standard errors for the estimated scale scores and percentages in the figures and tables presented on this website are available in the NAEP Data Explorer.

Top

NAEP Reporting Groups

Results are provided for groups of students defined by shared characteristics—gender, race or ethnicity, parents' education level, and type of school. Based on participation rate criteria, results are reported for groups of students only when adequate school representation and sufficient numbers of students are present. The minimum requirement is at least 62 students in a particular student group from at least five primary sampling units (PSUs). However, the data for all students, regardless of whether their group was reported separately, were included in computing overall results. Explanations of the reporting groups are presented below.

Gender

Results are reported separately for males and females. Gender was reported by the school.

Race/Ethnicity

Results are presented for students in different racial/ethnic groups according to the following mutually exclusive categories: White, Black, and Hispanic. Results for Asian/Pacific Islander and American Indian (including Alaska Native) students are not reported separately because there were too few students in the groups for statistical reliability. The data for all students, regardless of whether their racial/ethnic group was reported separately, were included in computing the overall national results.

Results by students’ race/ethnicity are presented in this report based on information collected from two different sources:

Observed Race/Ethnicity. Prior to 2004, students participating in the long-term trend assessment
were assigned to a racial/ethnic category based on the assessment administrator’s observation. The
results for the 2004 original assessment format and all previous assessment years are based on
observed race/ethnicity.

School-Reported Race/Ethnicity. Data about students’ race/ethnicity from school records were
collected in 2004, but were not collected for any of the previous NAEP long-term trend assessments.
The results presented in this report for the 2004 revised assessment format and for 2008 and 2012
are based on school-reported race/ethnicity.

Type of School

Results are reported by the type of school that a student attends—public or private. Private schools include Catholic schools and other private schools.

Parents' Education Level

Students were asked to indicate the extent of schooling for each of their parents—did not finish high school, graduated from high school, had some education after high school, or graduated from college. The response indicating the highest level of education for either parent was selected for reporting. The questions were presented only to students at ages 13 and 17. (Results for parental education are not reported at age 9 because research has shown that students' reports of their parents' education level are less reliable at this age.) Although students in previous long-term trend assessments were asked about their parents' level of of education, the wording of the question in the 2004 revised and 2008 reading trend assessment booklets was different from previous years. Consequently, trend results are reported only for the mathematics assessment.

Top

Exclusion Rates and Assessment Results

Some students selected for participation in the NAEP long-term trend assessments were identified as English language learners (ELL) or students with disabilities (SD). In previous long-term trend assessments, if it was decided that a student classified as SD or ELL could not meaningfully participate in the NAEP assessment for which he or she was selected, the student was, according to NAEP guidelines, excluded from the assessment. See tables that summarize the percentage of students identified, excluded, and assessed in long-term trend.

For each student selected to participate in NAEP who was identified as either SD or ELL, a member of the school staff most knowledgeable about the student completed an SD/ELL questionnaire. Students with disabilities were excluded from the assessment if an IEP (individualized education program) team or equivalent group determined that the student could not participate in assessments such as NAEP; if the student's cognitive functioning was so severely impaired that he or she could not participate; or if the student's IEP required that the student be tested with an accommodation or adaptation not permitted or available in NAEP, and the student could not demonstrate his or her knowledge of the assessment subject area without that accommodation or adaptation.

A student who was identified as ELL and who was a native speaker of a language other than English was excluded if the student received instruction in the assessment subject area (e.g., reading or mathematics) primarily in English for less than three school years, including the current year, or if the student could not demonstrate his or her knowledge of reading or mathematics in English without an accommodation or adaptation.

In recent years, changes in policy and legislation pertaining to civil rights have resulted in assessment accommodations being permitted so that SD and ELL students selected to participate in NAEP can do so under revised conditions. Beginning with the 2004 revised trend assessments, NAEP long-term trend assessments offer accommodations for these students. For consistency with the long-term trend assessments administered in past years, however, accommodations were not offered to students in the 2004 original assessment samples.

Statistical Significance

The differences between scale scores and between percentages discussed in the results on this website take into account the standard errors associated with the estimates. Comparisons are based on statistical tests that consider both the magnitude of the difference between the group average scores or percentages and the standard errors of those statistics. Throughout the results, differences between scores or between percentages are discussed only when they are significant from a statistical perspective.

All differences reported are significant at the 0.05 level with appropriate adjustments for multiple comparisons. The term "significant" is not intended to imply a judgment about the absolute magnitude or the educational relevance of the differences. It is intended to identify statistically dependable population differences to help inform dialogue among policymakers, educators, and the public.

Cautions in Interpretations

Users of this website are cautioned against interpreting NAEP results as implying causal relationships. Inferences related to student group performance or to the effectiveness of particular classroom practices, for example, should take into consideration the many socioeconomic and educational factors that may also impact performance.

The NAEP long-term trend scales make it possible to examine relationships between students' performance and various background factors measured by NAEP. However, a relationship that exists between achievement and another variable does not reveal its underlying cause, which may be influenced by a number of other variables. Similarly, the assessments do not reflect the influence of unmeasured variables. The results are most useful when they are considered in combination with other knowledge about the student population and the educational system, such as trends in instruction, changes in the school-age population, and societal demands and expectations.

Comparisons of the 2012 and 2008 results to the 2004 original or previous trend results should be interpreted with caution, bearing in mind the differences in assessment accommodations and changes to assessment procedures.

Top​​


Last updated 22 August 2013 (FW)
Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.
National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education