Interpreting NAEP Writing Results

Overview of the Assessment
Reporting the Assessment—Scale Scores and Achievement Levels
Description of Writing Performance by Item Maps for Each Grade
Results Are Estimates
NAEP Reporting Groups
Exclusion Rates
Statistical Significance
Cautions in Interpretations

Overview of the Assessment

NAEP assesses student performance in writing by administering assessments to sample groups that are representative of the nation's students. The types of prompts used as part of the NAEP writing assessment are determined by a framework developed with the help of researchers, policymakers, and the interested public as well as experts in the areas of writing and its measurement. Read more about what the assessment measures, how it was developed, who took the assessment, and how the assessment was administered.

The 2011 National Assessment of Educational Progress (NAEP) writing assessment at grades 8 and 12 was developed under a new framework that recognizes the significant role that computers play in the writing process, as well as the prevalence of computer technology in the lives of students and the increasing role of computers in learning activities. Therefore, results from 2011 cannot be compared to those from earlier assessments in writing. The 2011 writing assessment was administered at the national level only. In addition, the writing assessment was not administered at grade 4 in 2011; however, students in grade 4 were assessed in writing in 1998 and 2002. The results from previous writing assessments can be found in the NAEP Data Explorer.

Back to Top

Reporting the Assessment—Scale Scores and Achievement Levels

The results of student performance on the NAEP writing assessment are presented on this website in two ways: as average scores on the NAEP writing scale and as the percentages of students attaining NAEP writing achievement levels. The average scale scores represent how students performed on the assessment. The achievement levels represent how that performance measured up against set expectations for achievement. Thus, the average scale scores represent what students know and can do, while the achievement-level results indicate the degree to which student performance meets expectations of what they should know and be able to do.

Average writing scale scores are based on the NAEP writing scale, which ranges from 0 to 300. Because NAEP scales are developed independently for each subject, scores cannot be compared across subjects. Similarly, although the scales are identical for grades 8 and 12, the scale scores were derived separately; therefore, scores cannot be compared across grades.

NAEP reports not only average scale scores but also percentages of students performing at various achievement levels. The writing achievement levels adopted by the National Assessment Governing Board represent how student performance measured up to set expectations for achievement. With the placement of cut scores on the writing scales, the achievement levels are Basic, Proficient, and Advanced.

The Governing Board established its achievement levels in 2011 based upon the writing content framework and standard-setting process. A cross section of educators and interested citizens from across the nation were asked to judge what students should know and be able to do relative to the content reflected in the NAEP writing framework. As provided by law, NCES has determined that the achievement levels are to be considered on a trial basis and should be interpreted and used with caution. However, both NCES and the Governing Board believe these performance standards are useful for understanding trends in student achievement.

Back to Top

Description of Writing Performance by Item Maps for Each Grade

Writing items maps are illustrations that plot scale scores at the four achievement levels and describe the rating criteria for each level. The maps show the level of knowledge and skills demonstrated by students performing at different scale points on the assessment. The writing item maps provide samples of one of the three types of writing tasks to which students responded at each grade level.

Back to Top

Results Are Estimates

The average scores and percentages presented on this website are estimates because they are based on representative samples of students rather than on the entire population of students. Moreover, the collection of writing prompts used at each grade level is but a sample of the many tasks that could have been asked. As such, NAEP results are subject to a measure of uncertainty, reflected in the standard error of the estimates. The standard errors for the estimated scale scores and percentages in the figures and tables presented on this website are available through the NAEP Data Explorer.

Back to Top

NAEP Reporting Groups

Results are provided for groups of students defined by shared characteristics—for instance, gender, race/ethnicity, eligibility for free/reduced-price school lunch, students with disabilities, and students identified as English language learners. Based on participation rate criteria, results are reported for various student populations only when sufficient numbers of students and adequate school representation are present. The minimum requirement is at least 62 students in a particular group from at least five primary sampling units (PSU). However, the data for all students, regardless of whether their group was reported separately, were included in computing overall results. Explanations of the reporting groups are presented below.

Gender

Results are reported separately for males and females.

Race/Ethnicity

In compliance with standards from the U.S. Office of Management and Budget for collecting and reporting data on race/ethnicity, additional information was collected in 2011 so that results could be reported separately for Asian students, Native Hawaiian/Other Pacific Islander students, and students identifying with two or more races. Beginning in 2011, all of the students participating in NAEP were identified by school reports as one of the seven racial/ethnic categories listed below:

• White
• Black or African American
• Hispanic
• Asian
• Native Hawaiian or other Pacific Islander
• American Indian or Alaska Native
• Two or more races

Eligibility for Free/Reduced-Price School Lunch

As part of the Department of Agriculture's National School Lunch Program (NSLP), schools can receive cash subsidies and donated commodities in turn for offering free or reduced-price lunches to eligible children. Based on available school records, students were classified as either currently eligible for the free/reduced-price school lunch or not eligible. Eligibility for free and reduced-price lunches is determined by students' family income in relation to the federally established poverty level. Students whose family income is at or below 130 percent of the poverty level qualify to receive free lunch, and students whose family income is between 130 percent and 185 percent of the poverty level qualify to receive reduced-price lunch. For the period July 1, 2010 through June 30, 2011, for a family of four, 130 percent of the poverty level was $28,665 and 185 percent was $40,793. The classification applies only to the school year when the assessment was administered (i.e., the 2010–11 school year) and is not based on eligibility in previous years. If school records were not available, the student was classified as "Information not available." If the school did not participate in the program, all students in that school were classified as "Information not available."

Student with Disabilities (SD)

Results are reported for students who were identified by school records as having a disability. A student with a disability may need specially designed instruction to meet his or her learning goals. A student with a disability will usually have an Individualized Education Program (IEP) which guides his or her special education instruction. Students with disabilities are often referred to as special education students and may be classified by their school as learning disabled (LD) or emotionally disturbed (ED).

English Language Learners (ELL)

Results are reported for students who were identified by school records as being English language learners. (Note that English language learners were previously referred to as limited English proficient (LEP).)

Type of School

The national results are based on a representative sample of students in both public schools and nonpublic schools. Nonpublic schools include private schools, Bureau of Indian Affairs schools, and Department of Defense schools. Private schools include Catholic, Conservative Christian, Lutheran, and other private schools. However, in 2011 only Catholic school results met reporting standards.

Type of Location

NAEP results are reported for four mutually exclusive categories of school location: city, suburb, town, and rural. The categories are based on standard definitions established by the Federal Office of Management and Budget using population and geographic information from the U.S. Census Bureau. Schools are assigned to these categories in the NCES Common Core of Data based on their physical address. The locale codes are based on a school's proximity to an urbanized area (a densely settled core with densely settled surrounding areas).

Parental Education

Parents' highest level of education is defined by the highest level reported by eighth-graders and twelfth-graders for either parent.

Back to Top

Exclusion Rates

NAEP has established policies and procedures to maximize the inclusion of all students in the assessment. Every effort is made to ensure that all selected students who are capable of participating meaningfully in the assessment are assessed. While some students with disabilities (SD) and/or English language learners (ELL) can be assessed without any special procedures, others require accommodations to participate in NAEP. Still other SD and/or ELL students selected by NAEP may not be able to participate. Local school authorities determine whether SD/ELL students require accommodations or should be excluded because they cannot be assessed. The percentage of SD and/or ELL students who are excluded from NAEP assessments varies from one jurisdiction to another and within a jurisdiction over time. Read more about the potential effects of exclusion rates on assessment results.

See additional information about the percentages of students with disabilities and English language learners identified, excluded, and assessed in writing.

Back to Top

Statistical Significance

Differences between scale scores and between percentages that are discussed in the results on this website take into account the standard errors associated with the estimates. Comparisons are based on statistical tests that consider both the magnitude of the difference between the group average scores or percentages and the standard errors of those statistics. Throughout the results, differences between scores or between percentages are discussed only when they are significant from a statistical perspective.

All differences reported are significant at the 0.05 level with appropriate adjustments for multiple comparisons. The term "significant" is not intended to imply a judgment about the absolute magnitude or the educational relevance of the differences. It is intended to identify statistically dependable population differences to help inform dialogue among policymakers, educators, and the public.

Back to Top

Cautions in Interpretations

Users of this website are cautioned against interpreting NAEP results as implying causal relations. Inferences related to student group performance or to the effectiveness of public and nonpublic schools, for example, should take into consideration the many socioeconomic and educational factors that may also have an impact on performance.

The NAEP writing scale makes it possible to examine relationships between students' performance and various background factors measured by NAEP. However, a relationship that exists between achievement and another variable does not reveal its underlying cause, which may be influenced by a number of other variables. Similarly, the assessments do not reflect the influence of unmeasured variables. The results are most useful when they are considered in combination with other knowledge about the student population and the educational system, such as trends in instruction, changes in the school-age population, and societal demands and expectations.

Back to Top