• Introduction
• Summary of Findings
• References

The National Assessment of Educational Progress (NAEP) long-term trend writing assessment provides an important picture of students' progress over time because it compares performance on the same writing tasks, administered in identical fashion to comparable samples of students and yielding comparable scores. There have been six national assessments of writing conducted during the school years ending in 1984, 1988, 1990, 1992, 1994, and 1996. The 1996 assessment included the same set of 12 writing tasks that had been administered in the five previous assessments. Each of these trend assessments was administered to nationally representative samples of students in grades 4, 8, and 11.

Over the past three decades, many teacher educators and classroom teachers have been emphasizing the writing process. The writing process approach focuses on the iterative nature of writing, in which writers plan, write, and revise their ideas in several drafts before a final version is produced. It is during the revision or editing stages of this process that writers focus on correcting grammatical and mechanical errors. Grammatical and mechanical correctness is not viewed as an end in and of itself, but eliminating these errors is an important part of improving the final draft. This report focuses on what changes, if any, have occurred in student writing between 1984 and 1996, the period examined by the NAEP long-term trend writing assessment.

Results of the 1996 long-term trend writing assessment are reported in two publications. This report describes two aspects of writing for which change has been measured since 1984: writing fluency, as determined by holistic scoring; and mastery of the conventions of written English (spelling, punctuation, grammar) as determined by mechanics scoring. This report is supplementary to NAEP 1996 Trends in Academic Progress (Campbell, Voelkl, and Donahue 1997), the main report for the NAEP long-term trend assessment. That document reports trends in writing scores since 1984 as determined by primary trait scoring. This report presents the results of the holistic scoring of a subgroup of 4 of the 12 writing tasks, and the mechanics scoring of 2 of these 4 tasks.

The NAEP long-term trend writing assessments discussed here and in NAEP 1996 Trends in Academic Progress should not be confused with the main NAEP writing assessments. The long-term trend writing assessment was begun in 1984, and has presented students with the same writing tasks in the five ensuing assessments. These writing tasks are completely different from the prompts in the main NAEP assessment.¹ The use of different writing prompts, as well as other procedural differences, precludes direct comparison of the results of the long-term trend assessment discussed here with those of the main assessment.

In order to assess students' abilities to write in a variety of formats and genres, the NAEP long-term trend writing assessment asks them to respond to several different tasks in each of three types of writing:

• informative tasks ask students to write descriptions, reports, and analyses;
• persuasive tasks ask students to write convincing letters and arguments; and
• narrative tasks ask students to write stories.

• primary trait scoring, as described in NAEP 1996 Trends in Academic Progress, measures success
• in accomplishing the specific task (e.g., writing persuasively);
• holistic scoring, reported here, measures fluency in a subgroup of 4 of the 12 tasks; and
• mechanics scoring, also reported here, measures conventions of written English using a subgroup of two of the four holistically scored tasks.

However, there are other aspects of writing that it is also important to assess. For instance, general writing quality or fluency—the student's capacity to organize and develop a written piece, to use correct syntax, and to observe the conventions of standard written English—is important. These aspects of written communication, taken together, are what holistic evaluation of writing addresses.²

The long-term trend writing assessment consisted of three distinct parts: primary trait, holistic, and mechanics scoring criteria.

• First, all 12 of the long-term trend writing tasks were scored using primary trait scoring criteria. The results of this are reported in NAEP 1996 Trends in Academic Progress.³
• Next, a subgroup of four of these tasks was scored holistically—two tasks at each grade level (figure A). Different scoring guides were used for holistic scoring of narrative, informative, and persuasive tasks.
• Lastly, to gain information about students' mastery of the conventions of written English, a subgroup of two of the holistic tasks was scored for mechanics—one at each grade level (figure A). For mechanics scoring, the same criteria were used to evaluate all tasks.

Holistic scoring is the most commonly used method for evaluating students' writing performance in the United States today. Holistic scoring for NAEP focuses on the writer's fluency in responding to a task relative to the performance of other students at that grade level (Cooper 1977). Fluency reflects a writer's facility with language both in terms of the development and organization of ideas and in the use of syntax, diction, and grammar. Holistic scoring methods were specifically designed to assess writing fluency. The underlying assumption of holistic scoring is that the whole piece of writing is greater than the sum of its parts. In holistic scoring, readers do not make separate judgments about specific aspects of a written response, but rather consider the overall effect, rating each paper on the basis of its general fluency.

In the NAEP long-term trend assessment, responses to four tasks are scored holistically, two tasks at each of the three grades (the same two tasks are administered at both 8th and 11th grades). The characteristics of general fluency are assessed on a six-point scale. In order to make comparisons of students' writing fluency across all 6 years of the assessment, all papers from the previous years were scored holistically, along with all of the 1996 papers. For each year, approximately 1,200papers⁴ from each grade are scored.

As is typical with all holistic scorings, raters are trained on a particular task immediately before scoring the papers written in response to that task. For each task, the papers from all years are randomly mixed and then assigned one of six scores. To detect changes in fluency from one assessment to another, the percentages of papers from each year within a given score category are compared. The comparisons reported here are for the first or base year and the current year, as in previous reports (e.g., Campbell et al. 1996).

Thus, while primary trait scoring is based on specific constant criteria and so permits year-to-year and grade-to-grade comparisons, holistic scoring allows within-grade comparisons of relative fluency over all years according to contemporaneous criteria.

Another set of analyses, applied to papers written for two of the tasks, focused on the mechanics of students' writing. While error counts do not fully reflect a writer's fluency and competency, many educators, policymakers, and parents are interested in the kinds of surface errors students make as they write (Shaughnessy 1977). Students' mastery of the sentence-level and word-level conventions of English, as well as their use of correct spelling and punctuation, was examined. In order to examine changes in students' success in using the conventions of written English, one task at each grade was selected for a detailed analysis of writing mechanics, including spelling, word choice, punctuation, and syntactic errors.

Modest improvements in writing fluency between 1984 and 1996 were seen in fourth- and eighth-grade students' essays.

At grade 4, holistic scoring of the persuasive task "Spaceship" showed no overall increase in students' writing fluency between 1984 and 1996. However, there was a significant increase in the percentage of papers rated in the upper half of the holistic scale (that is, papers receiving a rating of 4, 5, or 6). Fourth-graders writing for the narrative task "Flashlight" showed an increase in the percentage of papers receiving a rating of 4, but no change in the overall rating of performance between 1984 and 1996.

Eighth-graders' essays showed improvement in 1996 on both of the tasks analyzed holistically, the informative task "Food on the Frontier" and the persuasive task "Recreation Opportunities." At grade 11, no change was seen in writing fluency on either of these tasks when comparing 1996 papers to those written in 1984.

Differences in the use of grammar, spelling, and punctuation conventions between 1984 and 1996 were primarily in the direction of improvement at grades 8 and 11. For both 8th- and 11th-graders, the percentage of awkward sentences and punctuation error rates decreased, even as papers contained more sentences and more words. But there was a more mixed picture at grade 4: fourth-graders showed a decrease in one kind of error but an increase in three other kinds of errors.

English language conventions were examined in papers written in 1984 and 1996 for the task "Spaceship" at grade 4 and for the task"Recreation Opportunities" at grades 8 and 11. A subsample of papers from 1984 and 1996 had been coded by experts so that students' control of the conventions of the English language could be analyzed. Overall, these indicators of performance at the three grades suggest that there were some changes in students' mastery of English language conventions between 1984 and 1996.

The number of words and sentences written by 8th- and 11th-grade students increased between 1984 and 1996. Over the same period, however, there was no change in the rate of errors (number of errors per 100 words) in all three grades.

While there were increases in percentages of sentence fragments in 4th- and 8th-grade papers, there were declines in the percentage of awkward sentences in 8th- and 11th- graders' papers compared to 1984.

At all grades, the percentage of spelling errors remained unchanged, comparing 1984 to 1996. The percentage of incorrect word choices was unchanged in grades 8 and 11 but increased at grade 4. At grades 8 and 11, the punctuation error rate decreased while the number of punctuation marks used per paper increased.

Figure B provides a synopsis comparing student use of grammar, punctuation, and spelling conventions in 1984 and 1996. Measures in the first section are characteristics desirable for these NAEP long-term trend writing tasks. An increase in prevalence in 1996 compared to 1984is desirable, and "increase" is shown in bold text. Decreases in 1996 compared to 1984 are undesirable, and those cells are black with white text. The middle section contains characteristics that are neutral; that is, changes in these measures are of interest, but there is no clear advantage or disadvantage to either increase or decrease. The lower section contains measures of writing error, with the notation of increase or decrease. In this section, decreases are desirable and are in bold text, while increases (undesirable) are shown in black cells with white text. Throughout the table, empty cells indicate that no statistically significant change occurred in 1996 compared to 1984.

Figure B.—Average change from 1984 to 1996 in writing mechanics measures in this report

Of the measures of students' control of writing reported here, at grade 4, one measure of writing error showed improvement (that is, the percentage of sentences with end mark errors declined), while three showed increases in error rate (that is, the prevalence of sentence fragments, incorrect word choice, and comma/dash errors increased). At grade 4, there was no change in most characteristics reported here. At grades 8 and 11, students were writing more in 1996, although the rate at which they used more sophisticated sentence constructions had decreased. At grade 8, the good news was that two desirable characteristics improved and three errors decreased, and only two changes indicated problems (that is, the use of complex sentence structures decreased while sentence fragments increased). At grade 11, improvement occurred in two desirable characteristics and for three types of errors, while only one change in a desirable characteristic indicated a problem (that is, the use of complex sentence structures decreased).

Students at all three grade levels wrote at least as fluently in 1996 as they did in 1984, while students at grade 8 demonstrated improved fluency on the informative and persuasive tasks.

When writing mechanics are considered, the overall picture is of improvement in grades 8 and 11, but there are several declines at grade 4. In the 8th and 11th grades, students wrote more, as indicated by the increase in the number of words and sentences in their responses, while demonstrating no increase in the number of errors per 100 words. A summary of the measures can be seen in figure B.

Thus, increased instructional emphasis on writing processes over the 12 years between 1984 and 1996 appears associated with modest improvements in students' mastery of the conventions of written English at grades 8 and 11. During this time period, the overall fluency of 8th-graders' writing has also improved. It appears that the process approach to writing, in which planning, writing, and revision through several drafts are practiced, gives students the opportunity to write more and to employ editing strategies, which in turn affords them the opportunity to improve their mastery of the writing conventions reported here.

Data source: The National Assessment of Educational Progress (NAEP) 1996 Long-Term Trend Assessment. For technical information, see the complete report: Ballator, N., Farnum, M., and Kaplan, B. (1999). NAEP 1996 Trends in Writing: Fluency and Writing Conventions (NCES 1999-456). For additional details about long-term trend methodology, see Allen, N.L., Carlson, J.E., and Zelenak, C.A. (1999). The NAEP 1996 Technical Report (NCES 1999-452). Author affiliations: N. Ballator, M. Farnum, and B. Kaplan, Educational Testing Service. For questions about content, contact Arnold A. Goldstein (arnold.goldstein@ed.gov). To obtain the complete report (NCES 1999-456), call the toll-free ED Pubs number (877-433-7827), visit the NCES Web Site (http://nces.ed.gov), or contact GPO (202-512-1800).