• Overview
• Focused Holistic Writing
• Narrative Writing
• Conclusion
• References

The National Assessment of Educational Progress (NAEP) 1998 Writing Assessment measured student writing performance at the 4th, 8th, and 12th grades. Scoring guides for each grade allowed scorers to objectively evaluate students' work. This issue of NAEPfacts includes a 4th-grade narrative scoring guide, along with samples of student work at each of six levels of performance on the scoring guide.

The NAEP Writing Framework, developed by the National Assessment Governing Board (NAGB 1997, pp. 5-11), set six overarching goals for the NAEP 1998 Writing Assessment:

• Students should write for a variety of purposes: narrative, informative, and persuasive.
• Students should write on a variety of tasks and for many different audiences.
• Students should write from a variety of stimulus materials and within various time constraints.
• Students should generate, draft, revise, and edit ideas and forms of expression in their writing.
• Students should display effective choices in the organization of their writing. They should include detail to illustrate and elaborate their ideas, and use appropriate conventions of written English.
• Students should value writing as a communicative activity.

Scoring guides were developed for each writing purpose. The guides established six levels of student performance for each writing purpose, ranging from "Unsatisfactory" to "Excellent."² This issue of NAEPfacts includes the complete text of the fourth-grade "narrative" scoring guide (figure 1), the complete text of a narrative prompt asking students to write a story about a magic castle, examples of student writing at each of the six levels in response to the "magic castle" prompt, and a discussion of how the scoring guide applies to the six student writing samples.

The scorers of the NAEP 1998 Writing Assessment used a scoring method described as "focused holistic scoring." This approach combines holistic and "primary trait" scoring. A strict holistic approach to the scoring of writing treats a writing task as a "springboard" for writing. A particular writing task is given to students as a stimulus to engage them and inspire them to write, and students' responses are scored in terms of the overall writing quality. The "primary trait" method of scoring writing, on the other hand, is concerned with how well students respond to a specific topic. For example, if students are asked to write about whether they like adventure movies, students who do not address the topic of adventure movies will receive lower scores than those who do.

The "focused holistic scoring" approach used by NAEP, as with all holistic approaches, requires scorers to rate the overall quality of the writing, regardless of how students choose to respond to specific aspects of a given topic. In contrast to some holistic approaches to writing scoring that offer very general guidelines, NAEP scorers were given detailed scoring guides that focused their attention on specific characteristics of students' writing (organization, development, syntax, and mechanics). In this sense, the "traits" of writing now at issue for NAEP scoring of writing have shifted from topic-related traits of student responses to traits associated with overall quality of writing.

The basic assumptions of the NAEP focused holistic scoring approach are that

• Each of the factors involved in writing is related to all others and that no one factor can be separated from the others.
• A writer is entitled to make some mistakes, given the 25-minute time limit, the lack of recourse to a dictionary, and the lack of time for reviewing and editing.
• Scorers should read each response as a whole-without focusing on each mistake (but still being aware of them)-to judge the level of writing ability demonstrated by the student.
• After thorough training on the scoring of responses written on a given task, scorers should quickly read an entire response and assign a score based on the total impression conveyed by the response.
• Scorers should ignore their personal standards of what constitutes good writing and embrace the criteria of the scoring guide.
• Scorers should read supportively rather than critically.

Narrative writing involves the production of stories or personal essays. Practice with these forms helps writers to develop a facility for spontaneous and colloquial language. Also, informative and persuasive writing can benefit from many of the strategies used in narrative writing. For example, there must be an effective ordering of events when relating an incident as part of a report.

Sometimes narrative writing contributes to an awareness of the world as the writer creates, manipulates, and interprets reality. Such writing-whether fact or fiction, poem, play, or personal essay-requires close observation of people, objects, and places. Further, this type of writing fosters creativity, imagination, and speculation by allowing the writer to express thoughts and then stand back, as a more detached observer might, and grasp more fully what is being felt and why. Thus, narrative writing offers a special opportunity to analyze and understand emotions and actions.

Fourth-grade students were given a number of narrative topics on the NAEP 1998 Writing Assessment. One of these topics asked students to write about a child encountering a castle that appears overnight as if by magic. The complete text of this topic is given below:

     One morning a child looks out the window and discovers that a huge castle has appeared overnight. The child rushes outside to the castle and hears strange sounds coming from it. Someone is living in the castle!

     The castle door creaks open. The child goes in.

     Write a story about who the child meets and what happens inside the castle.

In the imaginative stories written for this topic, "Castle," characters sometimes appear and disappear rather suddenly. Students who received ratings in the upper half of the six levels on the scoring guide ("Sufficient" or better) were able to weave coherent stories, making effective use of suspense and surprise.

The "Unsatisfactory" rating was given to 2 percent of the responses to this topic. As the scoring guide (figure 1) indicates, responses at this level tended either to be so brief that they did not develop a story at all, or to be hard to understand throughout. In the response shown, the student only paraphrases the topic.

Student response: One morning a child looks out a window and sees that a castle appears overnight. She runs to the castle and inside she meet a giant. The giant let her in. He asked her her name. He was married to another giant.

The "Insufficient" rating was given to 12 percent of the responses to this topic. In "Insufficient" responses, students produced only the beginning of a story, wrote very disorganized stories, or wrote responses that were understandable only in part. In the response shown, the student begins to tell a story, introducing a new character, the "giant," but does not progress beyond that point.

Student response: He saw died fish when he walked in the door. Than he herd something a bat fly and turn in to a vampire. He ran out of the door yelling for help while running out the door. Then the vampire turned in to a bat again. And the boy never went there again. Then a year later a girl went there and opened the door and seen died fish at the door and seen a bat flying and than therd in to a vampire and yelled out the door yelling for help. And than the vampire turned in to a bat again. And it gos on on on on on on and on on on again.

The "Uneven" rating was given to 31 percent of the responses to this topic. In such responses, students attempted to tell an entire story, but the attempt was incomplete or disorganized. In the "Uneven" response shown, there is some dramatic action ("than he herd something a bat fly and turn in to a vampire"). That action, however, is repetitive, as the events are not connected to form a coherent story: "And than the vampire turned in to a bat again. And it gos on on on on on on and on on on again."

Student response:One day a 13 year old boy woke up and found beautiful castle with a purple, pink, red, blue, orange and yellow rainbow. He decided to get up. Then he got dressed and went to see what was in the castle. He walked to the door and knocked nobody answered so he knocked again still no answer.

The "Sufficient" rating was given to 38 percent of the responses to this topic. In such responses, students told complete stories that were organized and clear, but lacking in detail. In the "Sufficient" response, the student provides a clear but bare plot. He or she includes the vivid detail of the colors of the rainbow in the first sentence, but uses detail sparingly beyond that. Though the story does not conclude, enough action occurs that most of the development is clear. The simple, but essentially clear and correct, sentence structure and vocabulary are typical of responses at this level.

Student response: First He sees a dark room filled with object some big and some small some short some tall. As he is reaching for the light switch something grabs his hand and turns it on for him. As it turns on the boy sees a woman so beautiful she captures his eyes. They look so much alike. They walk through the castle telling each other stories about each other showing each other pictures from there life. The girl says she had a long lost brother from long ago. She says he looks like this and showed him the picture. The boy says he has a picture just like that of himself Then the girl realizes the boy in the picture is her long Lost Brother.

The "Skillful" rating was given to 14 percent of the responses to this topic. In such responses, students used details to develop their stories in parts of the response. They provided a good structure to their stories, though with occasional lack of transitions. In the sample "Skillful" response, the plot occasionally shifts abruptly, as when the boy "sees a woman" who looks like him and they suddenly start to "walk through the castle." Though the ending is concise, the student ties up the story with the revelation "Then the girl realizes the boy in the picture is her long Lost Brother."

Student response: "Wow a castle!" said John. He had know clue of how it got here or where it came from? He walked inside and found that it was rather damp. He wandered around until finally he saw someone. This person didn't look normal. He was dressed in royalty with a purple cape and a crown of jewels. Then the person spoke out. "There you are you're supposed to be training right now." John had know clue what he was talking about. Suddenly he thought of something, was this the King of the castle? He finally got the nerve to ask a question. He asked "Who are you." He answered "I'm the King." John was shocked. Then the King told him to get on his armor. John thought and thought. Then he knew what he was talking about. He thought he was a knight. John thought again. If he was to be a knight then he would never see his family again. Then he thought of his older sister, Jennifer. He decided to be a knight. After about 2 months he- finally was knighted. He fought many dragons and man. He finally died but is still a legend today.

The "Excellent" rating was given to 3 percent of the responses to this topic. Such responses may have excelled through good development of plot, characters, or dialogue. In the response shown, the student uses dialogue effectively, develops characters, and provides a coherent plot. The student shows good control of language for a fourth-grader and includes vivid details about appearance-"He was dressed in royalty with a purple cape and a crown of jewels."

Greenwald, E.A., Persky, H.R., Campbell, J.R., and Mazzeo, J. (1999). The NAEP 1998 Writing Report Card for the Nation and the States (NCES 1999-462). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office.

Moskal, B.M. (2000). Scoring Rubrics: What, When and How? Practical Assessment, Research & Evaluation, 7(3). Available: http://ericae.net/pare/getvn.asp?v=7&n=3

National Assessment Governing Board. (1997). Writing Framework and Specifications for the 1998 National Assessment of Educational Progress. Washington, DC: Author. Available: http://www.nagb.org