Student Performance

The Writing Computer-Based Assessment (WCBA) platform was updated following the usability study for the Grade 4 WCBA pilot assessment. The pilot investigated students’ ability to compose typed responses to different prompts under 30- and 20-minute conditions.

About 80 percent of students were given two 30-minute prompts. Responses were scored with a six-point rubric.

• Almost 39 percent of the responses received a score of 1 or 2. These responses showed low to marginal writing skills. They provided insufficient supporting statements and were not consistent in their use of proper grammar and mechanics.
• About 47 percent of student responses received a score of 3 or 4, indicating that they demonstrated some control of writing, most of their ideas were on topic, and they used simple organizational strategies in most of their writing.
• Approximately 14 percent of the responses, under the 30-minute conditions, demonstrated competent or effective writing skills and received a score of 5 or 6. These responses showed more control in their writing, were fairly balanced, and reflected some awareness of audience and voice as well as control of grammar and mechanics.

About 20 percent of students were given three 20-minute prompts. Given the quality of writing that students performed under this time constraint, NAEP created a 4-point rubric to better meet the range of performance. The scale is described below:

• Score: 4
• Rubric Descriptor: Responses demonstrate competent skill in responding to the writing task. Elements of the response are generally controlled and clearly support the writer’s purpose and audience.
  
  
• Score: 3
• Rubric Descriptor: Responses demonstrate adequate skill in responding to the writing task. Some elements are controlled and support the writer’s purpose and audience.
  
  
• Score: 2
• Rubric Descriptor: Responses demonstrate uneven skill in responding to the writing task. Some elements are not controlled and unevenly support the writer’s purpose and audience.
  
  
• Score: 1
• Rubric Descriptor: Responses demonstrate marginal to insufficient skill in responding to the writing task. Elements are seldom controlled and provide almost no support for the writer’s purpose audience.

Under this condition, 44 percent of responses received a score of 1 and 6 percent of responses received a score of 4.

Scorers observed that students’ writing in the 20-minute condition seemed less well-developed and more list-like. The time constraint may have affected the students’ ability to provide complete and well-organized responses.

During the usability study, students were asked about their exposure to computers at home and in school. Students’ experience writing with computers may have had an influence on the length of their responses to the computer-based writing tasks.

As specified by the NAEP writing framework, students were asked to compose responses that fulfilled one of three purposes: to persuade, to explain, or to convey experience.

On 30-minute tasks that asked students to persuade:

• Approximately 48 percent of student responses received a 1 or a 2
• Approximately 43 percent of student responses received a 3 or a 4
• Approximately 10 percent of student responses received a 5 or a 6

On 30-minute tasks that asked students to explain:

• Approximately 36 percent of student responses received a 1 or a 2
• Approximately 49 percent of student responses received a 3 or a 4
• Approximately 15 percent of student responses received a 5 or a 6

On 30-minute tasks that asked students to convey experience:

• Approximately 36 percent of student responses received a 1 or a 2
• Approximately 47 percent of student responses received a 3 or a 4
• Approximately 17 percent of student responses received a 5 or a 6

Under the 30-minute prompt condition, student responses with fewer than 100 words received scores from 1-6, as did student responses with more than 500 words.

The average word count for responses that received a score of 6 was 231, while the average word count of responses that received a 1 was 56.

The average word count for the 30-minute prompt condition was 110. The lowest performing 20% of students typed 60 words on average, while the highest performing 20% of students typed 179 words on average.

During the usability study, students were given a five-minute typing test. Students typed on average 12 words per minute, with a range of 4 to 25 words. The average accuracy rate was 83 percent.

In 2010, fourth-grade students wrote on average 159 words per response under the paper-and-pencil mode (when given two 25-minute prompts). The number of words fourth-grade students typed on the computer in 2012 was about 110 (for two 30-minute prompts).

Under the 30-minute prompt condition, 69 percent of the students typed fewer than 1000 keystrokes.

• Approximately 91 percent of the lowest performing students typed 1000 keystrokes or less.
• Approximately 29 percent of the highest performing students typed 1000 keystrokes or less.

Twenty-nine percent of students typed 1001-2000 keystrokes.

• Approximately 9 percent of the lowest performing students typed 1001-2000 keystrokes.
• Approximately 63 percent of the highest performing students typed 1001-2000 keystrokes.

Fifty-five percent of students used the backspace key 1-100 times under the 30-minute prompt condition.

• Approximately 65 percent of the lowest performing students used the backspace key 1-100 times.
• Approximately 38 percent of the highest performing students used the backspace key 1-100 times.

Thirty-three percent of students used the backspace key 101-200 times.

• Approximately 26 percent of the lowest performing students used the backspace key 101-200 times.
• Approximately 43 percent of the highest performing students used the backspace key 101-200 times.

Twelve percent of students used the backspace key more than 201 times.

• Approximately 9 percent of the lowest performing used the backspace key more than 201 times.
• Approximately 20 percent of the highest performing students used the backspace key more than 201 times.

Under the 30-minute prompt condition, 39 percent of students used the spell check tool at least four times.

• Approximately 30 percent of the lowest performing students used the spell check at least four times.
• Approximately 47 percent of the highest performing students used the spell check at least four times.

Twenty-five percent of students did not use the spell check tool.

• Approximately 35 percent of the lowest performing students did not use the spell check tool.
• Approximately 17 percent of the highest performing students did not use the spell check tool.