- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
Highlights of Performance of Fourth-Grade Students on the 2012 NAEP Computer-Based Pilot Writing Assessment
December 8, 2015
Author: Sheida White, NCES; Young Yee Kim, AIR; Jing Chen, NCES; Fei Liu, MacroSys.
Download the complete report in a PDF file for viewing and printing.
(1.32 MB)In 2012, the National Center for Education Statistics (NCES) administered the National Assessment of Educational Progress (NAEP) computer-based pilot writing assessment at grade 4, providing students with a laptop computer and asking them to complete two 30-minute writing tasks. NCES conducted the current study to answer the following two questions about the assessment:
- Can fourth-graders fully demonstrate their writing skills on the computer?
What factors are related to fourth-graders’ writing performance on the computer?
Can fourth-graders fully demonstrate their writing skills on the computer?
1. Writing performance
The 2012 NAEP computer-based pilot writing assessment proved challenging for fourth-grade students as a group, especially low-performing students. More than two-thirds (68 percent) of fourth-graders’ responses received scores in the bottom half of the 6-point scoring scale used in this study. These responses were not consistent in their use of grammar and mechanics, provided insufficient supporting statements, and reflected little or no awareness of audience and purpose.
While fourth-graders had similar overall average scores on the 2012 NAEP computer-based writing assessment and on a paper-based pilot writing assessment administered in 2010, an analysis of 15 writing tasks common to both assessments revealed a different story. The average score of high-performing fourth-graders was higher on the computer than on paper, whereas low- performing students did not appear to benefit from using the computer. This finding suggests that low-performing fourth-graders did not fully demonstrate their writing ability on the computer in the 2012 NAEP computer-based pilot writing assessment, and that the use of the computer may have widened the writing achievement gap.
Since typing speed is relevant to the amount of text produced, it was assessed in a usability study conducted in 2011. Fourth-graders’ average typing speed was 12 words per minute, compared to 30 words per minute for eighth-graders.
While fourth-graders had similar overall average scores on the 2012 NAEP computer-based writing assessment and on a paper-based pilot writing assessment administered in 2010, an analysis of 15 writing tasks common to both assessments revealed a different story. The average score of high-performing fourth-graders was higher on the computer than on paper, whereas low- performing students did not appear to benefit from using the computer. This finding suggests that low-performing fourth-graders did not fully demonstrate their writing ability on the computer in the 2012 NAEP computer-based pilot writing assessment, and that the use of the computer may have widened the writing achievement gap.
2. Text length
The average number of words produced by fourth-graders, as a whole, was smaller on the 2012 computer-based pilot assessment than on the 2010 paper-based pilot assessment (110 vs. 159). In terms of ability levels, low-performing fourth-graders produced fewer words (60) than middle- and high-performing fourth graders (104 and 179, respectively) on the computer-based assessment. To place these numbers in a larger context, consider that on the 2011 NAEP writing assessment, eighth- and twelfth-graders produced more words on the computer than on paper (300 vs. 155 at grade 8; 389 vs. 235 at grade 12).
Since typing speed is relevant to the amount of text produced, it was assessed in a usability study conducted in 2011. Fourth-graders’ average typing speed was 12 words per minute, compared to 30 words per minute for eighth-graders.
3. Relationship between measured outcomes
Data from the computer-based pilot assessment demonstrated that there is a positive relationship between text length and fourth-graders’ writing performance, such that the longer a student’s response, the higher the score it received.What factors are related to fourth-graders’ writing performance on the computer?
1. Use of editing tools
In the computer-based pilot assessment, students’ actions on the computer were captured and analyzed for the lowest performing 20 percent of students, the highest performing 20 percent of students, and the middle-performing 60 percent of students. Compared to the middle- and high-performing students, a higher percentage of low-performing students:- used key presses less frequently;
- did not use the spellcheck function;
- did not accept any automated spelling corrections; and
- used the backspace key less frequently to edit their work.
Overall, students who accepted spelling corrections and used the backspace key more often were also likely to write longer responses.
2. Prior exposure to writing on the computer
Prior exposure to writing on the computer provides insight into fourth-grade students’ acquisition of keyboarding and word processing skills, which can impact writing performance. Internet access at home and using the internet to look for information are two such types of prior exposure.
Internet access at home: The 2012 fourth-grade writing data indicate that students with access to the Internet at home were more likely than those without access to:
There is an association between each of the above student actions and characteristics and students’ writing scores, and, directly, between students’ access to the Internet at home and their writing scores. For example, in 2012, 94 percent of high-performing students had access to the Internet at home, compared to 52 percent of low-performing students. Information about other factors related to writing performance can be found in the full report.
Using the Internet to look for information to include in their writing: Students who used the Internet to look for information to include in their writing were more likely to:
When asked about their preferred mode of writing, fourth-graders with the following types of prior exposure and demographic characteristics were more likely to say that they would prefer to take a writing assessment on paper:
Internet access at home: The 2012 fourth-grade writing data indicate that students with access to the Internet at home were more likely than those without access to:
- write longer responses;
- use the spellcheck tool more often;
- use the thesaurus tool more often; and
- use bold and italics for emphasis more often.
Students with access to a computer at home were also more likely to prefer to write on the computer than on paper, compared to those without access.
The percentage of fourth-graders without access to the Internet at home was higher for Black students, Hispanic students, students eligible for free or reduced-price lunch, English language learners, and students with a disability.
The percentage of fourth-graders without access to the Internet at home was higher for Black students, Hispanic students, students eligible for free or reduced-price lunch, English language learners, and students with a disability.
There is an association between each of the above student actions and characteristics and students’ writing scores, and, directly, between students’ access to the Internet at home and their writing scores. For example, in 2012, 94 percent of high-performing students had access to the Internet at home, compared to 52 percent of low-performing students. Information about other factors related to writing performance can be found in the full report.
Using the Internet to look for information to include in their writing: Students who used the Internet to look for information to include in their writing were more likely to:
- write longer responses;
- accept automated spelling corrections; and
- use the backspace key to edit their work.
There is a positive relationship between each of these student actions and students’ writing scores, and, directly between students’ use of the Internet to look for information to include in their writing and their writing scores. For example, 25 percent of high-performing fourth-graders reported they never looked for information on the Internet to include in their writing, compared to 32 percent of low-performing fourth-graders.
3. Preferred mode of writing
When asked about their preferred mode of writing, fourth-graders with the following types of prior exposure and demographic characteristics were more likely to say that they would prefer to take a writing assessment on paper:
- students with no access to the Internet at home;
- students with less experience looking for information on the Internet to include in their writing;
- students with less experience writing to friends and family using the Internet;
- male students;
- English language learners; and
- students with a disability
There is an association between each of these student characteristics and students’ writing scores, as well as between students’ preferred mode of writing and their writing scores. For example, in 2012, 11 percent of high-performing fourth-graders preferred writing on paper, compared to 27 percent of low-performing fourth-graders.
Conclusion
Whether or not the results of the NAEP 2012 computer-based pilot writing assessment offer a true measure of all students’ writing abilities has yet to be determined. The data indicate that fourth-graders, as a group, produce considerably shorter texts on the computer than on paper—a measured outcome that is related to writing performance. This suggests that when a student is only capable of producing a few sentences per hour on the computer in a NAEP-like assessment, he or she is likely to have difficulties with composing a whole essay. Part of that difficulty may have to do with simply finding the letters on the keyboard and the editing tools on the menu—taking time and cognitive resources away from critical thinking processes, such as finding the right word, varying sentences, and providing strong details and supportive evidence in light of the given audience and purpose of writing. The NAEP data also indicate that limited keyboarding and word processing skills may be associated with prior exposure to writing on the computer.
Whether or not the results of the NAEP 2012 computer-based pilot writing assessment offer a true measure of all students’ writing abilities has yet to be determined. The data indicate that fourth-graders, as a group, produce considerably shorter texts on the computer than on paper—a measured outcome that is related to writing performance. This suggests that when a student is only capable of producing a few sentences per hour on the computer in a NAEP-like assessment, he or she is likely to have difficulties with composing a whole essay. Part of that difficulty may have to do with simply finding the letters on the keyboard and the editing tools on the menu—taking time and cognitive resources away from critical thinking processes, such as finding the right word, varying sentences, and providing strong details and supportive evidence in light of the given audience and purpose of writing. The NAEP data also indicate that limited keyboarding and word processing skills may be associated with prior exposure to writing on the computer.
Download the complete report in a PDF file for viewing and printing.
(1.32 MB)
NCES 2015-119 See the entry in the NCES database for contact and ordering information, and for links to similar topics.
Last updated 15 December 2015 (MS)
