Many of the nation’s schools are equipped with computers, and an increasing number are making digital tools an integral component of the learning environment.The National Assessment of Educational Progress (NAEP) is evolving to address this changing landscape through its transition to digitally based assessments. Since 2001, NAEP has been exploring new testing methods and question types that reflect the growing use of technology in education. The goal is to be paperless by the end of the decade.
NAEP has introduced a variety of new question and task types in the assessments, including scenario-based tasks from the Technology and Engineering Literacy (TEL) assessment, interactive computer tasks, and hybrid hands-on tasks to capture what students know and are able to do in more authentic or direct ways. In 2016, NAEP mathematics and reading assessments were piloted on tablets with an attached keyboard, a stylus, and earbuds and used new testing methods and question types. Some questions included multimedia, such as audio and video. Other questions allowed the use of digital tools (such as an onscreen calculator) to form a response, or engaged students in solving problems within realistic scenarios.
Below is a timeline showing NAEP’s exploration of digitally based assessments.Research in the use of new technologies is ongoing throughout the introduction of digitally based assessments.
New technologies are improving NAEP’s ability to offer accommodations to increase participation and provide universal access to students of all learning backgrounds, including students with disabilities and English language learners. In a digital environment, what used to be an accommodation for paper-based testing becomes a seamless part of universal design, available to all students. That means that things like adjusting font size, having test questions read aloud in English (text-to-speech), use of higher contrast to improve readability, and using a highlighter tool are available to all students during the assessment.
Maintaining trend lines (the ability to compare performance results from one year to another) is a priority. To do this, NAEP is using a multistep process to move from paper to digital technology to protect trend reporting. The process involves two stages of piloting before administering an operational digitally based NAEP assessment:
Stage 1. Adapt the paper-based questions for tablet delivery and pilot them in the same year as a main paper-based NAEP administration. Comparing results from paper and digitally based versions of the same assessment content will allow NAEP to establish a link between administration modes and help interpret performance trends across the transition from paper to digital delivery.
Stage 2. Develop new assessment questions and innovative question types and tasks that make use of digital technologies. This new digital assessment content will be gradually introduced into the assessment after first studying the effects of including these new items and item types.
The NAEP program is preparing to administer the 2017 NAEP mathematics, reading, and writing assessments to students in grades 4 and 8 throughout the nation on NAEP-provided tablets. Approximately 140,000 students at grades 4 and 8 will be assessed in mathematics. In reading, approximately 140,000 students will be assessed at grades 4 and 8, and in writing, about 22,000 at each grade will be assessed. For mathematics and reading, it will be the first time that NAEP will report national public school results of digitally based assessments.
Of the students who are sampled, most students will take the assessments on tablets with keyboards. A subset of students will take paper-and-pencil versions of the assessments. NAEP is administering the assessments on tablets and in paper booklets to evaluate any differences in student performance. Each student will be assessed in one format and one subject only. NAEP will also administer writing assessments on tablets with keyboards at grades 4 and 8.
Some students will be selected for pilot assessments on tablets for mathematics and reading at grades 4 and 8, and for civics, geography, and U.S. history at grade 8 only. Results from these pilot assessments will not be released but will be used to inform future NAEP assessments. Additional subjects will be administered on tablets in 2018 and 2019.
The transition of NAEP assessments from pencil and paper to touch-screen tablets incorporates cutting-edge learning technologies. At the beginning of each assessment, students will take a brief interactive tutorial designed to teach them about the system and the tools they use to take the assessment. Some of the on-screen interactive tools may be familiar to students, while others may be unique to the NAEP assessment. Each assessment begins with a short tutorial that demonstrates these tools so that students may use them effectively during the assessment. Some parts of the tutorials are the same across subjects, while other parts are specific to each subject. For example, because mathematics uses different tools at each grade, there are different tutorials for each grade in mathematics.
You can now experience the same tutorial, either by tablet or computer, shown to students at the beginning of a NAEP assessment.
Tutorials for the 2016 assessments are available through the link below, following an introductory video. You can access five subject- and grade level-specific tutorials.
During testing, all necessary equipment (tablets, ear buds, and administrative equipment) will be provided by NAEP. The only resources a school will need to provide will be space, desks or tables, and electricity. Students will be provided a tutorial that will help them understand how to use the equipment and tools and enter their responses. Non-cognitive questionnaires that record the learning experiences of students will also be administered to students on tablets.
Technology allows more students than ever to participate in NAEP because of universal design features for accessibility. DBAs will feature tools such as zooming and text-to-speech, where appropriate by subject. These tools will provide students with disabilities and English language learners the support they may need.
Technology and Engineering Literacy (TEL) Assessment
In 2014, the computer-based Technology and Engineering Literacy (TEL) assessment was administered for the first time. Students were challenged to work through computer simulations of real-world situations they might encounter in their everyday lives. New types of assessment questions were used in TEL, including short answer, multiple-choice, and scenario-based tasks. Simulations were presented as interactive, multimedia scenario-based tasks set in a variety of problem-solving contexts. In each scenario, students performed a variety of actions using a diverse set of tools to solve problems and meet goals. The tasks helped assessment specialists monitor and collect data about student actions as the students interacted with the tasks. View the results of this innovative assessment, and try the tasks that students took. View the TEL tutorial.
Writing Computer-Based Assessment (WCBA)
The NAEP writing assessment, administered in 2011 at grades 8 and 12, included multimedia as well as word processing tools. The assessment included 22 writing tasks at grade 8 and 22 tasks at grade 12. Writing tasks were presented to students in a variety of ways, including text, audio, photographs, video, or animation on the computer. Students composed their responses on laptop computers.
Students were randomly assigned two writing tasks and had 30 minutes to complete each of the tasks. Before being presented with the first task, students were shown a tutorial to familiarize them with the way material is presented on the computer screen and to show them how to use the custom-developed software program provided in the assessment. Students completed their writing tasks on laptop computers provided by NAEP, using software similar to common word-processing programs. See what tools were used in the 2011 computer-based writing assessment.
In addition, NAEP conducted a study of computer-based writing at grade 4 in 2012. Lessons learned from this study provide insights into some of the challenges that were encountered and solutions that were applied for the administration. Find out what lessons were learned from the administration of the grade 4 writing computer-based assessment.
Science Interactive Computer Tasks (ICTs)
In 2009, interactive computer tasks (ICTs)--scenario-based tasks--were administered as part of the science assessment. ICTs challenge students to solve scientific problems and perform experiments, often by simulation. They provide students more opportunities than a paper-and-pencil assessment to demonstrate skills involved in doing science without many of the logistical constraints associated with a natural or laboratory setting. Explore the science ICTs.
Digitally based assessments are not a new venture for NAEP. NAEP began investigating the use of technology in assessments in 2001. Research in the use of new technologies is ongoing throughout the introduction of digitally based assessments and will continue to be part of the plan for the next iteration of assessments using new technologies.
Read more about the three early field investigations in NAEP's digitally based assessment: