Test Development

PIRLS 2016 included booklets containing reading passages and assessment items, as well as self-administered background questionnaires for principals, teachers, and students. The ePIRLS assessment consisted of simulated web-based tasks, associated assessment items, and background questions for students.

PIRLS Assessment Booklets

The assessment instruments include 4th-grade-level stories and informational texts collected from several different countries. Students are asked to engage in a full repertoire of reading skills and strategies, including retrieving and focusing on specific ideas, making simple and more complex inferences, and examining and evaluating text features. The passages are followed by open-ended and multiple-choice format questions about the text.

The 2016 assessment consisted of 15 booklets and 1 reader (presented in a magazine-type format with the questions in a separate booklet). The assessment was given in two 40-minute parts with a 5- to 10-minute break in between. Each of the booklets contained two parts—one block containing an literary experience passage and associated test items, and one block containing an informational passage and items—and each block occurred twice across the 15 total booklets. As the entire assessment consists of 12 blocks of passages and items, using different booklets allows PIRLS to report results from more assessment items than can fit in one booklet, without making the assessment longer. To provide good coverage of each skill domain, the test items developed require about 8 hours of testing time. However, testing time is limited to 80 minutes per student by clustering items in blocks and randomly rotating the blocks of items throughout the student test booklets (assessment booklets were randomly assigned to students within each sampled class). As a consequence, no student receives all items (there were a total of 175 items on the 2016 assessment), but each item is answered by a representative sample of students. This is consistent with other large-scale assessments, such as NAEP.

A total of 12 reading passages—two from PIRLS 2001, 2006 and 2011, two from 2006 and 2011, two from PIRLS 2011 only, and six new passages—were included in the 2016 assessment booklets used in all participating education systems. The use of common passages from the 2001 through the 2016 assessments allows for the analysis of change in reading literacy over the 15-year period between administrations for countries that participated in these cycles. The passages, as well as all other study materials, were translated into the primary language or languages of instruction in each education system.

PIRLS Background Questionnaires

Background questionnaires are administered to collect information about students' home and school experiences in learning to read. A student questionnaire addresses students' attitudes towards reading and their reading habits. The student questionnaire is administered after the assessment portion, taking about 30 minutes to complete. In all, PIRLS takes 1½ to 2 hours of each student's time, including the assessment and background questionnaire.

In addition, questionnaires were given to students' teachers and school principals to gather information about students' school experiences in developing reading literacy. The teacher and school questionnaires are administered either online from a secure website or via a hardcopy form. Teacher questionnaires take about 40 minutes to complete and ask teachers questions about their education and experience, available resources, and instructional practices. School questionnaires take about 40 minutes to complete and ask about school practices and resources.
In countries other than the United States, a questionnaire is also administered to parents. The United States has historically not administered the parent questionnaire.

ePIRLS Assessment

The ePIRLS assessment measured reading for informational purposes using a simulated Internet environment. The ePIRLS assessment consisted of five tasks and, like PIRLS, the assessment was delivered in two 40-minute assessment "tasks" with a 5-minute break in between, followed by a brief questionnaire. Approximately 65 percent of the reading items were multiple-choice and 35 percent were open response. Open response items were graded by trained scorers following international coding guidelines.

Translation

Source versions of all instruments (assessment booklets, questionnaires, and manuals) were prepared in English and translated into the primary language or languages of instruction in each education system. In addition, it was sometimes necessary to adapt the instrument for cultural purposes, even in countries that use English as the primary language of instruction. All adaptations were reviewed and approved by the TIMSS and PIRLS International Study Center at Boston College to ensure they did not change the substance or intent of the question or answer choices. For example, metric measurements were sometimes changed to standard measurements that would be more familiar to students (e.g., 16 meters to 52 feet).