In addition to a range of scale scores as the basic form of measurement, PISA describes student proficiency in terms of levels. Higher levels represent the knowledge, skills, and capabilities needed to perform tasks of increasing complexity. PISA results are reported in terms of percentages of the student population at each of the predefined levels. Descriptions of each proficiency level can be found here for mathematics literacy, here for science literacy, and here for reading literacy.
To determine the performance levels and cut scores on the literacy scales, IRT techniques were used. With IRT techniques, it is possible to simultaneously estimate the ability of all students taking the PISA assessment, as well as the difficulty of all PISA items. Estimates of student ability and item difficulty can then be mapped on a single continuum. The relative ability of students taking a particular test can be estimated by considering the percentage of test items they get correct. The relative difficulty of items in a test can be estimated by considering the percentage of students getting each item correct. In PISA, all students within a level are expected to answer at least half of the items from that level correctly. Students at the bottom of a level are able to provide the correct answers to about 52 percent of all items from that level, have a 62 percent chance of success on the easiest items from that level, and have a 42 percent chance of success on the most difficult items from that level. Students in the middle of a level have a 62 percent chance of correctly answering items of average difficulty for that level (an overall response probability of 62 percent). Students at the top of a level are able to provide the correct answers to about 70 percent of all items from that level, have a 78 percent chance of success on the easiest items from that level, and have a 62 percent chance of success on the most difficult items from that level. Students just below the top of a level would score less than 50 percent on an assessment at the next higher level. Students at a particular level demonstrate not only the knowledge and skills associated with that level but also the proficiencies defined by lower levels. Patterns of responses for students below level 1b for reading literacy and below level 1 for mathematics and science literacy suggest that these students are unable to answer at least half of the items from those levels correctly. For details about the approach to defining and describing the PISA levels and establishing the cut scores, see the OECD’s PISA 2012 Technical Report (forthcoming). Table AA2 shows the cut scores for each proficiency level for mathematics, science, and reading literacy.
|Table AA2. Cut scores for proficiency levels for mathematics, science, and reading literacy: 2012|
|Below level 1||0-358||0-335||0-262|
|Level 1||greater than 358-420||greater than 335-410||greater than 262-335 (1b)|
|greater than 335-407 (1a)|
|Level 2||greater than 420-482||greater than 410-484||greater than 407-480|
|Level 3||greater than 482-545||greater than 484-559||greater than 480-553|
|Level 4||greater than 545-607||greater than 559-633||greater than 553-626|
|Level 5||greater than 607-669||greater than 733-708||greater than 626-698|
|Level 6||greater than 669-1000||greater than 708-1000||greater than 698-1000|
|1The first reading literacy proficiency level is composed of levels 1a and 1b. The score range for below level 1 refers to scores below 1b.
SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2012.