Frequently Asked Questions
In addition to the following questions about PISA, more FAQs about international
assessments are available at: http://nces.ed.gov/surveys/international/faqs.asp
- What are the components of PISA?
- What subject areas are assessed in PISA?
- How many U.S. schools and students participated in previous PISA cycles?
- How does PISA select a representative sample of students?
- Which countries participate in PISA?
- How does PISA differ from other NCES international assessments?
- How do the results of PISA compare with the results in TIMSS?
- When are PISA data collected in the United States?
- Where can I get a copy of the PISA U.S. Report?
- When is PISA next scheduled to be administered?
- What are the components of PISA?
Assessment
In 2009, PISA was a paper-and-pencil assessment that measured 15-year-old students' capabilities
in reading, mathematics, and science literacy. Each student took a two-hour assessment.
Assessment items include a combination of multiple-choice and open-ended questions
that require students to come up with their own response.
Questionnaires
In 2009, students completed a 30-minute student
questionnaire about themselves. In addition, the principal of each participating
school completed a 30-minute questionnaire
about the school.
Top
- What subject areas are assessed in PISA?
PISA measures student performance in reading literacy,
mathematics literacy, and science literacy. Conducted every 3 years, each PISA data
collection effort assesses one of these three subject areas in depth, although all
three are assessed in each cycle. The subject covered in depth is considered the
major subject area, and the other two subjects are considered minor subject areas
for that assessment year. Assessing all three areas allows participating jurisdictions
to have an ongoing source of achievement data in every subject area, while rotating
one area as the main focus over the years.
PISA administration cycle
|
Assessment year
|
|
Subjects assessed
|
READING
Mathematics
Science
|
Reading
MATHEMATICS
Science
Problem solving
|
Reading
Mathematics
SCIENCE
|
READING
Mathematics
Science
|
Reading
MATHEMATICS
Science
Problem solving
|
Reading
Mathematics
SCIENCE
|
|
NOTE: Reading, mathematics, and science literacy
are all assessed in each assessment cycle of the Program for International Assessment
(PISA). A separate problem-solving assessment was administered in 2003 and is planned
for 2012. The subject in all capital letters is the major subject area for that
cycle.
|
In 2000, reading literacy was the major domain, covering two-thirds of the testing
time. In addition to a combined reading literacy score, results were reported for
three reading subscales: retrieving information, interpreting texts, and reflecting
on texts. In 2003, mathematics literacy was the major domain and scores were reported
on a combined mathematics literacy scale and four subscales: space and shape, change
and relationships, quantity, and uncertainty. In 2006, science literacy was the
major domain and scores were reported on a combined science literacy scale and three
subscales: identifying scientific issues, explaining phenomena scientifically, and
using scientific evidence. In 2009, reading literacy was again the main focus. More
information on the
PISA assessment frameworks can be found at the OECD website.
Top
- How many U.S. schools and students participated in previous
PISA cycles?
|
Assessment year
|
|
2000
|
3,700
|
145
|
56
|
70
|
85
|
|
2003
|
5,456
|
262
|
65
|
68
|
83
|
|
2006
|
5,611
|
166
|
69
|
79
|
91
|
|
2009
|
5,233
|
165
|
68
|
78
|
87
|
Top
- How does PISA select a representative sample of students?
To provide valid estimates of student achievement and characteristics,
PISA selects a sample of students that represents the full population of
15-year-old students in each participating country and jurisdiction. This population
is defined internationally as 15-year-olds attending both public and private schools
in grades 7-12. PISA requires a minimum of 4,500 students from a minimum of 150
schools in each participating country and jurisdiction. Within schools, a sample
of 35 students must be selected in an equal probability sample unless fewer than
35 students age 15 are available (in which case all students are selected). PISA
requires that students in the sample be 15 years and 3 months to 16 years and 2
months at the beginning of the testing period. The school response rate target is
85 percent for all countries and jurisdictions. A minimum participation rate of
65 percent of schools from the original sample of schools is required for a country
or jurisdiction’s data to be included in the international database. PISA also requires
a minimum participation rate of 80 percent of sampled students from schools within
each country and jurisdiction.
U.S. sampling frame
The PISA U.S. sample is drawn from the Common Core of Data (CCD) listing of public
schools supplemented with the Private School Universe Survey (PSS) listing of private
schools. The combination of these national listings has proven to be close to 100
percent complete.
U.S. sampling design
In 2009, the design for sample selection was a two-stage process, with the first
stage a sample of schools and the second stage a sample of students within schools.
The design is a stratified systematic sample, with sampling probabilities proportional
to measures of school size. The frame is implicitly stratified (i.e., sorted for
sampling) by stratification variables. In 2009, the PISA sample was stratified into
eight explicit groups based on control of school (public or private) and region
of the country (Northeast, Central, West, Southeast). Within each stratum, the frame
was implicitly stratified by five categorical stratification variables: grade range
of the school (five categories), type of location relative to populous areas (four
categories), the first three digits of the Zip Code, percentage of minority students
(above or below 15 percent), and estimated enrollment of 15-year-olds.
Substitute schools
Participating countries and jurisdictions are allowed to use substitute schools
(selected during the sampling process) to increase the response rate once the 65
percent minimum participation rate is reached. In accordance with PISA guidelines,
substitute schools are identified by assigning the two schools neighboring the sampled
school in the frame as substitutes to be used in instances where an original sampled
school refuses to participate. Substitute schools are required to be in the same
implicit stratum (i.e., have similar demographic characteristics) as the sampled
school.
Top
- Which countries participate in PISA?
- PISA 2000: 43 countries and education systems participated (11 of these administered
PISA 2000 in 2002).
- PISA 2003: 41 countries and education systems participated.
- PISA 2006: 57 countries and education systems participated.
- PISA 2009: 65 countries and education systems participated.
The list of countries and education systems that participated in each PISA cycle
is available at: http://nces.ed.gov/surveys/pisa/countries.asp.
Top
- How does PISA differ from other international assessments?
PISA differs from these studies in several ways:
Content
PISA is designed to measure "literacy" broadly, while other studies, such as
TIMSS and NAEP, have a stronger link to
curriculum frameworks and seek to measure students' mastery of specific knowledge,
skills, and concepts. The content of PISA is drawn from broad content areas, such
as space and shape for mathematics, in contrast to more specific curriculum-based
content such as geometry or algebra.
Tasks
In addition to the differences in purpose and age coverage between PISA and other
international comparative studies, PISA differs from other assessments in what students
are asked to do. PISA focuses on assessing students' knowledge and skills in reading,
mathematics, and science literacy in the context of everyday situations. That is,
PISA emphasizes the application of knowledge to everyday situations by asking students
to perform tasks that involve interpretation of real-world materials as much as
possible. Analyses based on expert panels' reviews of mathematics and science items
from PISA, TIMSS, and NAEP indicate that PISA items require multi-step reasoning
more often than either TIMSS or NAEP. The study also shows that PISA mathematics
and science literacy items often involve the interpretation of charts and graphs
or other "real world" material. These tasks reflect the underlying assumption of
PISA: as 15-year-olds begin to make the transition to adult life, they need to not
only comprehend what they read or to retain particular mathematical formulas or
scientific concepts, they need to know how to apply their knowledge and skills in
the many different situations they will encounter in their lives.
A recent study comparing the PISA and NAEP (grades 8 and 12) reading assessments
found that PISA and NAEP view reading as a constructive process and both measure
similar cognitive skills. There are differences between them, though, reflecting
in part the different purposes of the assessments. First, NAEP has longer reading
passages than PISA and asks more questions about each passage, which is possible
because of the NAEP passages' longer length. With regard to cognitive skills, NAEP has more emphasis
on critiquing and evaluating text, while PISA has more emphasis on locating information.
NAEP also measures students’ understanding of vocabulary in context and PISA does
not include any questions of this nature. Finally, NAEP has a greater emphasis on
multiple-choice items compared to PISA and the nature of the open-ended items differs,
where PISA open-ended items call for less elaboration and support from the text
than do those in NAEP.
To learn more about the differences in the respective approaches to the assessment
of mathematics, science and reading among PISA, TIMSS, and NAEP, see the following
papers:
Age-based sample
The goal of PISA is to represent outcomes of learning rather than outcomes of schooling.
By placing the emphasis on age, PISA intends to show what 15-year-olds have learned
inside and outside of school throughout their lives, not just in a particular grade.
Focusing on age 15 provides an opportunity to measure broad learning outcomes while
all students across the many participating nations are still required to be in school.
Finally, because years of education vary among countries and jurisdictions, choosing
an age-based sample makes comparisons across countries and jurisdictions somewhat
easier.
Information collected
The kind of information PISA collects also reflects a policy purpose somewhat different
from the other assessments. PISA collects only background information related to
general school context and student demographics. This differs from other international
studies such as TIMSS, which also collects background information related to how
teachers in different countries approach the task of teaching and how the approved
curriculum is implemented in the classroom. The TIMSS video studies further extend
this work by capturing images of instruction across countries. The results of PISA
will certainly inform education policy and spur further investigation into differences
within and between countries and jurisdictions, but PISA is not intended to provide
direct information about improving instructional practice in the classroom. The
purpose of PISA is to generate useful indicators to benchmark performance and inform
policy.
Top
- How does the performance of U.S. students in mathematics
and science on PISA compare with U.S. student performance on TIMSS?
The performance of U.S. students in grade 8 on TIMSS 2007, the most recent administration
of TIMSS, showed U.S. average scores higher than the TIMSS scale average in both
mathematics and science. In PISA 2009, the average scores of U.S. 15-year-old students
were below the OECD average—the average score of students in the 34
Organization for Economic Cooperation and Development (OECD) countries—in
mathematics and not measurably different from the OECD average in science. Such
differences are difficult to compare because, while both TIMSS and PISA measure
the mathematics and science achievement of students, they do so for different sets
of students, in different ways, and in different sets of countries.
TIMSS focuses on the mathematics and science achievement of students in the fourth
and eighth grades. In contrast, PISA aims to assess the mathematics and science
literacy of students near the end of their compulsory schooling, at age 15. These
students range across several grades in most countries.
TIMSS draws its content directly from the school curriculum and is designed to assess
how well students have learned what they have ostensibly been taught. TIMSS emphasizes
the links between achievement, mathematics and science curricula, and classroom
practices. In contrast, PISA’s intent is to measure a "yield" of the skills and
competencies accumulated and applied in real-world contexts by students at age 15.
PISA emphasizes the mastery of processes, understanding of concepts, and application
of knowledge. It draws not only from school curricula but also from learning that
may occur outside of school. PISA does not explicitly examine mathematics and science
curricula and classroom practices, though it does collect school information, including
school background information and information on school practices and resources.
Both assessments cover much of the world and include key economic partners and competitors,
but there is only partial overlap between the sets of participating countries. For
instance, only 27 of the 48 countries that participated in TIMSS 2007 at grade 8
participated in PISA 2009. Comparing the PISA countries with the TIMSS countries
highlights the different sets of countries participating in each study. For example,
European countries make up about two-thirds of all PISA countries but only one-third
of TIMSS countries, and Middle-Eastern countries make up about 3 percent of all
PISA countries but 25 percent of TIMSS countries. About 25 percent of TIMSS countries
participate in PISA, and about one-half of PISA countries are in TIMSS as well.
Top
- When are PISA data collected in the United States?
PISA operates on a 3-year cycle, with 2000 being the
first assessment year. For PISA 2000, the U.S. data collection began in April and
ended in May. For PISA 2003, the U.S. data collection was conducted in the spring
(the same as in 2000) and again in the fall, beginning in September and ending in
November. For PISA 2006 and 2009, the U.S. data collection was conducted only in
the fall (September–November).
Top
- Where can I get a copy of the U.S. PISA reports?
U.S. PISA report for PISA 2000, 2003, 2006, and 2009 can be downloaded using the
links below.
Top
- When is PISA next scheduled to be administered?
PISA is scheduled to be administered next in 2012, with
results to be reported at the end of 2013.
Top
(424 KB)
Contact Us