Skip Navigation
small NCES header image

Results From the 2000 Program for International Student Assessment of 15-Year-Olds in Reading, Mathematics, and Science Literacy

Mariann Lemke Hello, and welcome to today's StatChat on the PISA 2000 international assessments that focus on 15-year-olds' capabilities in reading literacy, mathematics literacy, and science literacy. I'm sure that you have many questions regarding this recent release, so let's get right to them...

Jon from Chicago, Illinois asked:
When will the data be available for secondary analysis?
Mariann Lemke: Jon, PISA data is available now on the OECD website
http://www.pisa.oecd.org
so go to it! We hope to release a datafile that also includes the U.S.-specific variables (such as race/ethnicity) in the near future.

Babette Mannbro from Boras, Sweden (From Havre, Montana) asked:
I was a teacher in Montana and I am now teaching in Sweden (part-time). I see a big difference here compared to Montana. What I really would like to know is where are the students from that are selected to take this test, what percentage of them are in private schools, what percentage of them are are homeschooled. I guess what I am trying to say is that I do believe that these tests are inadequate since I have seen the Swedish and part of the American Education first hand. Best Regards Babette Mannbro
Mariann Lemke: Babette, PISA only assesses students who are in school, so no homeschoolers would be included. The percentage of the students assessed who are from private/nonpublic schools is very small and so isn't reported out separately -- instead results are presented for the U.S. 15-year-old population as a whole.

John Panaretos from Athens, Greece asked:
Congratulation for your report! Here in Greece I have heard a complaint that the tests did not take into account the differences of the Greek educational system to the Anglo-Saxon one. What is the answer? Professor J. Panaretos Director of the Graduate Program Department of Statistics Athens University of Economics and Business
Mariann Lemke: John, I'm not sure what specific concerns that people might have raised with you, but I can say that all the PISA participating countries followed the same international guidelines as far as sampling and administration goes. All assessment instruments were translated and then those translations verified, so that should have kept the assessments standardized across countries.

Ellen from Topeka, Kansas asked:
Is there any special reason why Reading was selected to be the focus of this release?
Mariann Lemke: Ellen, of the 3 major subject areas, the participating countries chose reading to be the first major focus since it is such a key skill.

George from Portland, Maine asked:
Do we know anything about how they teach in other countries that scored higher than U.S. kids from which we can learn?
Mariann Lemke: George, the results from a large-scale assessment like PISA can't give us enough specific information about teaching/instructional strategies, but hopefully PISA's results will spur researchers and practitioners to look into what other countries (like those who scored higher than the U.S.) do that may make them successful.

Arnie from Burke, VA asked:
PISA results indicate US 15-yr-olds read about as well as those in the other OECD countries. NAEP, however, indicates that in 1998 about 67% of 8th graders and 60% of 12th graders were reading below the "Proficient" level. Can we infer, then, that most 15-yr-olds in OECD countries are not proficient readers?
Mariann Lemke: Arnie, unfortunately, it's difficult to make these kind of direct comparisons because PISA's levels do not correspond to the levels that NAEP has set (e.g. proficient, basic, etc). On PISA, about 40% of U.S. students were at level 2 or below (out of 5 levels).

Fred from Superior, Wisconsin asked:
How do we know that the other countries didn't just select their best students for the assessment?
Mariann Lemke: Fred, each country selected a nationally representative sample (following international guidelines) consisting of 15-year-olds from throughout their educational systems.

Penelope from East St. Louis, Illinois asked:
The finding regarding the relationship between socioeconomic status of the family and achievement is interesting, but I'm not quite sure I understand it thoroughly. Can you explain the relationship again, in simple terms?
Mariann Lemke: Penelope, in short, PISA shows that the relationship between SES is strong in the U.S. -- that is, students from lower SES backgrounds will on average score lower than those from higher SES backgrounds. However, the relationship between SES and literacy scores in the U.S. is not different than the OECD average, which means that while SES has a strong effect in the U.S., it has a similar effect in other countries also.

Bob from Los Angeles, California asked:
How do the results from PISA relate to results from other international assessments?
Mariann Lemke: Bob, since PISA assesses a different age level, using different kind of assessment instruments, we think that PISA will complement other international assessments and provide a rounded picture of how U.S. students perform compared to their international counterparts.

Fred from Winston Salem, NC asked:
How does PISA differ from TIMSS which also involves international comparisons?
Mariann Lemke: Fred, PISA assesses reading, mathematics, and science literacy -- an applied kind of knowledge of these subject areas. TIMSS is more strongly linked to school curriculum. TIMSS also assesses a different age level (8th grade) than PISA.

Jonathan from Boulder, Colorado asked:
One of the results you highlight is "The percentages of students who respond that they often or always try to relate new material to things they have already learned range from 15 percent in Italy to 90 percent in Hungary." This gap seems rather odd. Were tests done identically in all the countries? How can something this normal sounding have such a wide discrapancy in two European countires?
Mariann Lemke: Jonathan, the cultural and social context of a country may contribute to differences in how students responded -- or there may be real differences in what kind of learning strategies students use!

Dorothy from Springfield, Mass. asked:
In Sec. Paige's remarks he stated "PISA confirms results from other national and international studies that show there are gaps in performance between racial and ethnic groups in the United States. White and "other" 15-year-olds". Are there racial comparisons available for other countries as well as the U.S.?
Mariann Lemke: Dorothy, since racial and ethnic categories are very specific for each country, we can't compare across countries. However, other countries often do racial and ethnic comparisons within their own countries. PISA reports from other countries are available from the OECD website at:
http://www.pisa.oecd.org.

Tracey from Lost City, West Virginia asked:
How are the levels in PISA determined?
Mariann Lemke: Tracey, PISA's levels were determined by examining the difficulty of items. To reach a particular level, a student had to be able to complete a majority of the items at that level correctly.

Tom from Olney, Maryland asked:
I'd like to know if all coutries contribute equally in the cost for this study?
Mariann Lemke: Tom, PISA was organized by the OECD, an intergovernmental organization. Each member country pays dues to the OECD, and so for PISA, each country paid a portion of the international costs for PISA based on the level of dues they pay to the OECD as a whole. Each country also paid for the costs of administering the assessment in that country (which of course simply depends on prices/costs in that country!).

John from Washington, DC asked:
Did PISA collect any information on the overall climate of the school? Such as bullying, crime, and the like?
Mariann Lemke: PISA did collect some information students' and principals' perceptions of school climate. This information is available in the OECD's report on PISA, Knowledge and Skills for Life, which is available on their website at:
http://www.pisa.oecd.org.

Nancy from San Francisco, California asked:
Do you know whether the variation in student performance is related to differences between schools (i.e., students are selected for different types of schools) or within schools (i.e., students are selected for different levels of classes within a school)? And, does it vary from nation to nation?
Mariann Lemke: Nancy, in the US, most of the variation we see is within schools (over 80% for reading). However, in other countries, where students may be tracked into different kinds of schools, more variation is seen between schools.

Jerry from New York, New York asked:
I noticed that in some nations, there are a greater percentage of students performing in the top 75 percent of all international students. Does that mean these nations are doing a better job of educating all their students?
Mariann Lemke: Jerry, PISA measures how students perform based on learning they may have gained inside and outside of schools. There could be many reasons that some countries have a greater proportion of students achieving a particular score (to get into the top 75 percent, as in your example).

Linda from Denver, Colorado asked:
I teach at the high school level. Can I get copies of the PISA items to practice with my students?
Mariann Lemke: Linda, Released items from PISA 2000 are available in the U.S. report, available on the web at:
http://nces.ed.gov/surveys/pisa
and on the OECD's PISA website at:
http://www.pisa.oecd.org.

Brenda from Naples, Florida asked:
I noticed that girls and boys differed in their interest in different hobbies, like reading. Do you have this information by race and ethnicity?
Mariann Lemke: Brenda, this information isn't available in the initial U.S. report on PISA, but the PISA data will be made available so that if people want to do additional analyses such as the one you suggest, they will be able to.

Richard from Miami, Florida asked:
Did PISA collect information on teaching practices? And, if so, what did you learn?
Mariann Lemke: Richard, PISA did not collect information teaching practices because it just didn't make sense given PISA's design. Since PISA students are 15, they will have had lots of teachers and it would be hard to correlate any one teacher's practice with a student's performance. Also, in PISA, students are sampled from lots of different classrooms so it would be hard to pin down which particular teacher's actions were associated with different performance at the national level.

George from Santa Fe, New Mexico asked:
Aren't the comparisons a bit unfair? I mean, we have some very large high schools in the U.S., while other countries have smaller ones that specialize.
Mariann Lemke: George, PISA assesses students from all different kinds of schools in the U.S. and in other countries, so that the results will represent what 15-year-olds from the whole country can do. I think we have time for one more question.

Jim from Dallas, Texas asked:
Did PISA find that any particular study habits worked better, or led to higher achievement, than others? In any of the subjects?
Mariann Lemke: Jim, the results were mixed in terms of memorization and elaboration (relating new things to past learning) strategies. But the OECD shows in its report that "controlling the learning process" (that is, making sure you understand what you need to know) is associated with higher student performance within countries.

Thanks for all of the great questions. Unfortunately, I could not get to all of them, but please feel free to contact me if you need further assistance. Today's release of Outcomes of Learning: Results from the 2000 Program for International Student Assessment of 15-Year-Olds in Reading, Mathematics, and Science Literacy is the first report from this new international study. PISA will collect data every 3 years, with a different focus for each cycle. In 2003, the focus will be on mathematics literacy, and in 2006 on science literacy. We look forward to continuing this dialogue with all of you.

Back to StatChat Home



Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.
National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education