|
Results From the 2000 Program for International
Student Assessment of 15-Year-Olds in Reading, Mathematics, and Science
Literacy
 |
Hello, and welcome to today's
StatChat on the PISA 2000 international assessments that focus on
15-year-olds' capabilities in reading literacy, mathematics
literacy, and science literacy. I'm sure that you have many
questions regarding this recent release, so let's get right to
them... |
| Jon from Chicago, Illinois asked: |
| When will the data be
available for secondary analysis? |
| Mariann Lemke: |
Jon, PISA data is available now on the
OECD website
http://www.pisa.oecd.org
so go to it! We hope to release a
datafile that also includes the U.S.-specific variables (such as
race/ethnicity) in the near future. |
| Babette Mannbro from Boras, Sweden (From Havre, Montana)
asked: |
| I was a teacher in
Montana and I am now teaching in Sweden (part-time). I see a big
difference here compared to Montana. What I really would like to
know is where are the students from that are selected to take this
test, what percentage of them are in private schools, what
percentage of them are are homeschooled. I guess what I am trying to
say is that I do believe that these tests are inadequate since I
have seen the Swedish and part of the American Education first hand.
Best Regards Babette Mannbro |
| Mariann Lemke: |
Babette, PISA only assesses students
who are in school, so no homeschoolers would be included. The
percentage of the students assessed who are from private/nonpublic
schools is very small and so isn't reported out separately --
instead results are presented for the U.S. 15-year-old population as a
whole. |
| John Panaretos from Athens, Greece
asked: |
| Congratulation for your
report! Here in Greece I have heard a complaint that the tests did
not take into account the differences of the Greek educational
system to the Anglo-Saxon one. What is the answer? Professor J.
Panaretos Director of the Graduate Program Department of Statistics
Athens University of Economics and Business |
| Mariann Lemke: |
John, I'm not sure what specific
concerns that people might have raised with you, but I can say that
all the PISA participating countries followed the same international
guidelines as far as sampling and administration goes. All
assessment instruments were translated and then those translations
verified, so that should have kept the assessments standardized
across countries. |
| Ellen from Topeka, Kansas asked: |
| Is there any special
reason why Reading was selected to be the focus of this
release? |
| Mariann Lemke: |
Ellen, of the 3 major subject areas,
the participating countries chose reading to be the first major
focus since it is such a key skill. |
| George from Portland, Maine asked: |
| Do we know anything
about how they teach in other countries that scored higher than U.S.
kids from which we can learn? |
| Mariann Lemke: |
George, the results from a large-scale
assessment like PISA can't give us enough specific information about
teaching/instructional strategies, but hopefully PISA's results will
spur researchers and practitioners to look into what other countries
(like those who scored higher than the U.S.) do that may make them
successful. |
| Arnie from Burke, VA asked: |
| PISA results indicate
US 15-yr-olds read about as well as those in the other OECD
countries. NAEP, however, indicates that in 1998 about 67% of 8th
graders and 60% of 12th graders were reading below the "Proficient"
level. Can we infer, then, that most 15-yr-olds in OECD countries
are not proficient readers? |
| Mariann Lemke: |
Arnie, unfortunately, it's difficult to
make these kind of direct comparisons because PISA's levels do not
correspond to the levels that NAEP has set (e.g. proficient, basic,
etc). On PISA, about 40% of U.S. students were at level 2 or below
(out of 5 levels). |
| Fred from Superior, Wisconsin asked: |
| How do we know that the
other countries didn't just select their best students for the
assessment? |
| Mariann Lemke: |
Fred, each country selected a
nationally representative sample (following international
guidelines) consisting of 15-year-olds from throughout their
educational systems. |
| Penelope from East St. Louis, Illinois asked: |
| The finding regarding
the relationship between socioeconomic status of the family and
achievement is interesting, but I'm not quite sure I understand it
thoroughly. Can you explain the relationship again, in simple
terms? |
| Mariann Lemke: |
Penelope, in short, PISA shows that the
relationship between SES is strong in the U.S. -- that is, students
from lower SES backgrounds will on average score lower than those
from higher SES backgrounds. However, the relationship between SES
and literacy scores in the U.S. is not different than the OECD
average, which means that while SES has a strong effect in the U.S.,
it has a similar effect in other countries
also. |
| Bob from Los Angeles, California asked: |
| How do the results from
PISA relate to results from other international
assessments? |
| Mariann Lemke: |
Bob, since PISA assesses a different
age level, using different kind of assessment instruments, we
think that PISA will complement other international assessments and
provide a rounded picture of how U.S. students perform compared to
their international counterparts. |
| Fred from Winston Salem, NC asked: |
| How does PISA differ
from TIMSS which also involves international
comparisons? |
| Mariann Lemke: |
Fred, PISA assesses reading,
mathematics, and science literacy -- an applied kind of knowledge of
these subject areas. TIMSS is more strongly linked to school
curriculum. TIMSS also assesses a different age level (8th grade)
than PISA. |
| Jonathan from Boulder, Colorado asked: |
| One of the results you
highlight is "The percentages of students who respond that they
often or always try to relate new material to things they have
already learned range from 15 percent in Italy to 90 percent in
Hungary." This gap seems rather odd. Were tests done identically in
all the countries? How can something this normal sounding have such
a wide discrapancy in two European countires? |
| Mariann Lemke: |
Jonathan, the cultural and social
context of a country may contribute to differences in how students
responded -- or there may be real differences in what kind of
learning strategies students use! |
| Dorothy from Springfield, Mass. asked: |
| In Sec. Paige's remarks
he stated "PISA confirms results from other national and
international studies that show there are gaps in performance
between racial and ethnic groups in the United States. White and
"other" 15-year-olds". Are there racial comparisons available for
other countries as well as the U.S.? |
| Mariann Lemke: |
Dorothy, since racial and ethnic
categories are very specific for each country, we can't compare
across countries. However, other countries often do racial and
ethnic comparisons within their own countries. PISA reports from
other countries are available from the OECD website at:
http://www.pisa.oecd.org. |
| Tracey from Lost City, West Virginia
asked: |
| How are the levels in
PISA determined? |
| Mariann Lemke: |
Tracey, PISA's levels were determined
by examining the difficulty of items. To reach a particular level, a
student had to be able to complete a majority of the items at that
level correctly. |
| Tom from Olney, Maryland asked: |
| I'd like to know if all
coutries contribute equally in the cost for this study?
|
| Mariann Lemke: |
Tom, PISA was organized by the OECD, an
intergovernmental organization. Each member country pays dues to the
OECD, and so for PISA, each country paid a portion of the
international costs for PISA based on the level of dues they pay to
the OECD as a whole. Each country also paid for the costs of
administering the assessment in that country (which of course simply
depends on prices/costs in that
country!). |
| John from Washington, DC asked: |
| Did PISA collect any
information on the overall climate of the school? Such as bullying,
crime, and the like? |
| Mariann Lemke: |
PISA did collect some information
students' and principals' perceptions of school climate. This
information is available in the OECD's report on PISA, Knowledge and
Skills for Life, which is available on their website at:
http://www.pisa.oecd.org. |
| Nancy from San Francisco, California
asked: |
| Do you know whether the
variation in student performance is related to differences between
schools (i.e., students are selected for different types of schools)
or within schools (i.e., students are selected for different levels
of classes within a school)? And, does it vary from nation to
nation? |
| Mariann Lemke: |
Nancy, in the US, most of the variation
we see is within schools (over 80% for reading). However, in other
countries, where students may be tracked into different kinds of
schools, more variation is seen between
schools. |
| Jerry from New York, New York asked: |
| I noticed that in some
nations, there are a greater percentage of students performing in
the top 75 percent of all international students. Does that mean
these nations are doing a better job of educating all their
students? |
| Mariann Lemke: |
Jerry, PISA measures how students
perform based on learning they may have gained inside and outside of
schools. There could be many reasons that some countries have a
greater proportion of students achieving a particular score (to get
into the top 75 percent, as in your
example). |
| Linda from Denver, Colorado asked: |
| I teach at the high
school level. Can I get copies of the PISA items to practice with my
students? |
| Mariann Lemke: |
Linda, Released items from PISA 2000
are available in the U.S. report, available on the web at:
http://nces.ed.gov/surveys/pisa
and on the OECD's PISA website at:
http://www.pisa.oecd.org. |
| Brenda from Naples, Florida asked: |
| I noticed that girls
and boys differed in their interest in different hobbies, like
reading. Do you have this information by race and
ethnicity? |
| Mariann Lemke: |
Brenda, this information isn't
available in the initial U.S. report on PISA, but the PISA data will
be made available so that if people want to do additional analyses
such as the one you suggest, they will be able
to. |
| Richard from Miami, Florida asked: |
| Did PISA collect
information on teaching practices? And, if so, what did you
learn? |
| Mariann Lemke: |
Richard, PISA did not collect
information teaching practices because it just didn't make sense
given PISA's design. Since PISA students are 15, they will have had
lots of teachers and it would be hard to correlate any one teacher's
practice with a student's performance. Also, in PISA, students are
sampled from lots of different classrooms so it would be hard to pin
down which particular teacher's actions were associated with
different performance at the national
level. |
| George from Santa Fe, New Mexico asked: |
| Aren't the comparisons
a bit unfair? I mean, we have some very large high schools in the
U.S., while other countries have smaller ones that specialize.
|
| Mariann Lemke: |
George, PISA assesses students from all
different kinds of schools in the U.S. and in other countries, so
that the results will represent what 15-year-olds from the whole
country can do. I think we have time for one more
question. |
| Jim from Dallas, Texas asked: |
| Did PISA find that any
particular study habits worked better, or led to higher achievement,
than others? In any of the subjects? |
| Mariann Lemke: |
Jim, the results were mixed in terms of
memorization and elaboration (relating new things to past learning)
strategies. But the OECD shows in its report that "controlling the
learning process" (that is, making sure you understand what you need
to know) is associated with higher student performance within
countries. |
|
What's New?
