Other Organization SourcesAppendix A.5. International Association for the Evaluation of Educational Achievement
The International Association for the Evaluation of Educational Achievement (IEA) is composed of governmental research centers and national research institutions around the world whose aim is to investigate education problems common among countries. Since its inception in 1958, the IEA has conducted more than 30 research studies of cross-national achievement. The regular cycle of studies encompasses learning in basic school subjects. Examples are the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS). IEA projects also include studies of particular interest to IEA members, such as the TIMSS 1999 Video Study of Mathematics and Science Teaching, the Civic Education Study, and studies on information technology in education.
The international bodies that coordinate international assessments vary in the labels they apply to participating education systems, most of which are countries. IEA differentiates between IEA members, which IEA refers to as "countries" in all cases, and "benchmarking participants." IEA members include countries such as the United States and Ireland, as well as subnational entities such as England and Scotland (which are both part of the United Kingdom), the Flemish community of Belgium, and Hong Kong-CHN (which is a Special Administrative Region of China). IEA benchmarking participants are all subnational entities and include Canadian provinces, U.S. states, and Dubai in the United Arab Emirates (among others). Benchmarking participants, like the participating countries, are given the opportunity to assess the comparative international standing of their students' achievement and to view their curriculum and instruction in an international context.
Some IEA studies, such as TIMSS and PIRLS, include an assessment portion as well as contextual questionnaires to collect information about students' home and school experiences. The TIMSS and PIRLS scales, including the scale averages and standard deviations, are designed to remain constant from assessment to assessment so that education systems (including countries and subnational education systems) can compare their scores over time, as well as compare their scores directly with the scores of other education systems. Although each scale was created to have a mean of 500 and a standard deviation of 100, the subject matter and the level of difficulty of items necessarily differ by grade, subject, and domain/dimension. Therefore, direct comparisons between scores across grades, subjects, and different domain/dimension types should not be made.
Further information on the International Association for the Evaluation of Educational Achievement may be obtained from http://www.iea.nl.
Trends in International Mathematics and Science Study
The Trends in International Mathematics and Science Study (TIMSS, formerly known as the Third International Mathematics and Science Study) provides reliable and timely data on the mathematics and science achievement of U.S. fourth- and eighth-graders compared with that of their peers in other countries. TIMSS is on a 4-year cycle, with data collection occurring in 1995, 1999 (eighth grade only), 2003, 2007, and 2011. In 2011, a total of 77 education systems, including 63 IEA members and 14 benchmarking participants, participated in TIMSS. The next TIMSS data collection is scheduled for 2015. TIMSS collects information through mathematics and science assessments and questionnaires. The questionnaires request information to help provide a context for student performance, focusing on such topics as students' attitudes and beliefs about learning mathematics and science, what students do as part of their mathematics and science lessons, students' completion of homework, and their lives both in and outside of school; teachers' perceptions of their preparedness for teaching mathematics and science topics, teaching assignments, class size and organization, instructional content and practices, collaboration with other teachers, and participation in professional development activities; and principals' viewpoints on policy and budget responsibilities, curriculum and instruction issues, and student behavior, as well as descriptions of the organization of schools and courses. The assessments and questionnaires are designed to specifications in a guiding framework. The TIMSS framework describes the mathematics and science content to be assessed and provides grade-specific objectives, an overview of the assessment design, and guidelines for item development.
Progress in International Reading Literacy Study
The Progress in International Reading Literacy Study (PIRLS) provides reliable and timely data on the reading literacy of U.S. fourth-graders compared with that of their peers in other countries. PIRLS is on a 5-year cycle, with data having been collected in 2001, 2006, and 2011. In 2011, a total of 57 education systems, including 48 IEA members and 9 benchmarking participants, participated in PIRLS. The next PIRLS data collection is scheduled for 2016. PIRLS collects information through a reading literacy assessment and questionnaires that help to provide a context for student performance. Questionnaires are administered to collect information about students' home and school experiences in learning to read. A student questionnaire addresses students' attitudes toward reading and their reading habits. In addition, questionnaires are given to students' teachers and school principals to gather information about students' school experiences in developing reading literacy. In countries other than the United States, a parent questionnaire is also administered. The assessments and questionnaires are designed to specifications in a guiding framework. The PIRLS framework describes the reading content to be assessed and provides objectives specific to fourth grade, an overview of the assessment design, and guidelines for item development.
TIMSS and PIRLS Sampling and Response Rates
As is done in all participating countries and other education systems, representative samples of students in the United States are selected. The sample design employed by TIMSS and PIRLS in 2011 is generally referred to as a two-stage stratified cluster sample. In the first stage of sampling, individual schools were selected with a probability proportionate to size (PPS) approach, which means that the probability is proportional to the estimated number of students enrolled in the target grade. In the second stage of sampling, intact classrooms were selected within sampled schools.
TIMSS and PIRLS guidelines call for a minimum of 150 schools to be sampled, with a minimum of 4,000 students assessed. The basic sample design of one classroom per school was designed to yield a total sample of approximately 4,500 students per population.
About 23,000 students in almost 900 schools across the United States participated in the 2011 TIMSS, joining 600,000 other student participants around the world. Because PIRLS was also administered at grade 4 in spring 2011, TIMSS and PIRLS in the United States were administered in the same schools to the extent feasible. Students took either TIMSS or PIRLS on the day of the assessments. About 13,000 U.S. students participated in PIRLS in 2011, joining 300,000 other student participants around the world. Accommodations were not provided for students with disabilities or students who were unable to read or speak the language of the test. These students were excluded from the sample. The IEA requirement is that the overall exclusion rate, which includes exclusions of schools and students, should not exceed more than 5 percent of the national desired target population.
In order to minimize the potential for response biases, the IEA developed participation or response rate standards that apply to all participating education systems and govern whether or not an education system's data are included in the TIMSS or PIRLS international datasets and the way in which its statistics are presented in the international reports. These standards were set using composites of response rates at the school, classroom, and student and teacher levels. Response rates were calculated with and without the inclusion of substitute schools that were selected to replace schools refusing to participate. In TIMSS 2011 at grade 4 in the United States, the weighted school participation rate was 79 percent before the use of substitute schools and 84 percent after the use of replacement schools; the weighted student response rate was 95 percent. In TIMSS 2011 at grade 8 in the United States, the weighted school participation rate was 87 percent before the use of substitute schools and 87 percent after the use of replacement schools; the weighted student response rate was 94 percent. In the 2011 PIRLS administered in the United States, the weighted school participation rate was 80 percent before the use of substitute schools and 85 percent after the use of replacement schools; the weighted student response rate was 96 percent.
Further information on the TIMSS study may be obtained from
Stephen Provasnik
Assessments Division
International Assessment Branch
National Center for Education Statistics
1990 K Street NW, Room 9034
Washington, DC 20006
(202) 502-7480
http://nces.ed.gov/timss
http://www.iea.nl/timss_2011.html
Further information on the PIRLS study may be obtained from
Sheila Thompson
Assessments Division
International Assessment Branch
National Center for Education Statistics
1990 K Street NW, Room 9031
Washington, DC 20006
(202) 502-7425
http://nces.ed.gov/surveys/pirls/
http://www.iea.nl/pirls_2011.html