Education in States and Nations: Supplemental Notes

Education in States and Nations: 1991

(ESN) Indicators 16, 17, 18, 19, and 20: Note on class size, student use of technology, student time spent doing homework or watching television, instructional strategies in mathematics courses, and time in formal instruction

Notes on Figures and Tables

Brazil

In-school population only was sampled in two cities, Sao Paulo (in the South) and Fortaleza (in the Northeast).

Canada

Nine of ten provinces participated.

Emilia-Romagna, Italy; Portugal; Scotland

Combined school and student participation rate was below 80 percent but at least 70 percent; interpret results with caution because of possible nonresponse bias.

England

Combined school and student participation rate was below 70 percent; interpret results with extreme caution.

Israel

Hebrew-speaking schools.

Soviet Union

Fourteen of fifteen republics; Russian-speaking schools only.

Spain

All regions except Catalua. Spanish-speaking schools only.

Switzerland

Results represent percent of classrooms in schools; fifteen of twenty-six cantons included.

United States

The U.S. sample for the International Assessment of Educational Progress (IAEP) consisted of both public and private schools. Only 13-year-olds were included.

The state samples for the National Assessment of Educational Progress (NAEP), on the other hand, consisted of 8th grade classrooms only in public schools.

On average, students in the state samples were likely older than those in the U.S. sample.

Technical Notes

Calculation of class size

School administrators in schools participating in the International Assessment of Educational progress were asked the modal class size for the grade level to which most 13-year-old students were assigned. This class size refers to all academic subject areas, not just mathematics.

Data for the U.S. states do not include special education teachers.

Calculation of instructional hours per day for the U.S. States (Indicator 20)

The base estimates of school hours per day for the U.S. states come from the 1990-91 Schools and Staffing Survey. These estimates, however, include lunch hours and breaks. The estimate of instructional hours per day for the United States as a whole comes from the International Assessment of Educational Progress and includes only instructional hours. Formal instruction is that which takes place, generally in a classroom, between a teacher and a set group of students on a regularly scheduled basis.

In order to convert state-level school hours to instructional hours, the state-level estimates were totalled and averaged. That all-states average of school hours per day (6.5) had subtracted from it the U.S. estimate of instructional hours per day (5.6). The difference (.9 hours) was taken to be the amount of time on average, encompassed by lunch time and breaks. The difference was subtracted from each state's estimated school hours per day to arrive at an instructional hours per day figure for each state.

These figures were then checked against the state requirements for minimum length of the instructional school day, and all appeared equal to or greater than the minimums.

The International Assessment of Educational Progress (IAEP)

In conjunction with the International Assessment of Educational Progress, three questionnaires were administered to the national education minister, to administrators of participating schools, and to students sitting for the mathematics exam. Indicators here draw from results of the latter two.

The National Assessment of Educational Progress (NAEP)

In conjunction with the National Assessment of Educational Progress, three questionnaires were administered to administrators of participating schools, to teachers in participating classrooms, and to students sitting for the mathematics exam. Indicators here draw from results of all three of the questionnaires.

Comparing the IAEP and the NAEP

The IAEP and the NAEP employed mathematics tests that were so similar that comparisons across countries and states could be validly made. Each test administration was also accompanied by questionnaires that asked students, teachers, school administrators, and education ministers various questions about classroom practices, study habits, education system characteristics, and family backgrounds. While the tests given the students were very much alike, the IAEP and NAEP questionnaires were not so similar. Many questions were posed in the IAEP questionnaires that were not posed in the NAEP questionnaires, and vice versa.

Still, some questions are so naturally a part of such a study that they found their way into both sets of questionnaires. There were some questions common to both sets that were posed in an almost identical manner. Other questions, however, while much the same in meaning, contained differences in wording significant enough to, perhaps, influence some different responses. Other differences between the IAEP and NAEP could also influence some different responses. They include differing sample populations and times of administration.

The most straightforward way to determine if similar questions convey the same meaning to respondents and elicit the same responses is to compare the United States' average response in the IAEP to the United States' average response in the NAEP. They should be roughly similar numbers. If they are not, differences in question wording, time of administration, or sample populations may have influenced the results.

Only questions that appeared very similar in meaning between their IAEP and NAEP versions were chosen for inclusion in this report. Nonetheless, on two of these measures in particular, the United States' average from the IAEP appears very different than its average from the NAEP. These measures are the frequency of calculator use (in Indicator 17) and the frequency of group problem-solving in mathematics courses (in Indicator 19).

The following four tables make explicit this comparison between the questions that are common to the IAEP and NAEP and that are used in this report:

IAEP/NAEP differences in responses for processes indicators, by test administration and indicator (Table S17);
IAEP/NAEP sample population differences, by test administration and test or questionnaire characteristic (Table S18);
IAEP/NAEP question differences, by test and indicator (Table S19); and
Possible explanations of differences in U.S. responses between the IAEP and NAEP, by test and indicator (Table S20).

Table S17 IAEP/NAEP differences in responses for processes indicators, by test administration and indicator

---------------------------------------------------------------------------------------
                               1991            1992             1992            1990
                             IAEP U.S.       NAEP U.S         NAEP states'     NAEP U.S
Indicator                     average         average           range          average
----------------------------------------------------------------------------------------
17:
 # Calculator use (%)           54/1           71/1              47-88            61
   Computer use (%)             37/1           40/1              26-61            34
18:
   Homework hours (2+) daily    29/1           25/1              19-34            24
   TV hours (2+) daily          16/1           15/1              10-28            12
19:
   Ability groups (%)           56/2,*         61/3              25-84            63
 # Group problems (%) weekly    49/1           36/1              27-47            28
   Math tests (%) weekly        68/1           62/1              46-92            66
-----------------------------------------------------------------------------------------

# Large difference between IAEP and NAEP averages for the United States.

* Large standard error.
1 Student questionnaire.
2 School questionnaire.
3 Teacher questionnaire.

SOURCE: Educational Testing Service, International Assessment of Educational Progress, Learning Mathematics. U.S. Department of Education, , Data Compendium for the NAEP 1992 Mathematics Assessment of the Nation and the States.

Table S18 IAEP/NAEP U.S. sample population differences, by test administration and test or questionnaire characteristic

-----------------------------------------------------------------------------------
Test or questionnaire
 characteristic                  1991 IAEP                      1992 NAEP
------------------------------------------------------------------------------------
Time of administration           March 1991                     Jan.-Mar. 1992
Age of students                  Modal grade for 13-year-olds   8th grade only
                                 in March 1991
                                 o some 8th grade (60%)
                                 o some 7th grade (35%)
Sample frame                     o 96 schools                   o 406 schools
                                 o 1,407 students               o >2500 students in each state
                                 o 71% overall response rate    o 6% LEP and disabled students
                                                                   did not participate
                                 o 77% school response rate     o 75% overall response rate
                                 o 92% student response rate    o 84% school response rate
                                                                o 89% student response rate
Public/private                  Only public                     Public and private
Questionnaire forms             o students                      o students
                                o school administrators         o teachers
                                o education ministers           o school administrators
-----------------------------------------------------------------------------------------

Table S19 IAEP/NAEP question differences, by test and indicator

-----------------------------------------------------------------------------------------------------------------------
Indicator                   IAEP                                                NAEP
------------------------------------------------------------------------------------------------------------------------
17:
   Calculator use          "Ever use calculators in school"                     (inverse of) "Never or hardly ever use
                                                                                calculators in math class"
   Computer use
                           "Ever use computers for school work or homework"     (inverse of)  "Never or hardly ever use
                                                                                 computers for school work"
18:
   Homework hours          "Spend 2 hours or more on all homework everyday"     "2 or more hours each day on homework
                                                                                (in all subjects)"
   TV hours                "Watch TV 1 hour a day or less"                      (inverse)  "Watch TV 1 hour a day or
                                                                                 less" (inverse)
19:
   Ability groups         "Schools where math classes are based on ability"     Teachers report:  "Yes, students are
                                                                                grouped by ability" (% of students)
   Group work             "Solve problems in groups at least once a week"       "Work in small groups in Math Class at
                                                                                least weekly"
   Take math tests        "Take a math test or quiz at least once a week"      "Take Math tests at least once a week"
------------------------------------------------------------------------------------------------------------------------

Table S20 Possible explanations of differences in U.S. responses between the IAEP and NAEP, by test and indicator

----------------------------------------------------------------------------------------------------------------------
Indicator Possible explanations
------------------------------------------------------------------------------------------------------
17:
Calculator use o Slight differences in question wording may elicit different responses.
o It may be that 7th graders are less likely to use calculators than 8th
graders, so much so that the fact that the IAEP
sample includes some 7th graders affects the responses.
Computer use o The rise in computer use over time may explain the small difference.
Computer use increased from 34 to 40 percent between the 1990 and 1992
administrations of the NAEP. The U.S. average for computer use in the 1991
IAEP was 37 percent, in between the earlier and later NAEP averages.
18:
Homework hours o The difference is small.
TV hours o The difference is small.
19:
Ability groups o The difference is small.
Group work o Differences in question wording may account for difference in responses.
The IAEP question may imply that when the entire class is working on a
problem, that's group work. The NAEP asks specifically for small group
work.
Take math tests o The difference is small. It may be accounted for by difference in question
wording. The IAEP includes "quizzes," while the NAEP question does not.
------------------------------------------------------------------------------------------------------

Number and size of schools

Secondary school completion