Skip Navigation
Illustration/Logo View Quarterly by  This Issue  |  Volume and Issue  |  Topics
Education Statistics Quarterly
Vol 5, Issue 3, Topic: NAEP Trial Urban District Assessment
The Nation’s Report Card: Reading 2002, Trial Urban District Assessment
By: Anthony D. Lutkus, Arlene W. Weiner, Mary C. Daane, and Ying Jin
 
This article was originally published as the Executive Summary of the Statistical Analysis Report of the same name. The sample survey data are from the National Assessment of Educational Progress (NAEP) 2002 Trial Urban District Reading Assessment and 2002 Reading Assessment.
 
 

The National Assessment of Educational Progress (NAEP) is the nation’s ongoing representative sample survey of student achievement in core subject areas. NAEP, known as the Nation’s Report Card, is authorized by Congress and administered by the National Center for Education Statistics (NCES) of the Institute of Education Sciences in the U.S. Department of Education. NAEP regularly reports to the public on the educational progress of students in grades 4, 8, and 12.

In 2002, NAEP assessed the reading and writing performance of the nation’s fourth-, eighth-, and twelfth-grade students. NAEP also conducted assessments of fourth- and eighth-graders’ reading and writing in most of the states.

In 2001, after discussion among NCES, the National Assessment Governing Board (NAGB), and the leadership of the Council of the Great City Schools, Congress appropriated funds for a trial district-level assessment and NAGB passed a resolution approving the selection of five large urban districts for participation in the Trial Urban District Assessment, a special project within NAEP. Thus, this report presents, for the first time, district-level results of NAEP reading assessments in five urban public school districts: Atlanta City, Chicago School District 299, Houston Independent School District, Los Angeles Unified, and New York City Public Schools. Throughout this report, the districts are referred to simply as Atlanta, Chicago, Houston, Los Angeles, and New York City. The five districts participated voluntarily in the NAEP 2002 Trial Urban District Assessment in reading at grades 4 and 8. Results are also included in this report for the District of Columbia, which, in 2002 and past years, has been included in the main assessments with states and other jurisdictions. Data for public schools across the nation and for central city public schools are provided for comparison purposes.* The public schools also included charter schools, which in some cases were not managed by the urban school district.

NAEP does not provide scores for individual students or schools. It reports results for groups of students (e.g., fourth-graders). For each group in each table in the report, assessment results are described in one of two ways: the group’s average reading score on a scale from 0 to 500 or the percentage of students in the group who reached each of three achievement levels: Basic, Proficient, and Advanced. The Proficient level for each grade is defined by NAGB as representing “solid academic performance,” which demonstrates “competency over challenging subject matter” for the grade assessed. Basic indicates partial mastery of skills that are fundamental for proficient work. Advanced denotes superior performance.

The achievement levels are performance standards adopted by NAGB as part of its statutory responsibilities. The achievement levels are a collective judgment of what students should know and be able to do for each grade tested. As provided by law, NCES, upon review of a congressionally mandated evaluation of NAEP, determined that the achievement levels are to be used on a trial basis and should be interpreted with caution. However, both NCES and NAGB believe that the performance standards are useful for understanding trends in student achievement. They have been widely used by national and state officials and others as a common yardstick of academic performance.

The results are based on representative samples of students for the nation, for schools in central cities, and for participating districts. In order to obtain reliable and representative data, a large proportion of the selected schools and students must participate. All six districts met the NCES statistical participation criteria for NAEP samples at grade 4, but New York City data are not reported for grade 8 because eighth-grade participation did not meet the criteria.

Some students are identified by the school districts as students with disabilities or limited-English-proficient students. Some of these students are excluded from the assessment, and others are tested with accommodations related to their status. Because the percentages of students identified, excluded, and assessed with accommodations vary across the urban districts, that variability should be taken into consideration in interpreting the results and making comparisons.

For example, in the case of fourth-grade students, the percentages of students identified as having disabilities or limited English proficiency ranged from 8 to 51 percent, the percentages of fourth-grade students excluded for these reasons ranged from 2 to 17 percent, and the percentages assessed with accommodations ranged from 1 to 8 percent. At the eighth grade, the percentages of students identified with disabilities or limited English proficiency ranged from 6 to 35 percent, the percentages of eighth-grade students excluded for these reasons ranged from 2 to 7 percent, and the percentages assessed with accommodations ranged from 0 to 8 percent.

Back to top


Overall Reading Results for the Urban Districts

Average scale scores

Results for grade 4
  • The average scale scores for fourth-graders ranged from 191 in the District of Columbia and Los Angeles to 206 in Houston and New York City.
  • The average score for public school students in the nation as a whole was higher than the average score in each of the urban districts, and the national average score in central city public schools was higher than the average score in each of the urban districts except Houston and New York City (figure A).
  • The average scale scores in Houston and New York City were higher than those of the other urban districts and were not found to differ significantly from each other.
Results for grade 8

Results for New York City schools at grade 8 are not reported because they did not meet participation rates.

  • The average scale scores for eighth-graders ranged from 236 in Atlanta to 249 in Chicago.
  • The national average scores for public school students and for students in the central city public schools were higher than the average score in any of the urban districts (figure B).
  • The average scale scores in Chicago and Houston were higher than those of the other urban districts and were not found to differ significantly from each other.

Reading achievement levels

Results for grade 4
  • The percentages of fourth-graders performing at or above Proficient ranged from 10 percent in the District of Columbia to 19 percent in New York City. The percentage of students performing at or above Proficient in public schools in the nation was 30 per-cent, and for students in central city public schools it was 21 percent.
  • Any apparent differences between the percentages of students performing at or above Proficient in the urban districts were not found to be statistically significant.
  • The percentages of students performing at or above Basic ranged from 31 percent in the District of Columbia to 48 percent in Houston. In public schools across the nation, 62 percent of students performed at or above Basic. In central city schools in the national sample, 51 percent performed at or above Basic.
  • The percentages of students performing at or above Basic were higher in Houston and New York City than in the other urban districts.
Results for grade 8
  • The percentages of eighth-graders performing at or above Proficient ranged from 8 percent in Atlanta to 17 percent in Houston. Thirty-one percent of students in public schools in the nation and 23 percent in central city public schools performed at or above Proficient.
  • The percentages of students performing at or above Proficient in national public schools and central city public schools were higher than the percentages in each of the urban districts.
  • The percentages of students performing at or above Basic ranged from 42 percent in Atlanta to 62 percent in Chicago. Seventy-four percent of public school students in the nation and 64 percent in central cities performed at or above Basic.
  • The percentages of students performing at or above Basic in Chicago and Houston did not differ significantly from each other and both were higher than the comparable percentages in Atlanta, the District of Columbia, and Los Angeles.

Back to top


Results for Student Subgroups

In addition to providing average scores and achievement levels for the nation, for states, and, in this report, for urban districts, NAEP reports provide results for subgroups of students defined by various background and contextual characteristics (e.g., gender, eligibility for free or reduced-price lunch, and level of parents’ education).

Gender

Results for grade 4
  • Both male and female fourth-grade students in Atlanta, Chicago, the District of Columbia, and Los Angeles had average scores that were below the national average score for their counterparts in central city public schools.
  • Female students in the urban districts outscored male students, on average. The exception was Houston, where the apparent difference was not statistically significant.
Results for grade 8
  • The average score for male eighth-graders in central city public schools across the nation was higher than the average for male eighth-graders in each of the urban districts. The average score for female students in all urban districts except Chicago was below the national average for female students in central city public schools.
  • In all urban districts, female students had higher average scores than male students.
Race/ethnicity

Whereas White students constituted 60 percent of the national public sample at grade 4 and 64 percent at grade 8, in the urban districts, White students made up a maximum of 15 percent of the samples at grade 4 (New York City) and 11 percent of the samples at grade 8 (Chicago). Black or Hispanic students constituted majorities in the urban districts in the trial assessment. Hispanic students made up half or more of the sample in Houston and Los Angeles at both grades 4 and 8. Black fourth- and eighth-grade students made up more than 80 percent of the sample in both Atlanta and the District of Columbia.

Results for grade 4
  • In the five urban districts in which a reliable comparison could be made, White fourth-graders had higher average scores than their Black and Hispanic peers. In Chicago, Hispanic students had higher average scores than Black students.
  • The average scores for Black students in Chicago, the District of Columbia, and Los Angeles were lower than the national average for Black students in central city public schools.
  • The average scores for Hispanic students in Chicago, Houston, and Los Angeles were lower than the national average for Hispanic students in central city public schools.
  • The average score for Asian/Pacific Islander students in New York City was higher than the national average for Asian/Pacific Islander students in central city public schools.
Results for grade 8
  • White eighth-graders had higher average scores than Black eighth-graders in Atlanta, Houston, and Los Angeles. The apparent difference in Chicago was not found to be statistically significant, and the sample size in the District of Columbia was insufficient to permit a reliable comparison.
  • The average scores for Black students in Atlanta, the District of Columbia, and Los Angeles were lower than the national average for Black students in central city public schools.
  • The average score for Hispanic students in Los Angeles was lower than the national average for Hispanic students in central city public schools.
  • The average score for White students in Houston was higher than the national average for White students in central city public schools.
Eligibility for free/reduced-price lunch

The federal program providing free/reduced-price school lunch is administered by the U.S. Department of Agriculture (USDA) for children near or below the poverty line. Eligibility is determined by the USDA’s Income Eligibility Guidelines (http://www.fns.usda.gov/cnd/governance/notices/IEGs/IEGs.htm). At grade 4, the percentages of students in the urban districts eligible for free/reduced-price lunch ranged from 72 percent in Houston to 88 percent in Chicago. By comparison, 43 percent of fourth-graders in public schools nationally were eligible. At grade 8, the percentages of students eligible for free/reduced-price lunch in four of the urban districts ranged from 68 percent to 84 percent. By comparison, 34 percent of eighth-graders in public schools nationally were eligible. (Information on the free/reduced-price lunch data for eighth-graders in Los Angeles is not reported because these data did not meet reporting standards.)

Results for grade 4
  • In each of the urban districts, fourth-grade students not eligible for free/reduced-price lunch had higher average scores than students who were eligible.
  • The average scores for eligible students in Atlanta, Chicago, the District of Columbia, and Los Angeles were lower than the national average for eligible students in central city public schools.
  • The average scores for students in Atlanta, the District of Columbia, and Los Angeles who were not eligible for the program were lower than the national average for students in central city public schools who were not eligible.
Results for grade 8
  • Eighth-grade students not eligible for free/reduced-price lunch had higher average scores than eligible students in each of the urban districts except Chicago, where the apparent difference was not statistically significant.
  • The average scores for eligible students in Atlanta and the District of Columbia were lower than the national average for eligible students in central city public schools.
  • The average scores for students in Atlanta, the District of Columbia, and Houston who were not eligible for the program were lower than the national average for students in central city public schools who were not eligible.
Figure A. Cross-district comparisons of average reading scale scores, grade 4 public schools: By urban district, 2002
Figure A. Cross-district comparisons of average reading scale scores, grade 4 public schools: By urban district, 2002
‡Although deemed sufficient for reporting, the target response rate specified in the NAEP guidelines was not met.

NOTE: The between-district comparisons take into account sampling and measurement error and that each district is being compared with every other district shown. Significance is determined by an application of a multiple-comparison procedure.

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2002 Trial Urban District Reading Assessment and 2002 Reading Assessment. (Originally published as figure 2.1 on p. 14 of the complete report from which this article is excerpted.)

Figure B. Cross-district comparisons of average reading scale scores, grade 8 public schools: By urban district, 2002
Figure B. Cross-district comparisons of average reading scale scores, grade 8 public schools: By urban district, 2002
NOTE: The between-district comparisons take into account sampling and measurement error and that each district is being compared with every other district shown. Significance is determined by an application of a multiple-comparison procedure.

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2002 Trial Urban District Reading Assessment and 2002 Reading Assessment. (Originally published as figure 2.2 on p. 15 of the complete report from which this article is excerpted.)

Parents’ highest level of education

Eighth-grade students who participated in the Trial Urban District Assessment were asked to indicate the highest level of education they thought that their parents had completed. Five response options were offered: did not finish high school, graduated from high school, some education after high school, graduated from college, and “I don’t know.”

  • In comparison with the other urban districts, the District of Columbia had the highest percentage of eighth-graders (40 percent) who reported that at least one parent had graduated from college.
  • In each of the urban districts, the percentage of students who reported that at least one parent had graduated from college was lower than that of public schools nationally.
  • For students who reported that at least one parent had graduated from college, the average scores for students in Atlanta, Chicago, the District of Columbia, and Los Angeles were lower than the national average for students in central city public schools.

Back to top


Footnotes

*“Central city” includes a nationally representative sample of public schools located in central cities within metropolitan statistical areas as defined by the federal Office of Management and Budget. A central city is a city of 50,000 or more that is the largest in its metropolitan area, or can otherwise be regarded as “central.” The term means “a city that is central,” not “the central part of a city” or the “inner city.” Note that central cities encompass wider areas than what is commonly referred to as “the inner city.”


Data source: The National Assessment of Educational Progress (NAEP) 2002 Trial Urban District Reading Assessment and 2002 Reading Assessment.

For technical information, see the complete report:

Lutkus, A.D., Weiner, A.W., Daane, M.C., and Jin, Y. (2003). The Nation's Report Card: Reading 2002, Trial Urban District Assessment (NCES 2003–523).

Author affiliations: A.D. Lutkus, A.W. Weiner, M.C. Daane, and Y. Jin, Educational Testing Service.

For questions about content, contact Arnold Goldstein (arnold.goldstein@ed.gov ).

To obtain the complete report (NCES 2003–523), call the toll-free
ED Pubs number (877–433–7827) or visit the NCES Electronic Catalog ( http://nces.ed.gov/pubsearch ).


back to top