Overview   NAEP Activities   National   State   Long-Term Trend   Selected Schools   HS Transcript Study   Special Studies   NAEP Technical   Documentation   Parents   Researchers   Media   Educators

The Arts   Civics   Economics   Foreign Language   Geography   Mathematics    Reading   Science   U.S. History   World History   Writing

2003 Trial Urban District Assessment

Introduction
What was assessed?
Who was assessed?
Accommodations
How to Interpret NAEP Results
Scale Score and Achievement-Level Results for the Trial Urban District Assessment
Snapshot Reports in Reading

Introduction

Federal appropriations authorized for the No Child Left Behind Act supported a multi-year study of the feasibility of a Trial Urban District Assessment of Educational Progress. In 2002, NAEP conducted the first Trial Urban District Assessment (TUDA) in reading and writing in five urban districts. In 2003, nine urban districts (including the original five) participated in the TUDA in reading and mathematics at grades 4 and 8. Only public-school students were sampled in the TUDA. For the purpose of comparison, results for the District of Columbia public schools, which normally participate in NAEP's state assessments, are also reported.

The trial design calls for a sufficient sample size to make reliable district-level comparisons. Because individual states have assessments based on a variety of scores, scales, and test designs, districts have not been able to validly compare themselves to a district in another state or to other districts with similar characteristics. The TUDA makes such comparisons possible.

Back to Top

What was assessed?

Students read three types of texts representing the following different contexts for reading:

• reading for literary experience,
• reading for information, and
• reading to perform a task (at grade 8)

The NAEP 2003 TUDA in reading used the same reading framework as the national and state assessments. Each student read one or two passages and responded to approximately 13-20 questions in 50 minutes. See sample questions from the 2003 reading assessment in the NAEP Questions Tool.

Back to Top

Who was assessed?

In 2003, the TUDA was conducted at grades 4 and 8 in nine urban districts: Atlanta City, Boston School District, Charlotte-Mecklenburg Schools, City of Chicago School District 299, Cleveland Municipal School District, Houston Independent School District, Los Angeles Unified School District, New York City Public Schools, and San Diego City Unified School District. As in 2002, results for the District of Columbia (DC) public schools that normally participate in NAEP's state assessments are also reported.

Representatives of the Council of Great City Schools worked with the staff of the National Assessment Governing Board (NAGB) to identify districts for the trial assessment. Districts were selected that permitted testing of the feasibility of conducting NAEP over a range of characteristics, such as district size, minority concentrations, federal program participation, socioeconomic conditions, and percentages of students with disabilities (SD) and limited-English-proficient (LEP) students.

Sampling for the TUDA was modeled on the procedure for sampling states. The number of participating schools ranged from 50 to 118 per district at the fourth grade and from 16 to 83 per district at the eighth grade. The number of participating students per district ranged from 1,445 to 2,806 at the fourth grade and from 1,038 to 1,963 at the eighth grade.

By undertaking the Trial Urban District Assessment, NAEP continues a tradition of extending its service to education, while preserving the rigorous sampling, scoring, and reporting procedures that have characterized prior NAEP assessments at both the national and state level.

Accommodations

It is NAEP's intent to assess all selected students from the target population. Beginning in 2002, students with disabilities and limited English proficient students who require accommodations have been permitted to use them in NAEP, unless a particular accommodation would alter the skills and knowledge being tested. For example, in a reading assessment, NAEP does not permit the reading passages to be read aloud.

To see a description of NAEP's inclusion policy and the types of accommodations provided, see Inclusion of Special-Needs Students.

Back to Top

How to Interpret NAEP Results

Any noted differences in performance between groups of students are based on statistical tests. The tests consider both the size of differences between averages or percentages and the standard error of those statistics. Every test score estimate has a standard error—a range of up to a few points above or below the score—due to sampling error and measurement error. Statistical tests are used to determine whether the differences between average scores are significant. Therefore, not all apparent differences may be found to be statistically significant. All the differences discussed were tested for statistical significance at the .05 level.

The reader is cautioned to rely on the reported differences in the text and tables, which are statistically significant, rather than on the apparent magnitude of any difference. The standard errors are available on the NAEP Data Tool, where there are procedures to test for statistically significant differences.

For more information on scale scores, achievement levels, and statistical significance of differences, go to Interpreting NAEP Reading Results.

Back to Top

Scale Score and Achievement-Level Results for the Trial Urban District Assessment

In order to provide a context for the data collected in the 2003 TUDA, results are presented for students attending public schools in the nation as a whole, as well as for public schools located in large central cities across the nation. "Urban districts" refers to the ten districts reported in this trial study. Eight of the ten urban districts consist entirely of schools in cities with a population of 250,000 or more (i.e., large central cities as defined by NCES); two of them (Charlotte and Los Angeles) consist primarily of schools in large central cities, but also have from one-quarter to one-third of their fourth- and eighth-grade students enrolled in surrounding urban fringe or rural areas. All of the data for both districts were used to compare with data from large central cities and the nation.

NOTE: For Charlotte and Los Angeles, statistical comparisons restricted to just the schools in large central cities, as distinct from the whole-district comparisons used here, are available from the Data Tool. The results of significance tests in this report for these two districts may differ slightly from those found by type of location in the Data Tool.

• Average NAEP reading scores, grade 4 public schools: By urban district, 2003
• Average NAEP reading scores, grade 8 public schools: By urban district, 2003
• Percentage of students at or above each achievement level in reading, grades 4 and 8 public schools: By urban district, 2003
• Selected reading scale score percentiles, grades 4 and 8 public schools: By urban district, 2003
• Cross-district comparisons of average reading scale scores, grade 4 public schools: By urban district, 2003
• Cross-district comparisons of average reading scale scores, grade 8 public schools: By urban district, 2003
• Cross-district comparisons of percentage of students at or above Proficient in reading, grade 4 public schools: By urban district, 2003
• Cross-district comparisons of percentage of students at or above Proficient in reading, grade 8 public schools: By urban district, 2003
• Students with disabilities and limited-English-proficient students identified, excluded, and assessed with accommodations as a percentage of all students, grades 4 and 8 in public schools: By urban district, 2003
• Average reading scale score and achievement-level results of students with disabilities and of limited-English-proficient students, grades 4 and 8 in public schools: By urban district, 2003

Download a copy of the Trial Urban District Assessment Highlights Report with more extensive results, including subgroup performance at the district level.

Select a district and download a copy of a one-page district Snapshot report in reading.

Download, view, and print the slides for the Associate Commissioners' presentation as a zipped PowerPoint file (1.2 MB).

Explore the district data in more detail on the NAEP Data Tool. There you will find data on demographic subgroup performance as well as contextual background data at the district level.

Back to Top

---

Last updated 7 May 2004 (CC)