2003 Trial Urban District Assessment
Introduction IntroductionFederal appropriations authorized for the No Child Left Behind Act supported a multi-year study of the feasibility of a Trial Urban District Assessment of Educational Progress. In 2002, NAEP conducted the first Trial Urban District Assessment (TUDA) in reading and writing in five urban districts. In 2003, nine urban districts (including the original five) participated in the TUDA in reading and mathematics at grades 4 and 8. Only public-school students were sampled in the TUDA. For the purpose of comparison, results for the District of Columbia public schools, which normally participate in NAEP's state assessments, are also reported. The trial design calls for a sufficient sample size to make reliable district-level comparisons. Because individual states have assessments based on a variety of scores, scales, and test designs, districts have not been able to validly compare themselves to a district in another state or to other districts with similar characteristics. The TUDA makes such comparisons possible. What was assessed?Students answered questions based on five mathematics content areas:
In addition, each question measured one of three mathematical abilities:
The NAEP 2003 TUDA in mathematics used the same mathematics framework as the national and state assessments. Each student answered a total of approximately 45 questions in 50 minutes. Who was assessed?In 2003, the TUDA was conducted at grades 4 and 8 in nine urban districts: Atlanta City, Boston School District, Charlotte-Mecklenburg Schools, City of Chicago School District 299, Cleveland Municipal School District, Houston Independent School District, Los Angeles Unified School District, New York City Public Schools, and San Diego City Unified School District. As in 2002, results for the District of Columbia (DC) public schools that normally participate in NAEP's state assessments are also reported. Representatives of the Council of Great City Schools worked with the staff of the National Assessment Governing Board (NAGB) to identify districts for the trial assessment. Districts were selected that permitted testing of the feasibility of conducting NAEP over a range of characteristics, such as district size, minority concentrations, federal program participation, socioeconomic conditions, and percentages of students with disabilities (SD) and limited-English-proficient (LEP) students. Sampling for the TUDA was modeled on the procedure for sampling states. The number of participating schools ranged from 50 to 118 per district at the fourth grade and from 16 to 83 per district at the eighth grade. The number of participating students per district ranged from 1,515 to 2,978 at the fourth grade and from 1,125 to 1,956 at the eighth grade. By undertaking the Trial Urban District Assessment, NAEP continues a tradition of extending its service to education, while preserving the rigorous sampling, scoring, and reporting procedures that have characterized prior NAEP assessments at both the national and state level. AccommodationsIt is NAEP's intent to assess all selected students from the target population. Beginning in 2002, students with disabilities and limited-English-proficient students who require accommodations have been permitted to use them in NAEP, unless a particular accommodation would alter the skills and knowledge being tested. For example, in a mathematics assessment, students may not use calculators for questions not intended for calculator use. To see a description of NAEP's inclusion policy and the types of accommodations provided, see Inclusion of Special-Needs Students. How to Interpret NAEP ResultsAny noted differences in performance between groups of students are based on statistical tests. The tests consider both the size of differences between averages or percentages and the standard error of those statistics. Every test score estimate has a standard error—a range of up to a few points above or below the score—due to sampling error and measurement error. Statistical tests are used to determine whether the differences between average scores or percentages are significant after considering statistical errors. Therefore, not all apparent differences may be found to be statistically significant. All the differences discussed were tested for statistical significance at the .05 level. The reader is cautioned to rely on the reported differences in the text and tables, which are statistically significant, rather than on the apparent magnitude of any difference. The standard errors are available on the NAEP Data Tool, where there are procedures to test for statistically significant differences. For more information on scale scores, achievement levels, and statistical significance of differences, go to Interpreting NAEP Mathematics Results. Scale Score and Achievement-level Results for the Trial Urban District AssessmentIn order to provide a context for the data collected in the 2003 TUDA, results are presented for students attending public schools in the nation as a whole, as well as for public schools located in large central cities across the nation. "Urban districts" refers to the ten districts reported in this trial study. Eight of the ten urban districts consist entirely of schools in cities with a population of 250,000 or more (i.e., large central cities as defined by NCES); two of them (Charlotte and Los Angeles) consist primarily of schools in large central cities, but also have from one-quarter to one-third of their fourth- and eighth-grade students enrolled in surrounding urban fringe or rural areas. All of the data for both districts were used to compare with data from large central cities and the nation. NOTE: For Charlotte and Los Angeles, statistical comparisons restricted to just the schools in large central cities, as distinct from the whole-district comparisons used here, are available from the Data Tool. The results of significance tests in this report for these two districts may differ slightly from those found by type of location in the Data Tool.
Download a copy of the Trial Urban District Assessment Highlights Report with more extensive results, including subgroup performance at the district level. Select a district and download a copy of a one-page district Snapshot report in mathematics. Download, view, and print the slides for the Associate Commissioners' presentation as a zipped PowerPoint file (1.2 MB). Explore the district data in more detail on the NAEP Data Tool. There you will find data on demographic subgroup performance as well as contextual background data at the district level.
|
