 |
 |
 |
 |
 |
<< Go Back |
| Note 6: NAEP, NELS, and HS&B Transcript Studies |
There are various ways to measure the academic coursework that students complete. For example, one can measure the number of courses a student has completed in different subjects (i.e., whether a student completed two, three, or four courses in mathematics). Another method is to measure the highest level of coursework completed in different subjects (i.e., whether a student’s most academically challenging mathematics course was algebra I, trigonometry, or calculus). If one is interested in how common it is for students to complete certain courses, one can measure the frequency with which certain courses are taken as a proportion of all courses taken. Based on these three methods, analysts have created different taxonomies to categorize high school and postsecondary student coursetaking. This supplemental note describes three such taxonomies used in the analyses of individual indicators in The Condition of Education. Indicators 21 and 22 use an “academic pipeline” to classify coursetaking data according to the highest level of coursework completed by high school graduates. These data come from transcripts of graduates of public and private high schools, which were collected as part of the U.S. Department of Education’s National Assessment of Educational Progress (NAEP), National Education Longitudinal Study of 1988 (NELS:88), and the High School and Beyond Longitudinal Study (HS&B). (It is important to note that comparability among these data sets cannot be perfect both because (1) the Secondary School Taxonomy (SST), discussed below, was revised in 1998 and (2) these data come from different transcript collections, thus introducing the possibility of minor variations in the coding methodology even though steps were taken to replicate the data collection and coding methodology in each study.) Indicator 30 uses a “credit ratio” to classify coursetaking data according to the frequency with which postsecondary courses were completed. These data come from transcripts of three cohorts of different NCES longitudinal studies:
The analyses reported in indicator 30 are based on a subsample of students from each cohort who were in 12th grade on schedule in 1992 and who earned a bachelor’s degree within 8.5 years of their graduation from high school. Indicator 18 uses five remediation categories to measure the number of remediation courses taken by students who were also in the 12th grade on schedule and entered college by 2000. These estimates come from the postsecondary transcripts in the NELS:88/2000-PETS study. ACADEMIC PIPELINES FOR HIGH SCHOOL COURSETAKING Academic “pipelines” organize transcript data in English, science, mathematics, and foreign language into levels based on the normal progression and difficulty of courses within these subject areas. Each level includes courses either of similar academic challenge and difficulty or at the same stage in the progression of learning in that subject area. In the mathematics pipeline, for example, algebra I is placed at a level lower in the pipeline continuum than is algebra II because algebra I is traditionally completed before (and is generally less academically difficult or complex than algebra II). Classifying transcript data into these levels allows one to infer that high school graduates who have completed courses at the higher levels of a pipeline have completed more advanced coursework than graduates whose courses fall at the lower levels of the pipeline. Tallying the percentage of graduates who completed courses at each level permits comparisons of the percentage of high school graduates in a given year who reach each of the levels, as well as among different graduating classes. The high school courses taken by students are sorted into the academic levels of the pipeline after they have been organized according to the Classification of Secondary School Courses (CSSC) and the Secondary School Taxonomy (SST). All courses in a student’s transcript are coded with a CSSC value after checking course titles on the student’s transcripts with course catalogs from the student’s high school describing the contents of those courses. These coded courses are then assigned to broader course groupings, forming the academic levels of the pipeline in each subject area, using the SST. Transcript studies are a reliable source of information, but they do have limitations. One limitation is that transcript studies can describe the intended—but not the actual—curriculum. The content and instructional methods of one course taught in one school by a certain teacher may be different from the content and instructional methods of another course classified as having the same CSSC code taught in another school, or even the same school, by a different teacher. Nevertheless, validation studies and academic research have shown significant differences between the highest level of academic courses completed by students and their scores on tests of academic achievement (Chaney, Burgdorf, and Atash 1997; Berends, Lucas, and Briggs 2002). In classifying students’ courses from their transcripts according to a pipeline, only the courses completed with a passing grade in a subject area are included and not courses attempted. The pipeline also does not provide information on how many courses graduates completed in a particular subject area. Graduates are placed at a particular level in the pipeline based on the level of their highest completed course, regardless of whether they completed courses that would fall lower in the pipeline. Thus, graduates who completed year 3 of (or 11th-grade) French did not necessarily complete the first 2 years. Mathematics Pipeline Originally developed by Burkam and Lee (NCES 2003–01; NCES 2003–02), the mathematics pipeline progresses from no mathematics courses or nonacademic courses to low, middle, and advanced academic coursework. Each level in the pipeline represents the highest level of mathematics coursework that a graduate completed in high school. Thus, a graduate whose highest course is at the low academic level progressed no further in the mathematics pipeline and did not complete a traditional algebra I course, a prerequisite for higher level mathematics in high school. The mathematics pipeline has eight levels: no mathematics; nonacademic; low academic; middle academic I; middle academic II; advanced I; advanced II; and advanced III. Middle levels I and II and advanced levels I, II, and III can be combined to create one middle level and one advanced level, respectively, thus creating a five-level pipeline (no mathematics; nonacademic; low academic; middle academic; and advanced academic). No mathematics No coursework completed in mathematics by graduates, or only basic or remedial-level mathematics completed. It is thus possible for a graduate to have taken one or more courses in mathematics, but to be placed in the no mathematics level. Nonacademic level Highest completed courses are in general mathematics or basic skills mathematics, such as general mathematics I or II; basic mathematics I, II, or III; consumer mathematics; technical or vocational mathematics; and mathematics review. Low academic level Highest completed courses are preliminary courses (e.g., prealgebra) or mathematics courses of reduced rigor or pace (e.g., algebra I taught over the course of 2 academic years). Considered to be more academically challenging than nonacademic courses, courses at this level include prealgebra; algebra I, part I; algebra I, part II; and geometry (informal). Middle academic level The middle academic level is divided into two sublevels, each of which is considered to be more academically challenging than the nonacademic and low academic levels, though level I is not considered as challenging as level II.
Advanced academic level The advanced academic level is divided into three sublevels, each of which is considered more academically challenging than the nonacademic, low academic, and middle academic levels, though level I is not considered as challenging as level II, nor level II as challenging as level III.
Science Pipeline Unlike mathematics and other subjects, such as foreign languages, coursework in science does not follow a common or easily defined sequence. Depending on a school’s curriculum, students can choose from several courses with minimal sequencing requirements. Consequently, the method used to construct the science pipeline differs from that used to construct the mathematics pipeline. First, all science courses were placed in one of four groups based on subject matter: (1) life science (biology); (2) chemistry; (3) physics; and (4) all other physical sciences (e.g., geology, earth science, physical science). Second, a pipeline was constructed for each of these four groups. Third, the pipelines for chemistry, physics, and all other physical sciences were combined into a single pipeline (a physical science pipeline). Finally, the physical science and life science pipelines were combined to create a single science pipeline. The final pipeline has seven levels: no science; primary physical science; secondary physical science and basic biology; general biology; chemistry I or physics I; chemistry I and physics I; chemistry II or physics II or advanced biology. No science Includes graduates who did not complete any courses in science or who completed only basic or remedial-level science. It is possible for a graduate to have taken one or more courses in science but to be placed in the no science level. Primary physical science Highest completed course is in basic physical sciences: applied physical science; earth science; college preparatory earth science; or unified science. Secondary physical science and basic biology Highest completed course is astronomy; geology; environmental science; oceanography; general physics; basic biology I; or consumer or introductory chemistry. General biology Highest completed course is general biology I; secondary life sciences (including ecology, zoology, marine biology, and human physiology); or general or honors biology II. Chemistry I or physics I Highest completed course is introductory chemistry; chemistry I; organic chemistry; physical chemistry; consumer chemistry; general physics; or physics I. Chemistry I and physics I Highest completed courses include one level I chemistry course (see above) and one level I physics course (see above). Chemistry II or physics II or advanced biology Highest completed course is advanced biology; International Baccalaureate (IB) biology II; IB biology III; AP biology; field biology; genetics; biopsychology; biology seminar; biochemistry and biophysics; biochemistry; botany; cell and molecular biology; cell biology; microbiology; anatomy; miscellaneous specialized areas of life sciences; chemistry II; IB chemistry II; IB chemistry III; AP chemistry; physics II; IB physics; AP physics B; AP physics C: mechanics; AP physics C: electricity/magnetism; or physics II without calculus. CREDIT RATIOS FOR POSTSECONDARY COURSES Courses recorded on students’ transcripts were assigned 6-digit codes using the College Course Map (CCM), which is a modification of the Classification of Instructional Programs (CIP). NCES developed the CIP taxonomy in 1981 as a standard for reporting enrollments and credentials in postsecondary programs. Because the CIP taxonomy was developed to report on postsecondary programs, rather than postsecondary courses, a new taxonomy, the College Course Map (CCM), was developed that retained the basic CIP structure but is more appropriate for transcript analyses. The CCM taxonomy was first published in 1990 for use with the NLS:72/86 transcript data, was modified in 1999 for use with the HS&B-So:PETS data, and modified again in 2003 for use with the NELS:88/2000 transcript data. Each 6-digit code represents a discrete subject matter, or a “course,” and reflects the finest level of detail in the taxonomy. Specific course titles may vary across institutions. For example, “introduction to accounting” may be “accounting I” at some institutions, but all introductory accounting courses would have the same 6-digit code regardless of their actual title. The 30 most commonly completed courses for each cohort are identified using “credit ratios,” calculated by summing all the undergraduate credits earned in each of the more than 1,000 6-digit course categories and then dividing that sum by the total number of credits earned. Credit ratios were computed for each of the three weighted samples. Table 30-1 shows that the credit ratios for the “top 30” courses for the 1992 cohort range from 3.2 percent for English composition to 0.6 percent for introduction to computing. Adelman (forthcoming-a) suggests that with such a large number of course categories, for any one category to contain 0.5 percent of all credits represents a substantial amount. The institutional selectivity categories for the 1992 cohort in table 30-2 are from the American Freshman (Higher Education Research Institute 1992). The selectivity indicator includes five categories: “highly selective,” “selective,” “nonselective,” “open door,” and “not ratable” (principally less-than-2-year institutions and specialized conservatories of art and music). Institutions from the last two categories, “open door” and “not ratable,” are not included in the analysis reported in indicator 30. Selectivity is a relative measure based on a number of factors, including the ratio of acceptances to applicants and the average composite SAT score of students in the entering class. In the 1992 cohort, 7.2 percent of students earned a degree from highly selective institutions, 26.1 percent from selective institutions, and 65.8 percent from nonselective institutions. POSTSECONDARY REMEDIATION COURSES The remediation categories used in indicator 18 are based on the following “if-then-else” coding logic:
This coding logic identifies students with the most serious problem (reading) first. Reading was judged to be the most serious remedial problem because two-thirds of the students who required remediation in reading were also enrolled in a minimum of two other remedial courses. The second level identified students whose only remedial problem was mathematics, and who required, at most, two remedial mathematics courses. In the third level of the logic cascade, 60 percent of the students enrolled in three or more remedial courses other than remedial reading (the logic allowed this combination to include three or more mathematics courses). The fourth level identified those students who completed only one remedial course other than reading or mathematics. The residual group of students completed no remedial courses. |
 |
|
|
