|View Quarterly by:
This Issue | Volume and Issue | Topics
|This article describes the NCES Data on Career/Technical Education Statistics (CTES), this issue's featured topic.|
Vocational education is designed to ensure that students obtain marketable job skills that complement their academic skills. While the importance of obtaining job skills has long been recognized for students who do not continue their education beyond high school, these skills are also important for the large number of students who need or want to combine work with college attendance, who are unsure of their future education plans, who plan to earn a subbaccalaureate degree (i.e., a postsecondary credential below the baccalaureate level), or who plan to enter a technical field for which a "hands-on" or applied curriculum provides valuable groundwork for more abstract study in later years. Yet this important sector of the education system is often overlooked. Most people would be surprised to learn what we in fact know about vocational education-for example, that almost all public high school students take at least one vocational education course, that 16 percent of all public high school credits are earned in vocational education, and that 49 percent of all students seeking subbaccalaureate degrees major in vocational fields.
These statistics were derived from the National Center for Education Statistics (NCES) Career/Technical Education Statistics (CTES), a data collection and reporting system designed to provide detailed information on vocational education at the national level. As discussed below, the CTES was developed to meet Congress' need for data to inform federal vocational education legislation. This information is also of use to state and local vocational education administrators, who need data to support their efforts to allocate resources, defend program expenditures, and develop policies and programs for vocational education. By focusing on a major sector of secondary and postsecondary education, this information also helps provide a more complete description of the American education system.
The CTES differs significantly from the typical NCES data collection effort because it is a "derived," or "synthetic," system that collects data from a wide range of preexisting education surveys. Since vocational education is embedded in the larger framework of general education, it makes sense that the data collection system for vocational education should itself be embedded in general education surveys. But this "sensible" system belies a troubled past; NCES collection of Career/Technical Education Statistics (CTES) has a relatively brief but tumultuous history.
History of CTES1
The federal government has supported vocational education programs since 1917, when the Smith-Hughes Act was passed to help schools train workers for the country's rapidly growing economy. In the 1960s, the focus of federal legislation shifted to ensuring that all students had equal access to vocational education programs. With this shift in focus (and with improved procedures for data collection and analysis) came an interest in collecting detailed national data to track student participation in vocational education. As a result, Congress instituted new requirements for states to include information on vocational education expenditures and student enrollments in their annual reports to Congress. Vocational education enrollments were to be broken out by race, student disability status, and other student characteristics, for all secondary, postsecondary, and adult education students in the state.
Initially, responsibility for collecting these data was given to the Bureau of Occupational and Adult Education (BOAE), the precursor to today's Office of Vocational and Adult Education (OVAE). Unfortunately, because these new data requirements were beyond most states' recordkeeping capabilities at the time, missing, inconsistent, and noncomparable data were common. To correct these problems, Congress established Project Baseline to work with individual states to improve their administrative records and data submissions. Nonetheless, the BOAE data collection remained problematic and was discontinued in 1976.
But Congress had not given up. The 1976 Amendments to the Vocational Education Act maintained the requirement for data collection on all schools and students, but moved responsibility for the collection of these data from BOAE to NCES. Given the problems encountered by BOAE, NCES spent almost 2 years designing a new data collection system, called the Vocational Education Data System (VEDS). But VEDS still had to rely on the collection of data from state administrative records, and this information remained intractable.
NCES was not oblivious to this problem. A 1983 internal validity study of VEDS data concluded that the data suffered from a lack of comparability. Not only were data inconsistent from one state to another, they were also inconsistent within individual states, when data from different state sources were compared. Data also varied from year to year in inexplicable ways. Comparability was also limited by some states' continued inability to provide complete data.
Many factors accounted for these data problems, but the root cause was simply that the data collection system was inappropriate for the task at hand. State administrative records are designed to meet the needs of states and localities, and thus reflect the unique education conditions, policies, and structures of each state, as well as their specific data-gathering formats and capabilities. For example, one state may choose to count vocational education enrollments at the course level, so that a student who enrolls in two vocational courses would count as two course-enrollments; another state might count these enrollments at the student level, so that the same student would count as one student-enrollment. States may also differ in the courses included in their definition of vocational education or may categorize students differently by race, economic status, and other measures. Finally, the collection of records data on all schools, teachers, or students is costly and time-consuming; most states can afford to collect such information for only the most basic and critical features of their education systems. VEDS was in effect asking states to use their administrative records in ways for which they were not intended and which may, in some cases, have decreased their value for state purposes while at the same time increasing costs.
Although both Project Baseline (prior to 1976) and NCES had worked with states to implement quality control procedures-and many improvements were made-the complexity of the data required by the VEDS mandate ultimately extended beyond the limits of this approach to data collection. These continuing problems led the Office of Management and Budget (OMB) to deny approval for the collection of VEDS data after 1983-making VEDS the only NCES data collection to have this ignoble distinction. VEDS also became a key example of the need for larger changes within NCES. In 1986, the National Academy of Sciences released a report evaluating the quality of NCES data. The report noted a number of data quality problems in existence at that time and cited VEDS as illustrative of "virtually every problem encountered in our review" (Levine 1986, p. 15).
Difficult though VEDS may have been for NCES at the time, it motivated a number of changes that have improved the quality of NCES data. Chief among these changes were new review processes for data collections and reports, and the implementation of cooperative systems between NCES and state and local education representatives. These cooperative systems are designed to ensure that consistent standards are established and maintained whenever NCES data collections rely on the voluntary cooperation of states and localities. Finally, the VEDS experience led Congress to change its mandate for vocational education data-a change that led directly to improvements in the NCES vocational education data collection system. From the ashes of VEDS, the CTES arose.
The Carl D. Perkins Vocational Education Act of 1984 directed NCES to develop "a national vocational education data reporting and accounting system using uniform definitions." While this mandate required the collection of information as detailed as that collected by VEDS, it differed from VEDS in one critical way-the 1984 mandate allowed NCES to collect data using sample surveys rather than census counts. This change freed NCES from a reliance on state administrative records, the data source that had proved infeasible in VEDS.
With the lessons learned from VEDS and the flexibility to use sample surveys, NCES fashioned a new approach to vocational education data collection: the Career/Technical Education Statistics (CTES). The CTES "collects" Career/Technical Education Statistics by consolidating and analyzing information from existing NCES education surveys, supplemented by relevant surveys conducted by other federal offices and by special-purpose NCES surveys conducted through the Fast Response Survey System (FRSS) and the Postsecondary Education Quick Information System (PEQIS).
The CTES has three main advantages over VEDS. Most importantly, CTES data collections are almost all nationally representative sample surveys that use common, comparable definitions and data elements. (A few CTES data collections are census collections that rely on cooperative systems to ensure valid data reporting.) It is because of its uniformity in data definitions and data collection strategies that the CTES can effectively overcome the data-quality problems that plagued VEDS. Second, the CTES approach allows NCES to relate vocational education to the larger education system or to other parts of the system. So, for example, postsecondary students who major in vocational fields can be compared to students who major in academic fields, high school students who take extensive numbers of vocational education courses can be compared to students who take little or no vocational education, and teachers of vocational education courses can be compared to teachers of academic or other elective courses. Third, the VEDS reliance on administrative records data provided a limited data set that at best (if it could have been collected in a consistent and comparable manner) would have yielded counts of students, faculty, courses, and other school features. Little can be done with administrative records data beyond such basic counting. In contrast, the wide range of cross-sectional and longitudinal surveys in the CTES provides a much richer data source for policy analysis and research purposes. With CTES, one can explore, for example, how vocational education coursetaking interacts with academic coursetaking, how students who complete vocational education programs transition to the labor market or further education, and how the attrition rate of high school vocational education teachers compares to that of other teachers.
The CTES can be divided into two major components: first, a data collection component; second, a data-reporting component that disseminates the findings from analyses of CTES data.
Data collection component
With the exception of special-purpose FRSS or PEQIS surveys, the CTES data collection component relies on extant national surveys that contain information relevant to vocational education. Collectively, these data sources provide information on the vocational education system at the secondary and postsecondary levels, and on adult education and training taken for work-related reasons. In table 1, the CTES data collections are divided into surveys of the secondary-level education system, surveys of the nal surveys that span multiple education levels.
Data on student participation in vocational education provides a specific illustration of how CTES is used to generate information. The CTES collects data on participation in vocational education from three main sources. At the secondary level, NCES routinely collects detailed information on the coursetaking of high school students from high school transcript studies. The most recent transcript study was for the high school class of 2000. These data provide a wealth of information on participation in vocational education that is far more accurate and complete than that obtained from other sources, such as student self-reports. These data have told us, for example, that
At the postsecondary level, the National Postsecondary Student Aid Study (NPSAS) provides information on students' major program of study. In combination with the Beginning Postsecondary Students Longitudinal Study (BPS), these data can also be used to monitor the persistence rates and labor market outcomes of students who pursue vocational majors. Data collected indicate that
Analysis and reporting component
The data collections in the CTES provide the raw data for analyses. The specific analyses to be conducted are determined based on data availability, collaboration with other U.S. Department of Education offices, and input from experts in the field. To disseminate the findings from these analyses, NCES produces a number of reports. At the core of this reporting system is Vocational Education in the United States, a quadrennial publication. This report provides a comprehensive summary of the condition of vocational education, based on the most recent data available from all CTES sources. To date, three editions of Vocational Education in the United States (in 2000, 1995, and 1992) have been published. Between editions of this report, NCES releases a number of more focused reports on specific vocational education topics or findings from specific survey efforts. The CTES reports published since 1990 are listed in table 2.
NCES is currently preparing vocational education reports on
CTES and the National Assessment of Vocational Education
In addition to providing the data for NCES analyses and reports, the CTES also contributes to the National Assessment of Vocational Education (NAVE), which has been mandated by Congress in each authorization of federal vocational education legislation since 1976.2 Conducted by the Department of Education in conjunction with an independent advisory panel, NAVE is designed to inform debate on the reauthorization of federal legislation by providing information on vocational education in general and on the implementation of federal vocational education legislation in particular. Although NAVE studies typically conduct original data collections, the CTES provides an important source of supplemental information. For example, transcript studies provided key data for the last two NAVE reports to Congress, in 1989 and 1994. For the current NAVE, two upcoming CTES reports examining trends in student participation are being developed in cooperation with NAVE staff to provide data for their 2002 report to Congress.
Improving the CTES
Since the initiation of the CTES in the 1980s, NCES has continuously worked to expand and improve the system. These improvement efforts include the convening of a Vocational Education Advisory Panel; the establishment of a Vocational Education Technical Review Panel (TRP) and a CTES planning document; the development of a coordination system for working with OVAE; the adoption of a system for the review of survey instruments that feed into the CTES; and the establishment of a CTES Web Site.
Vocational Education Advisory Panel
In 1997, NCES sponsored a 1-day meeting of representatives of U.S. Department of Education offices, national education associations, state education administrators, and independent researchers, all of whom were familiar with vocational education at the secondary, postsecondary, or adult education levels. This advisory panel was convened to advise NCES on the vocational education data that were most needed and on how NCES vocational education publications could be improved. This meeting proved so useful that it led to the creation of a permanent Vocational Education TRP.
Vocational Education Technical Review Panel
Drawing largely from the original Vocational Education Advisory Panel, the Vocational Education TRP was established in 1998 to regularly provide input on vocational education data collection and reporting needs. Working with our main CTES contractor, the TRP provides advice on the content of survey instruments and on research and reporting priorities for vocational education. It is at the request of the Vocational Education TRP that the CTES is producing its first report on adult work-related education.
To help guide the work of the Vocational Education TRP, NCES has supported the development of a written "planning document." This document outlines the key issues that should be addressed by the CTES, the data available to address these issues, timelines for the revision of CTES data collection instruments and the release of new survey data, and a CTES publications plan. The planning document is designed to be a "living document" that is continuously updated as data collections and research priorities evolve. Collectively, the information in the planning document allows for better planning and prioritizing of future CTES activities.
Coordination with OVAE
The Department of Education's Office of Vocational and Adult Education also collects information on vocational education, both to monitor compliance with federal legislation and, through the research of the National Research Center for Career and Technical Education, to describe and improve program practice. To ensure that the CTES effectively complements OVAE's data collection efforts, an OVAE point-of-contact has been established. The CTES coordinator and OVAE contact person regularly discuss data issues of mutual concern.
Survey instrument review
Most NCES survey studies have a TRP that provides input on survey development and survey administration issues. As part of the preparation for each survey administration, the relevant TRP meets to review the content of the survey instrument, sampling procedures, and other survey issues. The CTES coordinator attempts to have formal or informal involvement in this TRP review process for all data collections within the CTES. It is through this process that the CTES data collections can be modified to collect more reliable and complete Career/Technical Education Statistics.
CTES Web Site
To help disseminate information about the CTES as well as information from the CTES, a CTES Web Site3 has been established within the NCES Web Site. The CTES Web Site provides a brief overview of the purpose and structure of the CTES, lists the data collections in and reports from the CTES, and provides links to other sites with information on vocational education. The Web Site also includes the latest edition of the CTES Update, a quarterly newsletter for Vocational Education TRP members.
Since its first struggles with VEDS in the 1970s, NCES has made enormous strides in constructing a vocational education data collection and reporting system that meets the highest statistical standards. The success of this effort is reflected in the fact that federal legislation since 1984 has supported the CTES by continuing to authorize NCES to collect and report vocational education data from national sample surveys. Nonetheless, the CTES faces a number of continuing challenges. In the "Note From NCES" introducing this issue of the Quarterly, Val Plisko describes some of the more general challenges faced by CTES because of its structure as a synthetic, or derived, data system. In this section, some of the challenges that arise from the unique nature of the vocational education system are noted.
First, vocational education is changing. Depending on where one looks, the changes may look more like an evolution or more like a revolution, but overall change is the norm. These changes reflect ongoing reforms and debate about the nature of vocational education-including who it should serve, its curriculum focus, even what it should be called (e.g., the former American Vocational Association is now called the Association for Career and Technical Education). It is difficult to determine both what should be measured in the CTES and how it should be measured, when knowledgeable people disagree on what vocational education is or should do. For example, not everyone agrees whether the new applied academics courses that are growing in popularity should be considered vocational or academic. The "career cluster" concept is changing notions about the value of depth versus breadth of preparation in secondary-level vocational education. There is also debate about the relative importance of the various goals that have been espoused for vocational education-such as providing students with general preparation for the labor market, providing students with job-specific skills, reducing high school dropout rates, preparing students for further education at the postsecondary level, and ensuring that disabled and disadvantaged students have equitable access to technical and work-related education. This debate also includes dissenting views on the extent to which vocational education programs should focus on academic skills and, by extension, the extent to which vocational education should be evaluated based on students' academic performance.
The process of change itself is also problematic for the CTES. While it is generally agreed that the changes and reforms occurring in vocational education are largely positive, monitoring a system in the midst of reform is like shooting at a moving target. It becomes particularly difficult to assess outcomes, since by the time we can measure the outcomes of today's vocational education system, the system that produced those outcomes will most likely no longer exist.
An additional problem arises from the absence of a clear and consistent high school vocational curriculum path. Because of this lack of a structured, definable program, it is unclear who should count as a vocational education participant or what it means to participate in the high school vocational education curriculum-particularly when 97 percent of all public school students take at least one vocational education course.
We employ a number of strategies to deal with these challenges. First, we attempt to reach consensus on what the CTES should measure by seeking input from the Vocational Education TRP, OVAE staff, and NAVE staff (when appropriate), as well as constituent feedback obtained at professional meetings, through the CTES Web Site, or via each report's customer feedback form. Second, every attempt is made to use only the most recent data to describe the vocational education system and to report those data in a timely manner. The CTES planning document is helpful in this regard, as it facilitates the scheduling of data analyses and reporting to coincide with new data releases. Third, we try to describe the vocational education system and its participants in a variety of ways that can provide more than one perspective on a given issue.
Defining student participation at the secondary level provides a good example of the use of multiple approaches to describing vocational education. In an upcoming report on this topic, we define participation in a variety of ways, reflecting different levels and types of involvement in the vocational curriculum. One measure simply counts the number of vocational education courses students take. Another measure counts the number of courses students take across all occupational preparation areas of the vocational curriculum (e.g., agriculture, business); students who take at least three such courses are called "vocational investors." At another level are "vocational concentrators," students who take at least three courses in any single occupational preparation area (e.g., three courses in agriculture). Other measures factor in advanced coursetaking and work experience credits. We are also considering using vocational-academic curriculum linkages to describe participation in vocational education; for example, reporting the percentage of students who took a drafting course who also took a geometry course.
Although the uncertainties and debates in vocational education can make working on the CTES difficult, these problems also make working with CTES quite interesting. As we strive to meet the challenges facing CTES, we welcome feedback on both our progress in developing the CTES and our lack of progress. Meanwhile, we hope the CTES will continue to contribute to a greater awareness and understanding of the role of vocational education within the larger education system.
1 This section draws heavily from a report by Hoachlander and Levesque (1993) that includes a more detailed history of the collection of national Career/Technical Education Statistics.
2These mandates occurred in 1976, 1984, 1990, and 1998.
Hoachlander, E.G., and Levesque, K. (1993). Improving National Data for Vocational Education: Strengthening a Multiform System. Berkeley, CA: National Center for Research in Vocational Education, University of California at Berkeley.
Levine, D.B. (Ed.). (1986). Creating a Center for Education Statistics: A Time for Action. National Research Council, Commission on Behavioral and Social Science and Education, Committee on National Statistics. Washington, DC: National Academy Press.