About GEMEnA
NOTE: The Interagency Working Group on Expanded Measures of Enrollment and Attainment (GEMEnA) is no longer active. This site provides historical documentation of GEMEnA’s work.
From 2009 to 2017, the Interagency Working Group on Expanded Measures of Enrollment and Attainment (GEMEnA) worked to develop and validate national measures of the participation in and credentialing of education and training for work, and to build government-wide consensus for the adoption of these measures in key federal data collections. More specifically, GEMEnA engaged in a rigorous process of survey item development to validate core items on 1) the attainment of non-degree credentials, including industry-recognized certifications, occupational licenses, and educational certificates, and 2) enrollment in education and training that prepares people for work.
GEMEnA developed a core set of survey items to measure the prevalence and key characteristics of certifications and licenses, which a number of federal agencies have added to their data collections. NCES fielded an in-depth survey of US adults, the Adult Training and Education Survey (ATES), to provide detailed data on credentials in 2016. Although ATES has been discontinued, many ATES items were adopted (or adapted) as part of the new National Training, Education, and Workforce Survey (NTEWS), sponsored by the National Center for Science and Engineering Statistics, within the National Science Foundation (NSF), and co-sponsored by NCES. The NTEWS is part of a broader NSF effort to understand the skilled technical workforce.
The Definitions tab provides GEMEnA's working definitions of certifications, licenses, and certificates. The Surveys link lists the federal data collections that use GEMEnA-developed items.
The Milestones link provides a timeline of GEMEnA's work.
GEMEnA consisted of staff from these federal offices:
- Bureau of the Census, U.S. Department of Commerce
- Bureau of Labor Statistics, U.S. Department of Labor
- Council of Economic Advisers
- National Center for Education Statistics, U.S. Department of Education
- National Center for Science and Engineering Statistics, National Science Foundation
- Office of Statistical and Science Policy, Office of Management and Budget
- Office of the Under Secretary, U.S. Department of Education
To ensure the relevance of GEMEnA's work for answering critical policy and research questions, GEMEnA met regularly with a panel of experts and collaborated with a wide range of partners and stakeholders.
Definitions
Working Definitions of Non-Degree Credentials
GEMEnA has developed the following working definitions of industry-recognized certifications, occupational licenses, and educational certificates. To maintain consistency with federal statistical data, GEMEnA encourages those who collect information on the attainment of these credentials to align such collections with these definitions. To the extent that these definitions are used as the basis for legislation, regulation, accountability systems, and administrative data systems, users will be able to compare results to nationally-representative data.
Certification: A credential awarded by a certification body based on an individual demonstrating through an examination process that he or she has acquired the designated knowledge, skills, and abilities to perform a specific job. The examination can be either written, oral, or performance-based. Certification is a time-limited credential that is renewed through a recertification process.
License: A credential awarded by a government agency that constitutes legal authority to do a specific job. Licenses are based on some combination of degree or certificate attainment, certifications, assessments, or work experience; are time-limited; and must be renewed periodically.
Educational certificate: A credential awarded by an educational institution based on completion of all requirements for a program of study, including coursework and test or other performance evaluations. Certificates are typically awarded for life (like a degree). Certificates of attendance or participation in a short-term training (e.g., 1 day) are not in the definitional scope for educational certificates.
Surveys
The exhibit below provides an overview of federal data collections that include questions on education, training, and non-degree credentials for work.
Federal Surveys with Certification and Licensure Questions | |||
|---|---|---|---|
| Survey/Office | Administration cycle | Population | Notes and links |
| Education Longitudinal Study (ELS), NCES, ED | 10-year longitudinal study | Cohort of students who were in grade 10 in 2002 (Class of '04) | Seven questions included in last follow-up in 2012, when students were about 26. Link to ELS website. |
| High School Longitudinal Study (HSLS), 2nd Follow-up, NCES, ED | 8-year longitudinal study | Cohort of students who were in grade 9 in 2009 (Class of '13) | Questions first included in 2016 2nd follow-up, when students were about 20. Link to HSLS website. |
| National Postsecondary Student Aid Study (NPSAS), NCES, ED | Cross-sectional survey, every 4 years | Students enrolled in Title- IV-eligible postsecondary insitutions | Four questions first included in the 2015-16 administration. Link to NPSAS website. |
| Beginning Postsecondary Students Survey (BPS), NCES, ED | 6-year longitudinal study; new cohort every 8 years | Beginning postsecondary students enrolled for first time | Two questions first included in 2014 follow-up of 2011-12 beginning students. Link to BPS website. |
| Baccalaureate and Beyond (B&B), NCES, ED | 10-year longitudinal study; new cohort every 8 years | Recent bachelor's degree recipients | Two questions first included in 2012 follow-up of 2008 bachelor's degree recipients. Link to B&B website. |
| Current Population Survey (CPS), Census Bureau | 8-month rotation panels | Civilian noninstitutional adults age 16 years and over | Three questions first asked of all rotation panels in Jan. 2015 and in months 1 and 5 thereafter. Link to CPS tables. Link to CPS. |
| Wave 13, 2008 Survey of Income and Program Participation (SIPP), Census Bureau | Summer-Fall 2013 | Adults in households who are at least 18 or have finished high school | Nine questions included in the final wave (wave 13) of the 2008 SIPP. Data and report released Jan. 2014. Link to SIPP report. |
| Survey of Income and Program Participation (SIPP), Census Bureau | 2.5 to 4-year rotating panel | Adults in households who are at least 18 or have finished high school | Two questions included in initial wave beginning in 2014. Link to SIPP website. |
| Program for the International Assessment of Adult Competencies (PIAAC), NCES, ED (in partnership with the Organization for Economic Cooperation and Development) | Cross-sectional survey, every 10 years | Adults in households, ages 16 to 65 | Two questions to be included in the U.S. version of the 2022-23 PIAAC Background Questionnaire. Link to PIAAC website. |
| Adult Training and Education Survey (ATES), part of the 2016 National Household Education Survey (NHES), NCES, ED | One-time administration | Adults in households, ages 16 to 65 | Included detailed questions on adults' 3 most important certifications and licenses. Link to NHES website. |
| National Survey of College Graduates (NSCG), National Center for Science and Engineering Statistics (NCSES), National Science Foundation | 4-cycle biennial rotating panel; new cohort every 2 years | Adults under age 76 with at least a bachelor's degree | Eleven questions first included in 2015 administration. Link to NSCG website. |
| National Training, Education, and Workforce Survey (NTEWS), National Center for Science and Engineering Statistics (NCSES), National Science Foundation | 2022 (planned); 4-cycle biennial rotating panel; new cohort every 2 years | Adults in households, ages 16 to 75 | Will include detailed questions on adults' 2 most important certifications and licenses. Link to NSF STW website. |
GEMEnA Validated Core Certification/Licensure Survey Item
The four validated core survey questions ask:
- Whether the adult has a certification or license
- Who provided the certification or license (used to distinguish between certifications and licenses)
- The subject area of the certification or license
- Whether the certification or license is required for the adult's current job
Milestones
| Date | Event | Description |
|---|---|---|
| Feb 2018 | Release of first full-scale ATES data | The National Center for Education Statistics releases detailed data on the prevalence and characteristics of work credentials in the U.S. population from the 2016 Adult Training and Education Survey, conducted as part of the National Household Education Survey. |
| April 2016 | Release of first CPS data on certifications and licenses. | The Bureau of Labor Statistics releases 2015 CPS data on the prevalence of certifications and licenses in the civilian noninstitutional population 16 years and over. |
| Jan.-Jul. 2016 | Administration of ATES within NHES:2016 | Based on the findings of the 2014 NHES feasibility study and the development work listed below, the ATES instrument was revised and administered as part of the NHES study in 2016. The administration involved a national household sample of adults ages 16-65, and will produced the Education Department's first statistics on the prevalence and key characteristics of non-degree credentials among the US adult population. Link to ATES questionnaire |
| Jan. 2015 | CPS collects data on certifications and licenses | The Current Population Survey fielded three new items on certifications/licenses with all rotation groups in January and February of 2015. It will continue collecting these data in months 1 and 5 on an ongoing basis. |
| Jan. 2015 | NSCG collects data on certifications and licenses | The 2015 administration of the National Survey of College Graduates included eleven questions on certifications and licenses in the college graduate population. |
| Dec. 2014 | Third Expert Panel Meeting | At this meeting, the panel provided feedback on a plan to collect data on the "most important" rather than the "most recent" certification/license and discussed the results of the NATES Nonresponse Bias Study. Link to the meeting notes. |
| Mar. 2014 | Second Expert Panel Meeting | At this meeting, the panel provided feedback on the certificates and initial job training programs sections of ATES in preparation for the 2016 NHES administration. |
| Feb.-Aug. 2014 | Administration of 2014 National Household Education Study (NHES) Feasibility Study | This study was designed to test the feasibility of administering the Adult Training and Education Survey (ATES) as part of NCES' on-going NHES Program. One test in this study involved comparing response rates for a longer (ATES) version of the questionnaire that asks about both credentials and training for work, and shorter "Credentials for Work" (CWS) instrument that focused on work credentials. A report on the ATES feasibility study will be posted in late 2015. |
| Jan.-Apr. 2014 | Cognitive interviews on the 2016 ATES instrument | These interviews were used to test and modify the wording of the final 2014 ATES instrument for an expected full-scale administration within the 2016 NHES. The interviews focused on questions that were still undergoing revision at the end of the 2014 ATES cognitive interviews, including how to capture educational certificates, and on refining the apprenticeship section based on the concept interviews. |
| Jan. 2014 | Release of first official federal statistics | Release of the first official federal statistics on the prevalence of certifications and licenses in the U.S. adult population from Wave 13 of the 2008 Survey of Income and Program Participation: Measuring Alternative Educational Credentials: 2012. |
| Dec. 2013-Jan. 2014 | Concept interviews on formal training | The cognitive interviews revealed that adults include many programs under the "apprenticeship" umbrella. These interviews were designed to identify the key characteristics of formal training experiences that distinguish the different programs that adults describe as apprenticeships, including internships, externships, practica, and student teaching. Link to the concept interview report. |
| Aug.-Sep. 2013 | Cognitive interviews on the 2014 Adult Training and Education Survey (ATES) instrument | These interviews were used to test and modify the wording of the initial 2014 ATES instrument. |
| May 2013 | Focus groups on work-related training | A planned 2014 field-test of NCES' National Household Education Survey (NHES) provided the opportunity to test the feasibility of incorporating a survey on adult education and training into this on-going data collection. For this purpose, further development work was conducted. First, GEMEnA sponsored a background report on participation in non-credit education and training. Second, focus groups (see report) provided information on how adults describe these education and training experiences. That information was then used to revise NATES for the 2014 Adult Training and Edcation Survey (ATES), to be field-tested as part of NHES. |
| Jan.-May 2013 | Administration of 2013 National Adult Training and Education Survey (NATES) response rate study | For household surveys, NCES is moving from CATI to mailout administration of paper-and-pencil surveys. The NATES study was conducted to determine whether a mailout survey asking about adults' education and training could yield adequate response rates, and included a field-based non-response bias study. A final report is due in Summer 2014. Link to NATES instrument. |
| Nov. 2012 | First Expert Panel meeting | The Expert Panel was formed to provide an external sounding board for GEMEnA's work. At this meeting, the panel was informed of GEMEnA's work to-date, and suggested areas of focus to guide subsequent instrument design. Link to the meeting summary. |
| Sep. 2012 | IWG name changed to Interagency Working Group on Expanded Measures of Enrollment and Attainment (GEMEnA) | In 2012, the IWG decided to expand its work to include the development of measures related to training for work. This new name was adopted to reflect the group's expanded charge. The measurement of certificates, certifications, and licensure remain as more specific goals within this broader framework. |
| Sep.-Oct. 2012 | Second round of focus groups on certificates | Because the 2010 ATES found educational certificates to be a particuarly difficult concept to meaure, these focus groups were conducted to provide more information on how adults describe these certificates. Findings were used to draft the certificate section of the 2013 NATES instrument. Link to the focus group report. |
| Aug.-Sep. 2012 | Cognitive interviews on the 2013 National Adult Training and Education Survey (NATES) instrument | The 2013 National Adult Training and Education Survey (NATES) was a revision of the 2010 ATES, and was designed to test the feasibility of a mail-out administration (see below). These interviews were used to test and modify the wording of the initial NATES instrument. |
| Jan. 2012 | Presentation on ATES at Federal Committee on Statistical Methodology research conference | The presented paper on "Measurement Strategies for Identifying Holders of Certificates and Certifications" focused on the identification of over- and under-reporting within the 2010 ATES. |
| Sep. 2010-Jan. 2011 | Administration of 2010 Adult Training and Education Survey (ATES) pilot study | The 2010 ATES pilot study was designed as an initial test of key questions on certificates, certification, and licensure. It used a 15-minute CATI (computer-assisted telephone interview) instrument. A seeded sample also provided a validity check. See the ATES Pilot Study report for details, including a copy of the instrument in Appendix E. |
| Mar.-Jul. 2010 | Cognitive interviews on the 2010 Adult Training and Education (ATES) instrument | These interviews were used to test and modify the wording of the initial 2010 ATES pilot test instrument. A summary of the cognitive interviews is included as Appendix G in the ATES Pilot Study report. |
| Dec. 2009 | Focus groups on educational certificates and certification | These focus groups provided information on how adults describe educational certificates, certification, and licensure; this information was used to draft the initial version of the 2010 Adult Training and Education Survey (ATES) pilot test instrument. A summary of the focus groups is included as Appendix F in the ATES Pilot Study Report |
| Dec. 2009 | Interagency Working Group on Certificates and Certifications (IWG) begins monthly meetings | This first name of the group reflects the initial focus on certifications and certificates. Link to monthly meeting minutes. |
| Nov. 2009 | Subbaccalaureate Credentials Roundtable, Brookings Institute, Washington, DC | This meeting reiterated the need for data on educational certificates and professional certification and licensure. At the meeting, the Census Bureau, Education Department, and Bureau of Labor Statistics agreed to collaborate to explore the potential for filling these data gaps. Link to meeting notes. |
| Dec. 2000 | Report on the Federal Interagency Committee on Educational Attainment | The committee recommended that statistical agencies develop a taxonomy of occupational certifications and licenses and survey items to measure their prevalence. No further action was taken at the time. Link to educational attainment report. |
Partners and Stakeholders
Federal Statistical Agency Partners:
- Bureau of Labor Statistics, U.S. Department of Labor
- Census Bureau, U.S. Department of Commerce
- National Center for Education Statistics, U.S. Department of Education
- National Center for Science and Engineering Statistics, National Science Foundation
Other Federal Partners:
- Council of Economic Advisers, Executive Office of the President
- Office of Statistical and Science Policy, Office of Management and Budget, Executive Office of the President
- Office of the Under Secretary of Education, U.S. Department of Education
Expert Panelist Organizations:
- ACT Foundation
- American Association of Community Colleges
- Center for Regional Economic Competitiveness
- The Center for Law and Social Policy (CLASP)
- Council for the Advancement of Adult Literacy
- Education and Employment Research Center, Rutgers University
- Georgetown Center for Education and the Workforce, Georgetown University
- George Washington Institute of Public Policy, George Washington University
- Humphrey School of Public Affairs, University of Minnesota
- Lincoln Educational Services
- Lumina Foundation
- National Association of State Workforce Agencies
- Northern Virginia Community College
- Prince George's Community College, Workforce Development and Continuing Education
- Richard and Rhoda Goldman School of Public Policy, University of California at Berkeley
Federal Stakeholders:
- Administration for Children and Families, U.S. Department of Health and Human Services
- Better Buildings Initiative (Workforce), U.S. Department of Energy
- Directorate for Education and Human Resources, National Science Foundation
- Employment and Training Administration, U.S. Department of Labor
- Federal Aviation Administration, U.S. Department of Transportation
- Health Resources and Services Administration, U.S. Department of Health and Human Services
- National Institute of Standards and Technology, U.S. Department of Commerce
- Office of Career, Technical, and Adult Education, U.S. Department of Education
- U.S. Department of Defense
- U.S. Department of Veterans Affairs
Other Stakeholders:
The Expert Panel
The GEMEnA Expert Panel provided guidance and support to GEMEnA's four strands of work. The Panel met regularly to provide periodic review of GEMEnA plans and products.
The members of the Expert Panel were (affiliations were those at the time of GEMEnA's work):