Development of Statistical Standards at NCES

NCES first adopted written statistical standards in the spring of 1987. These standards were the result of a multi-year evaluation and planning process that included a recommendation for the development of statistical standards from the Committee on National Statistics at the National Academy of Science. With that recommendation, a statistical standards program was initiated at NCES in 1985. Using the Energy Information Administration's Standards Manual and the Census Bureau's technical paper on "Standards for Discussion and Presentation of Errors in Survey and Census Data," NCES staff, in consultation with outside experts developed the 1987 version of NCES statistical standards.

With the adoption of this first set of standards, the Agency Director called for a formal evaluation to start the following fall, to insure that the standards were fully implemented and to identify any difficulties with the standards. In 1989, the Center undertook a full-scale revision of the 1987 standards. The revisions were developed by NCES staff, and reflected their first-hand experiences in using the 1987 standards. After multiple reviews of interim drafts by NCES staff and the NCES Advisory Council of Education Statistics, NCES Senior Staff accepted the revised standards in the spring of 1992.

At the June 1992 release of the NCES Statistical Standards report, the Acting Commissioner summarized the standards in the following statement:

They: (1) codify how we expect to behave professionally, (2) indicate the basis on which we expect to be judged by our peers in the statistical community, (3) represent the quality we expect in any of our efforts or those of our contractors and grantees, (4) provide a means to assure consistency among the studies the Center conducts, and (5) document for users, the methods and principles the Center employs in the collection of data.

The Acting Commissioner also reiterated the Center's commitment to periodic evaluations of the implementation of the standards and to a periodic review of the standards' operational feasibility.

The current revision process began in the summer of 1999 with a review of existing standards from a number of national and international statistical policy agencies and committees and from other international and national statistical agencies. At the same time the 1992 NCES Statistical Standards were made available on the Web, and NCES staff were given a 30-day period to submit comments concerning potential revisions and additions to the NCES standards. Following these activities an agency-wide Steering Committee was formed to work on the standards revision process. The Steering Committee formed 15 Working Groups that comprised more than one-half of the NCES staff to work on the set of topics identified in the 1999 reviews.

Each Working Group drafted their assigned standards; each of which underwent a multi-step review process. Following a 30-day NCES staff comment period, the working group members made revisions, the Steering Committee reviewed the drafts and submitted them to Senior Staff. The drafts were then reviewed by Senior Staff, modified as necessary, and then shared with a group of 40 to 50 representatives of the contractors who work with NCES on data collection, analysis, and dissemination. Additional revisions were incorporated following the input from this broad group. NCES also commissioned the National Institute of Statistical Sciences to convene an independent review panel of statistical experts to review and comment on the draft standards prior to final acceptance by the Steering Committee and Senior Management. The standards on this website are the result of the efforts of the many persons who participated in this multi-stage review process, but ultimately NCES takes responsibilities for any lack of clarity or completeness.

During the recent NCES standards revision, the Office of Management and Budget (OMB) issued government-wide guidelines for ensuring and maximizing the quality of information disseminated by Federal agencies. The OMB guidelines direct all agencies covered by the Paperwork Reduction Act (44 U.S.C. chapter 35) to develop and implement procedures for reviewing and substantiating the quality of information disseminated by the agency. In order to meet these goals, each agency is required to develop and promulgate quality guidelines.

In response to the OMB guidelines, the federal statistical agencies collaborated to identify a set of activities that are essential to maintaining the quality and credibility of statistical data. The NCES draft revised standards are organized around the shared framework for federal statistical agencies. NCES remains committed to the principles outlined by the 1992 NCES Acting Commissioner; what is more, these principles are reaffirmed in the OMB call for data quality guidelines.

Continue ...