Periodically, statistical questions arise that break new ground for NCES or in some cases the discipline. The Statistical Standards Program consults and advises in these situations, and in some instances convenes a panel of experts to consult on or review specific problems. Examples of this process include a review of the response probability decision rules that are applied in NCES assessments. In this case, analysis conducted by NCES Assessment Division staff identified an issue that was pursued in a collaborative analysis with staff at the Education Statistical Services Institute. The results of these analyses were the focus of several NCES seminars and at the request of the Advisory Council an expert review panel was convened to study these issues. Another example involved a review of procedures used across NCES data collections for computing response rates. In this case, we invited the National Institute of Statistical Sciences to commission a panel to review and comment on the procedures used by NCES. The results of that review were used by NCES staff in the revision of standards related to response rates and imputation.
The Statistical Standards Group recently sponsored the development of a handbook of NCES data collections that documents key aspects of the design, collection, and processing of NCES surveys. Related reports that summarize nonresponse rates and imputation procedures used in specific NCES data collections are also under development. The Program is currently sponsoring software development and testing in several areas concerning the analysis of complex sample survey data, and in statistical methods to protect confidential data.
The Statistical Standards Program has initiated potentially long-term research in two areas. The ESSI will extend some earlier developmental work they conducted on total survey error for NCES; this project will be monitored by Program staff, but conducted independently by ESSI staff. The second project involves an increased level of participation on the part of Program staff. The second area involves two separate projects on the imputation of complex sample survey data—one takes a design-based approach and the other takes a model-based approach.