Table of Contents

Forum Guide to Supporting Data Access for Researchers: A State Education Agency Perspective National Cooperative Education Statistics System

Letter from John Easton Director of the Institute of Education Sciences

Working Group Members

Foreword

The National Cooperative Education Statistics System

Development of Forum Products

Relationship of this Guide to the Family Educational Rights and Privacy Act

Chapter 1 Data Partnerships: An Opportunity to Benefit Education Agencies and the Research Community

Introduction: Data Partnerships Improve Education and Research

Foundations for Data Sharing

Challenges to Sharing Data

Document Purpose and Audience

Improving Data Access for Researchers

Chapter 2 Core Practices for Effectively Managing Data Requests

Core Practice 1: Help Researchers Understand Agency Data and the Data Request Process

Core Practice 2: Create Effective Data Request Forms for Researchers

Core Practice 3: Review Data Requests Strategically

Core Practice 4: Manage the Data Request Process Efficiently

Core Practice 5: Release Data Appropriately

Core Practice 6: Monitor Data Use

End Notes

Appendix A

Appendix B

Appendix C

Appendix D

Appendix E

Appendix F

Appendix G

Appendix H

Forum Guide to Supporting Data Access for Researchers: A State Education Agency Perspective National Cooperative Education Statistics System

The National Center for Education Statistics (NCES) established the National Cooperative Education Statistics System (Cooperative System) to assist in producing and maintaining comparable and uniform information and data on early childhood, elementary, and secondary education. These data are intended to be useful for policymaking at the federal, state, and local levels.

The National Forum on Education Statistics (the Forum) is an entity of the Cooperative System and, among its other activities, proposes principles of good practice to assist state and local education agencies in meeting this purpose. The Cooperative System and the Forum are supported in these endeavors by resources from NCES.

Publications of the Forum do not undergo the same formal review required for products of NCES. The information and opinions published here are those of the Forum and do not necessarily represent the policy or views of the U.S. Department of Education or NCES.

July 2012

This publication and other publications of the National Forum on Education Statistics may be found at the websites listed below.

The NCES Home Page address is http://nces.ed.gov

The NCES Publications and Products address is http://nces.ed.gov/pubsearch

The Forum Home Page address is http://nces.ed.gov/forum

This publication was prepared in part under Contract No. ED-CFO-10-A-0126/0002 with Quality Information Partners, Inc. Mention of trade names, commercial products, or organizations does not imply endorsement by the U.S. Government.

Suggested Citation

National Forum on Education Statistics. (2012). Forum Guide to Supporting Data Access for Researchers: A State Education Agency Perspective. (NFES 2012-809). U.S. Department of Education.Washington, DC: National Center for Education Statistics.

Technical Contact

Ghedam Bairu

(202) 502–7304

ghedam.bairu@ed.gov

Letter from John Easton Director of the Institute of Education Sciences

Dear Reader,

Welcome to this first-of-its kind publication, the Forum Guide to Supporting Data Access for Researchers. Let me congratulate and thank the Data Use (Researchers) Working Group members responsible for creating this valuable resource. I think that you will find that it provides a goldmine of helpful and useful guidance for both researchers and state agency staff. In the long run, I hope that this guide assists in developing and sustaining more productive partnerships between researchers and policy makers and practitioners.

Due in large part to my own experience in working with a large local education agency, I am a strong believer in the power and utility of researcher policy maker/practitioner partnerships. For them to work best, however, the partners need to be equals and have equal voices. Researchers bring strong technical and theoretical skills to the table; policy makers and practitioners bring their experience and expertise, and above all, their questions and problems that are grounded in the real challenges that they face every day. I urge that those of you in state and local agencies make sure that your voices are heard in planning and designing research that you find helpful and relevant.

Sincerely,

John Q. Easton

Working Group Members

This guide was developed through the National Cooperative Education Statistics System and funded by the National Center for Education Statistics (NCES) of the U.S. Department of Education. A volunteer working group of the National Forum on Education Statistics produced this document.

Chair

Kathy Gosa

Kansas State Department of Education

Members

Bruce Dacey

Delaware Department of Education

Tom Howell

Michigan Center for Educational Performance and Information

Tom Ogle

Missouri Department of Elementary and Secondary Education

Jay Pennington

Iowa Department of Education

Brian Snow

Maine State Department of Education

Pat Sullivan

Texas Education Agency

Peter Tamayo

State of Washington Office of Superintendent of Public Instruction

Levette Williams

Georgia Department of Education

Consultants

Tom Szuba

Quality Information Partners

Jay Pfeiffer

MPR Associates

Project Officers

Ghedam Bairu

National Center for Education Statistics (NCES)

Ruth Neild

National Center for Education Evaluation and Regional Assistance (NCEE)

Foreword

The National Cooperative Education Statistics System

The work of the Forum is a key aspect of the National Cooperative Education Statistics System. The Cooperative System was established to produce and maintain, with the cooperation of the states, comparable and uniform education information and data that are useful for policymaking at the federal, state, and local levels. To assist in meeting this goal, the National Center for Education Statistics (NCES), within the U.S. Department of Education, established the Forum to improve the collection, reporting, and use of elementary and secondary education statistics. The Forum deals with issues in education data policy, sponsors innovations in data collection and reporting, and provides technical assistance to improve state and local data systems.

Development of Forum Products

Members of the Forum establish task forces to develop best practice guides in data-related areas of interest to federal, state, and local education agencies. They are assisted in this work by NCES, but the content comes from the collective experience of the state and school district task force members who review all products iteratively throughout the development process. Documents prepared, reviewed, and approved by task force members undergo a formal public review. This public review consists of focus groups comprised of representatives of the product’s intended audience, review sessions at relevant regional or national conferences, or technical reviews by acknowledged experts in the field. In addition, all draft documents are posted on the Forum website prior to publication so that any interested individuals or organizations can provide feedback. After the task force oversees the integration of public review comments and reviews the document a final time, publications are subject to examination by members of the Forum standing committee that is sponsoring the project. Finally, the entire Forum (approximately 120 members) reviews and formally votes to approve all documents prior to publication. NCES provides final review and approval prior to publication.

Relationship of this Guide to the Family Educational Rights and Privacy Act

While the Family Educational Rights and Privacy Act (FERPA) does not have a research exception, per se, FERPA does allow state education agencies (SEAs) to share data under certain circumstances. SEAs wishing to share education data with members of the researcher community will have to review research proposals one-by-one to evaluate whether the sharing is permissible under the audit and evaluation exception or the studies exception described in FERPA. Data sharing for research is permissible under the studies exception if the research is for, or on behalf of, a school, school district, or postsecondary institution and is for the purpose of developing, validating, or administering predictive tests; administering student aid programs; or improving instruction. The audit or evaluation exception permits data sharing to evaluate federal- or state-supported education programs. Education programs must be “principally engaged in the provision of education.” This includes early childhood programs, elementary and secondary education, postsecondary education, special education, job training, career and technical education, and adult education. Any program that is administered by an education agency or institution falls within this exception. Any entity disclosing personally identifiable information (PII) from education records is specifically required to use reasonable methods to ensure to the greatest extent practicable that data are shared in compliance with FERPA and its regulations. The information in this guide relating to protecting confidential data and ensuring data security, including the specific terms as well as the content of data sharing agreements, is offered in support of the use of such reasonable methods. Information about FERPA is available through the U.S. Department of Education’s Privacy Technical Assistance Center

http://www2.ed.gov/policy/gen/guid/ptac/index.html. See especially

http://www2.ed.gov/policy/gen/guid/fpco/pdf/reasonablemtd_agreement.pdf.

Chapter 1

Data Partnerships: An Opportunity to Benefit Education Agencies and the Research Community

Introduction: Data Partnerships Improve Education and Research

Data are an integral component of our education system. As such, most state education agencies (SEAs) view responding to requests for data as a major responsibility to their stakeholders. People use data to assess student achievement, allocate resources, and evaluate the effectiveness of instruction, curricula, schools, and staff. Much of this analysis is conducted by education researchers, who often have advanced training in research and evaluation, statistics, and related methodologies. Sometimes other stakeholders, such as community members, advocacy organizations, and public interest groups, also engage in education research.

Most SEAs have considerable experience with data sharing and fully appreciate that collaborating with researchers is a wise investment in education. Properly conducted research is based on both sound methods and quality data—and can provide many benefits to education agencies, including, for example, new information on the status of schools and students, improvements to research and evaluation methods, and technical enhancements to datasets. SEAs that collaborate with researchers can expect to not only improve the body of academic literature on education, but also inform policy, advance pedagogy, and positively impact the education of individual students.

Partnerships with researchers can lead to numerous other tangible benefits to education agencies, such as

- encouraging research projects that reflect an education agency’s information needs and priorities;

- supporting data-driven decisionmaking by educators and policymakers, including instructional and management choices that directly affect the quality of teaching and learning;

- providing access to experts who can design programs that permit more robust analytical studies (e.g., with pre- and post-tests, pilots, and control and treatment groups);and

- supplementing an agency’s research capacity and/or building the research skills of staff who will work alongside members of the research community while reviewing and servicing data requests.

The significance of SEA collaboration with the research community has increased dramatically with the advent of statewide longitudinal data systems (SLDS) currently under development in many education agencies.[1] These systems improve the ability of states to efficiently and accurately manage, analyze, share, and use education data, which, in turn, fuels research focused on closing achievement gaps and improving achievement throughout a student's entire education experience. This expansion of education data systems serves as a foundation for research and evidence-based action in an education system that is becoming more broadly defined to include early childhood, K-12, and postsecondary institutions, as well as workforce preparation and performance.

Research based on quality data can provide valuable insight into education policies and practices. Collaboration between SEAs and the research community is driven by an overarching goal of using the knowledge gained from research to improve student achievement.

Foundations for Data Sharing

Responding to data requests is a substantial undertaking that must be managed effectively given its demand on SEA resources and far-reaching implications on the quality of education research. After all, the accuracy and validity of research results are determined not only by the characteristics of the project’s design and methods, but also by the quality and timeliness of the data provided by the education agency.When responding to data requests, SEA staff members need to understand numerous dimensions of a research proposal, including which data are being requested and for what purpose; which statutory protections apply to the data requested; whether the requested data are appropriate to address the research question(s); which data are actually available from the agency; which data can be shared; and which data need to be masked, de-identified, or otherwise altered to protect individual privacy. Establishing this critical information for evaluating a data request will not occur through haphazard communications with applicants but, instead, should be a component of an organization’s formal system for managing how data sharing occurs.

Data requests are numerous and often repetitive in nature, such as:

- multiple requests from a single researcher at one time;

- similar requests from multiple researchers over time; or

- identical requests repeated at regular intervals (e.g., quarterly, annually, etc.).

The development of a robust data sharing infrastructure is an investment in more efficient responses to data requests over time.

A fundamental step for SEAs wishing to maximize relationships with researchers is the development of a robust data sharing infrastructure. The establishment of a data sharing infrastructure means that there is a common understanding of how data sharing occurs within the mission of the organization; how sharing decisions are made and by whom; and how data sharing should be implemented at an organizational level rather than being driven by a few individuals.

Properly conducted data sharing is appropriate. As such, policies governing sharing data with researchers should be transparent to all stakeholders in the education community.

Data Governance: Data sharing occurs within the broader context of data governance and management practices in an education agency.[2] It includes establishing responsibility for individual data elements, datasets, and databases—and continuously improving data systems through the institutionalized development and enforcement of policies, roles, responsibilities, and procedures. Data governance identifies master data sources (i.e., authoritative data sources) and defines responsibilities for accessing and maintaining these data in order to safeguard the quality, integrity, privacy, and security of data.When establishing a data sharing infrastructure, data governance must be expanded to ensure that the standards of the agency are upheld throughout the data sharing process.

Data sharing is not an individual responsibility. It occurs within the broader context of data governance policies that apply to the entire education agency.

Challenges to Sharing Data

While data sharing can provide insight into many aspects of education, supplying data to the research community comes at a cost to the SEA. For example, responding to the growing volume of data requests can become a full-time job for one or more staff members in an education agency. There are also very practical concerns about sharing education data (e.g., privacy issues, data ownership realities, and the potential for misusing or misinterpreting data), especially when agency staff are dealing with researchers who are not familiar with a state’s education data system and data governance policies.

Resource Allocation: Providing data to researchers can be a substantial undertaking that involves considerable resources. Staff time required to establish an infrastructure, implement core data sharing practices, and manage and monitor requests can create a significant resource burden for an SEA. Staffing constraints in many state agencies can further intensify this burden. Moreover, out-of-pocket expenses can be incurred when conducting researcher training, reviewing data requests (especially requests requiring legal review), and engaging in ongoing oversight of the research. SEAs must carefully consider the resources that can be allocated to a data sharing program and then establish policies that focus those resources on high-priority issues and ensure that the program does not exceed the resources allotted to this core function.

Sound policies and processes governing access to education data can help to minimize the costs and risks commonly associated with data sharing.

Data Limitations: Much of the data in an SLDS will be statutorily protected by FERPA or another statute. It is important for SEAs to consider any use and sharing limitations associated with the various types of data they maintain. Additionally, data collected by SEAs are intended for specific purposes (often to reflect statutory requirements) and may not necessarily meet the precise needs of research projects.When reviewing data requests, SEAs should look for discrepancies between the data that are actually available and the data the researcher needs to answer the research question(s). Similarly, it is important to make sure that researchers are aware of any known limitations of a dataset. Typical considerations include collection dates, definitions, code sets, and business rules. For example, federal race and ethnicity reporting requirements have changed over time. As such, SEA staff should inform researchers of the significance of this change in collection practices on the comparability of the data over time (i.e., before and after the change was implemented). In broader terms, data quality is often unproven the first time new collections are administered, especially when respondents are not familiar with new request protocols. Providing access to metadata [3] about how to use agency data can help to ensure that researchers are aware of these types of limitations to data use.

SEAs vary tremendously in their needs, circumstances, structure, and settings—yet core practices apply to the vast majority of education organizations. When organizations adapt or adopt these types of recommended core practices, they

- invest in efficiencies that can lower operational costs and improve operations;

- confirm that existing policies meet expected standards and regulations;

- contribute to a body of best practices that advance the field of education data;

- support data-driven policy decisions and evidence-based actions in education agencies; and

- contribute to the development of data standards, which has advantages for both education agencies and researchers.

Document Purpose and Audience

This document recommends a set of “core” practices, operations, and templates that can be adopted and adapted by SEAs as they consider how to respond to requests for data about the education enterprise, including data maintained in longitudinal data systems.These recommendations reflect core practices and principles for managing the flow of data requests, establishing response priorities, monitoring appropriate use, protecting privacy, and ensuring that research efforts are beneficial to the education agency as well as the research community. It should be noted that a corollary, but significant, benefit to “best practice” data sharing is improved relationships between SEAs and the colleges, universities, and policy organizations that frequently employ education researchers.

The primary audiences for this document are data policymakers and managers in SEAs who are generally responsible for managing and responding to requests for data. An additional important audience for this resource is the research community that submits data requests to SEAs. Data requests come from diverse sources, ranging from members of the research community to advocacy organizations, the media, the public, and other parties interested in education data. This document is intended to help SEAs determine whether and how to voluntarily fulfill requests from researchers for access to education data, including confidential or otherwise restricted data. It does not focus on recommendations for dealing with requests for publicly available data, Freedom of Information Act requests, or requests from legislators or other stakeholders who may be in a position to require a response from the SEA. Although this document focuses on SEAs, other stakeholders, including staff in local education agencies (LEAs), may wish to consider adapting components of these core practices to suit their needs.

Improving Data Access for Researchers

Researchers who know where data are located, what data are available, and where and how data can be accessed will be better prepared to submit thorough and accurate data requests. As such, SEAs often find it useful to help researchers locate and access data that have already been made available to the public. Methods of assisting researchers on this front can range from simple to complex, and might include creating a catalog of data sources, developing web gateways for researchers, and developing education research data centers (ERDCs) for a researcher audience.

Catalogs of Data Sources. A catalog of data sources generally describes the types of data available to researchers, the location of these data, and contact information for assistance. When developing a catalog of data sources, it is important to consider the types of data that will interest researchers, and encourage researchers to align their projects with the research priorities of the SEA.

Gateways. A gateway is a website that serves as an entrance to another website. For example, if someone were to search the internet for “access to education data, state X,” they could be channeled through a gateway to a site that provides information about that SEA’s data priorities and processes for accessing data. Such a site may be designed to separate requests for access to restricted information from requests for publicly available data. Once established, a gateway can become an agency’s primary data portal, with links to secondary sources for data, such as adult education, career and technical education, higher education, workforce, and employment data.

When building a gateway, it is important to consider the questions researchers might have regarding access to data. Useful components to address researcher questions include

- a summary of the agency’s research priorities;

- an overview of the agency’s available data holdings;

- basic components of the data request process;

- application guidelines and deadlines;

- staff contacts;

- stipulations regarding how and where data can be accessed or used (e.g., on- or off-site);

- expected timelines for responding to data requests;

- cost/reimbursement policies; and

- security, confidentiality, and privacy protection requirements.

Researchers are also likely to have questions about guidelines for submitting a research proposal. As such, gateways often include general information addressing

- proposal formats (e.g., page length, margins);

- preliminary review processes;

- research and statistical integrity standards;

- technical merit expectations;

- security requirements; and

- interest in furthering the mission of the SEA.

Another purpose of a gateway is to provide a means for researchers to determine whether or not they are eligible to use data. In some agencies, for example, a researcher must be a U.S. citizen, be currently enrolled as a student in an institution of higher education, or be employed by an institution of higher education. An agency may wish to include additional requirements, such as those related to training and experience. Some agencies also publish lists of previously approved researchers and eligible organizations, such as non-profits, companies with state or federal contracts, and state or local governments.

Chapter 2

Core Practices for Effectively Managing Data Requests

The core practices described below can contribute to a robust data sharing infrastructure that includes policies and procedures for all aspects of the data sharing process—from assisting researchers prior to the submission of a data request through the completion of the research, the destruction of data, and the review and use of the research results by the SEA. These recommended core practices should be customized to best meet the data sharing, management, and security requirements of an education agency.

Data Request Core Practices

The following steps contribute to a comprehensive workflow system for handling data requests—and are each addressed as sections of this chapter:

1. Help Researchers Understand Agency Data and the Data Request Process

2. Create Effective Data Request Forms for Researchers

3. Review Data Requests Strategically

4. Manage the Data Request Process Efficiently

5. Release Data Appropriately

6. Monitor Data Use

Core Practice 1: Help Researchers Understand Agency Data and the Data Request Process

SEAs should expect that their efforts to share data with researchers will result in the production of useful and valid information about education policies, processes, and practices. Unfortunately, this expectation may be unrealistic if researchers do not understand the data—e.g., practices affecting its collection, guidelines for appropriate access and use, and ways to accommodate its limitations. By helping researchers better understand data available from the agency and the circumstances in which requests may be approved, SEAs greatly improve the likelihood that data will be used and interpreted appropriately within the context of a research plan.

An SEA’s capacity to train researchers to use data responsibly will vary based on its policies, funding, staff turnover, and the number and nature of research requests.

One effective way of improving a researcher’s data knowledge is to provide training materials about an agency’s use of data terms, definitions, and coding instructions. In order to more fully understand the appropriate uses and limitations of particular data elements and datasets, training materials should also include relevant metadata. Training that helps researchers gain the skills and knowledge to efficiently request and access data can improve the research process and decrease the amount of SEA staff resources needed to provide data sharing services. Because of the technical and data expertise required to properly access, use, and manage data, prospective researchers should meet any and all training expectations set forth by the education agency—not as a courtesy, but as a requirement prior to receiving access.

Helping researchers obtain an in-depth understanding of data is especially important when an SEA deals with requests for data that originate across the spectrum of the education community, including early childhood education, K-12, postsecondary, and labor force. Many data experts in these communities use terms that may, in theory, appear to be similar, but in practice are slightly misaligned or even widely divergent with common use in a K-12 setting. For example, data elements about “discipline” used by universities often refer to academic fields of study (“my discipline is economics”), whereas “discipline” in elementary and secondary schools usually refers to approaches to modify behavior following an infraction of rules or other codes of behavior. Similarly, every university wants to increase its “retention rate” (a measure of how many students stay enrolled over time), while the K-12 community works diligently to help students make sufficient academic progress so that their “retention rate” (the percentage of students held back to repeat a grade level) decreases.

Researchers who have previously requested data from an education agency, or who have worked with other education agencies, may believe that they are already familiar with routine requirements and expectations for data use. However, because procedures vary between states and agencies, processes and policies change over time, and expectations differ based on the type of data requested, both experienced and inexperienced researchers should be expected to demonstrate their understanding of the specific processes and standards required by the agency. Providing a detailed description of all expectations will help to ensure that researchers are aware of their responsibilities for complying with policies for protecting, managing, and using data.

When developing a program to prepare researchers to use agency data, planners may wish to consider such issues as whether training will be optional or required, who pays for training costs, at what time during the request/access/use process the training would be most effective, how accomplishing the training will be evaluated (e.g., will there be an exam or other assessment with a required passing score), and whether it will be offered in- person or online.

Training topics frequently include

- SEA research priorities. By making the SEA’s goals and priorities explicit, researchers can be encouraged to align their interests with the needs of the agency. Such alignment can promote collaboration between the researcher and the SEA, and result in research that is of use to both parties. Prospective researchers should be informed if SEA reviewers expect to assign priority to research plans that align with the SEA’s research agenda.

- Data request and evaluation overview. One of the benefits of formalizing data sharing practices is that information can be standardized and shared routinely with prospective researchers, thereby increasing the transparency of decisions made by the SEA and helping assure researchers that each data request will be considered in a fair and timely manner. Once aware of the request and evaluation process, including preliminary review steps, researchers will be able to improve the planning and submission of their requests. For example, some SEAs combine multiple requests from a particular researcher into a single large request, which may require alternative data masking strategies. Alerting researchers to such a practice will help them plan their data requests more thoughtfully.

- Data governance and privacy policies.[4] A summary of relevant aspects of the agency’s data governance program, including privacy and confidentiality safeguards, will help researchers submit more appropriate data requests. For example, a data request that is informed by an accurate understanding of privacy laws such as the Family Educational Rights and Privacy Act (FERPA) will have a much better chance of being successful than a poorly planned request for personally identifiable information about students. By providing this information to researchers, SEAs can reasonably expect that requests for data will meet baseline requirements required for review. Researchers, meanwhile, will better understand the differences between personally identifiable information and public data, and can tailor their requests accordingly. They will also be in a position to build appropriate privacy and confidentiality safeguards into their research design, including statistical standards, suppression rules, and other techniques for preventing the inadvertent disclosure of private information.Researchers should also understand that, as a component of an SEA’s privacy and confidentiality responsibilities, only data elements directly related to an approved research question can be released.While researchers may wish to have access to a multitude of indirectly related elements so that they can explore methodological options and alternative analyses, providing data for these purposes often violates privacy laws and policies. Thus, training should emphasize the importance of carefully defining research topics and requesting data accordingly.

- Data sources.A thorough description of an agency’s available data sources can result in more effective and efficient requests. Such a summary will often include the name of various collection instruments, as well as a list of what data are collected, available, reported, and used in the agency. Many states release annual reports and other public data files, including graduation rates, truancy rates, percentages of students receiving free or reduced price meals, and other data relevant to education researchers. Similarly, annual progress measures are often available at the school level and may be disaggregated by gender, race, grade level, and other common indicators. Ensuring that researchers are aware of these public data sources and providing them with tools to access these resources can relieve the burden on SEA staff with respect to processing requests for data that are already available to the public.

- Requests for too much information can actually be detrimental to research outcomes. For example, the more variables included in a cross-tabulated dataset, the more likely it will be necessary to suppress small cell values to protect personally identifiable information. Thus, researchers are likely to get “better” outcomes by limiting their data requests to what they really need.

- Metadata. Metadata, or “data about data,” are a key tool for explaining education data to external researchers. Metadata include definitions of data elements, coding options, and file layouts, as well as other characteristics of the data, such as usage guidance and business rules for accurate collection and reporting. Metadata are invaluable to stakeholders who need to use and apply data. For example, if a data element’s definition was changed in 2008, data users would benefit from knowing the nature of the change before attempting to compare data from 2005–2010. By accessing the metadata in an agency’s data dictionary prior to drafting a research proposal or data request, researchers can become familiar with the definitions used by the agency.[5]

- Data management expectations. Researchers should be informed of the ways in which an education agency can deliver data, including acceptable formats and media for transmission. Researchers should also be aware of an SEA’s requirements for appropriate data management, including confidentiality and security expectations, disclosure limitations, output rules (such as statistical methods for masking data), and obligations for data destruction at the conclusion of the allotted research time. If the agency monitors or audits research, training may also address these processes and expectations, including circumstances under which penalties and sanctions can be levied against researchers who fail to appropriately manage, protect, or destroy data. Disciplinary consequences for researchers who misuse data should be specified.

- Ethical and legal responsibilities.Well-designed research and data use adheres to a strict set of ethical considerations. Researchers should understand that the agency expects all research to follow relevant ethical standards related to, for example, analyzing data and deriving results, as well as the publication and communication of findings. In addition to encouraging high ethical standards in research, an SEA should emphasize that researchers must comply with all laws governing the collection and use of education data.

- Communications responsibilities. SEAs frequently require researchers to contact the agency if (or when) certain scenarios arise throughout the course of a research project. For example, most agencies expect researchers to share project findings with the agency prior to publication or to notify the agency of any modifications to agreed-upon activities (e.g., a change in timeline, an interest in providing data access to another member of the research team, or alterations to approved methodologies and plans). Similarly, breaches of data security often trigger communications responsibilities. Such requirements must be made clear and training should include not only the topics that warrant communication, but also the mechanisms (e.g., email, certified mail, verbal), contact person, and timelines for such communication (e.g., within 48 hours of an incident).

Simulated Datasets for Training Purposes

In an effort to provide hands-on data use training, some SEAs have constructed simulated datasets to help prospective researchers learn about the agency’s data and technical environment. Simulated data approximate real data without reproducing actual data. Allowing researchers to access simulated data can help them become familiar with data elements, practice managing data files, identify variables that can inform their research, and determine whether research questions can be answered with available data.

By helping researchers better understand agency data, an SEA makes an investment in sound and useful research. Researchers who understand agency metadata, who are aware of what data are available, and who are comfortable with procedures for appropriately accessing and using data are better prepared to conduct quality research that benefits the SEA and the broader education system.

Help Researchers Understand Agency Data and the Data Request Process: Action Items

- Develop policies about training topics and requirements for researchers who wish to use agency data. Identify those topics that are optional and those that are mandatory.

- Identify or develop resources (e.g., training materials) to help researchers better understand, request, and use agency data.

- Create a program to orient researchers to agency data, and determine whether it will be offered in person, online, or through print resources.

- Determine when training would be most useful to researchers during the data request/access/use timeline

Why Should an SEA Help Researchers Understand Agency Data?

- Researchers may be in a position to help the SEA advance its own research agenda.

- Not all researchers use the same terms, standards, and formats as the agency unless they have been trained to do so—meaning that untrained researchers are more likely to misuse data.

- Some researchers may not understand the agency’s application of FERPA, contractual obligations, and/or other privacy and security safeguards unless trained to do so.

- Few researchers will volunteer for training unless it is required for data access.

Core Practice 2: Create Effective Data Request Forms for Researchers

Creating standardized forms for researchers to use when submitting data requests will streamline both the request and evaluation process.When designed and implemented wisely, data request forms can help researchers accurately identify the data that they are requesting and present the request in a format that concisely, yet comprehensively, describes their vision for a research plan. The key to streamlining the data request process is to create forms that accurately capture the information needed by SEA staff to evaluate a request.

Data Request Forms

Depending upon the quantity of data requests received, an SEA may wish to develop two types of data request forms: a Preliminary Research/Data Request form (see appendix B for template) and a Full Research/Data Request form (see appendix C for template).

A Preliminary Research/Data Request form provides a basic overview of the proposed research, allowing the researcher to introduce the SEA to a research plan without a significant investment in detail, yet ensuring that reviewers have enough information to assess whether the SEA wishes to consider providing data for the proposed project. The agency can respond to the preliminary data request by refusing the request, suggesting changes that will improve the request, or inviting the submission of a Full Research/Data Request form.When designing the Full Research/Data Request form, an SEA should confirm that a typical response will provide enough information for the SEA to evaluate the following:

- Significance of the research proposal: Does the research address an important topic?

- Benefit to the SEA (alignment with SEA research priorities: Does the research align with SEA priorities and goals? Will the research resulting from the data request inform the practices of the education agency, improve education, or serve some other function that is important to the agency?

- Validity of the research plan: Is the researcher qualified to conduct the study? Are appropriate methods proposed for answering the research question? Will the plan yield valid results?

- Data needs: Has the researcher identified the data needed to conduct the research? Do all data requested directly support the research proposal? Does the agency have the requested data? Is access permissible with respect to legal, policy, and ethical considerations?

- Burden to SEA:Will it be necessary for the agency to customize the data, provide specialized training, or offer ongoing support for the research? Does the agency have the resources to support the proposed data request, and will the benefit of the research justify the expenditure of resources? What resources will be needed for ongoing monitoring and periodic research audits?

- Timing: Can the agency meet any time constraints related to the data request?

- Adherence to Policies: Has the researcher requested and received data previously? If so, did the researcher abide by the SEA’s data sharing policies and agreements?

- Compliance with FERPA: Is the research proposal compliant with FERPA?

The Full Research/Data Request form should build upon the Preliminary Research/Data Request form and require enough additional information to fully describe the proposed research project/data request. As such, it becomes the basis for next steps in the data sharing process, including the completion of the following standard forms and agreements:

- Data Sharing Agreement: The Data Sharing Agreement builds upon the information included in the Full Research/Data Request form. It should describe the purpose of the data release, justification for the research, plans for conducting the research, and all of the terms and conditions governing the subsequent release of data. Moreover, a Data Sharing Agreement should include a list of data elements and the collection years that will be shared, as well as confirmation that the research is in compliance with FERPA. A sample Data Sharing Agreement template is included in appendix D.

- Data Sharing Agreement: The Data Sharing Agreement builds upon the information included in the Full Research/Data Request form. It should describe the purpose of the data release, justification for the research, plans for conducting the research, and all of the terms and conditions governing the subsequent release of data. Moreover, a Data Sharing Agreement should include a list of data elements and the collection years that will be shared, as well as confirmation that the research is in compliance with FERPA. A sample Data Sharing Agreement template is included in appendix D.

- Agreement Modification:Significant changes to the research will necessitate a modification to the original Data Sharing Agreement. An Agreement Modification form allows researchers to submit their proposed changes to a research or data use plan for SEA review and consideration. A sample Agreement Modification template is included in appendix E.

- Personal Access Agreement: A Personal Access Agreement establishes the responsibility of the researcher to maintain the confidentiality of any data to which access is granted. A sample template for a Personal Access Agreement is included in appendix F.

- Certification of Data Destruction: Submission of this form confirms that the researcher has destroyed all restricted data according to the requirements of the SEA and as specified in the Data Sharing Agreement. A sample Certification of Data Destruction template is included in appendix G.

The templates for each of these forms in the appendices are based on current practices used by SEAs and national organizations. Each form is designed to build upon the information required in the previous form—for example, the Data Sharing Agreement encompasses aspects of the Full Research Request form, and the Agreement Modification is designed to build upon the information included in the Data Sharing Agreement.

While these templates are designed to be widely applicable in education agencies, they may be applied differently in different organizations depending on the nature of proposed research projects. For example, it may not be sufficient to have forms signed by only the principal investigator or lead researcher if a research project will employ a team of staff members that may have access to requested data. Under such a scenario, Personal Access Agreements and Data Sharing Agreements should be signed by all members of a research team, and the education agency could require that any new hires should also be trained on privacy laws and standards before signing the forms. In such an example, many SEAs may wish to require an agreement with the researchers’ organization (e.g., a university) to help ensure responsible oversight at an institutional level.

Create Effective Data Request Forms for Researchers: Action Items

- Identify the information needed by the SEA to evaluate data requests and research proposals, and then design forms that are likely to generate this information in an efficient manner. Commonly used forms include

- Preliminary Research/Data Request form (see appendix B for template);

- Full Research/Data Request form (see appendix C for template);

- Data Sharing Agreement form (see appendix D for template);

- Agreement Modification Request form (see appendix E for template);

- Personal Access Agreement form (see appendix F for template); and

- Data Destruction Certification form (see appendix G for template).

- Consider the use of a Preliminary Research/Data Request form to minimize the burden on prospective researchers while still providing adequate information for an initial review of the request.

- Design and implement a Full Research/Data Request form that requires all of the information needed for the request to be evaluated (and allocates adequate space for the researcher to provide all of the necessary information).

Why Should an SEA Devote Staff Time to Developing Data Request Forms for Researchers?

- Some researchers may not provide all of the information needed to evaluate their data requests unless the education agency clearly communicates its evaluation requirements. The more effective the data request form is, the less burden there is on SEA staff during the evaluation of the request and the less likelihood of unnecessary delays in the review process.

- Information collected in request forms is not only useful during the evaluation process, but also serves as the foundation for subsequent steps in the data release process (e.g., as the basis for data sharing agreements).

Core Practice 3: Review Data Requests Strategically

Receiving and reviewing requests can be an exciting aspect of engaging researchers, but it can also be a significant burden on an SEA. Core practices one and two, discussed above, can help ease the burden by streamlining the process.When reviewing data requests, SEAs not only have an opportunity to assess whether the data they have available for researchers will benefit the research plan, but also how the proposed research can be harnessed to improve the agency’s broader policies and operations (and even the education enterprise in general).

The policies and procedures for reviewing data requests discussed in this document refer to formal requests for data to be used in research. As such, the recommendations may or may not apply to Freedom of Information Act (FOIA) requests or state open records requests.

Strategies for Receiving Data Requests

In the recent past, data requests in many SEAs were serviced by the program office that fielded the inquiry. In other words, a request for special education data might be received, reviewed, and, if approved, fulfilled by the special education department without the involvement of any other staff in the agency.With the implementation of more formal data governance structures in recent years, data requests are increasingly channeled to a centralized point of contact so that all requests are evaluated in light of standard procedures (i.e., data governance and data sharing policies) and fulfilled using an authoritative data source (i.e., master datasets rather than copies of the data in operational settings that may be unapproved or time variant). In other SEAs, however, there is concern that centralizing the point of contact presents an unnecessary obstacle to researchers. To address this, some agencies have implemented what is commonly referred to as a “no wrong door” policy, whereby researchers can request data from any branch or office in an education agency (although they may still be routed internally to the centralized point of contact).

Assigning Review Responsibilities

Several methods of reviewing data requests are commonly used by education agencies, including staff review, data steward review, review boards, and legal counsel review. Agencies often incorporate one standard review method or, alternatively, integrate multiple methods depending on the nature and volume of requests.

Staff Review:A staff review involves one or more education agency staff members who are familiar with the organization’s research priorities and policies on data sharing—and who have been authorized to approve, deny, or amend a request.While a staff review is an efficient method of handling large numbers of data requests, it is important to ensure the staff review includes individuals who are qualified to make decisions that involve legal or ethical questions regarding data use and disclosure, or that they seek the input of data stewards and legal counsel when questions arise regarding whether the agency is permitted to fulfill the request.

Depending on the nature of the agency and the data requests, any combination of the following approaches to reviewing data requests may be appropriate:

- staff review;

- data steward review;

- review boards; and

- legal counsel review.

Data Steward Review.Another type of staff review involves data stewards—the organization’s data experts.Within a data governance structure, data stewards are those individuals who are responsible for data quality. Because of their in-depth knowledge of data, including metadata, data stewards are well qualified to determine whether the agency has the appropriate data to fulfill specific data requests. Data stewards often have expertise concerning data in a particular program area, and since many requests cross program areas, it is important to ensure that review involves all relevant data stewards.While experts with respect to data, data stewards are not always empowered to determine the legal or ethical merit of research proposals.

IRBs are an integral part of the research approval process at the university level. Education agencies considering data requests from university-affiliated researchers may require that a proposed research plan receive approval from a university’s IRB prior to consideration by the agency.

Review Boards:Review boards, including data governance committees and institutional review boards (IRBs), are usually comprised of high-level administrators and staff who are trained to evaluate requests based upon a strict set of ethical guidelines (especially when human subjects are involved in a research plan).While K-12 education agencies have not traditionally relied on IRBs to the degree that they are used in research universities, they or similar institutional oversight committees can be established in any organization.

Legal Counsel Review: Legal counsel review is necessary when there are questions regarding the legality of sharing data—e.g., whether a request for data complies with all applicable laws and whether releasing the requested data is required or prohibited by law, as well as the impact of contractual obligations or memoranda of understanding (MOUs) on the release of data. Legal review can be conducted as an independent form of review or in conjunction with other reviews, and may be a required step in some agencies.

Depending on the agency’s evaluation protocols, some data requests may require that the evaluation and/or approval be escalated within the organization. If, for example, a data steward is unable to determine the merit of a proposal, the escalation process might warrant that the decision be elevated to the superintendent’s office. Similarly, if the superintendent’s office has concerns about privacy restrictions related to a request, review might be escalated to legal counsel. Escalation may also become relevant when a request is denied or an applicant appeals a decision related to servicing a request.

Conducting the Review

Before reviewing a data request, it is often useful to confirm that the researcher has already checked public sources for the needed data. In order to more fully understand a researcher’s needs, some SEAs interview data requesters in person or by phone as a standard step in the process of determining what data are needed and how they propose to use the requested data. This level of personal communication between the SEA and the researcher can help to clarify questions that arise about the research plan, as well as any questions the researcher may have about the SEA’s review and evaluation process.

When beginning the review, several straightforward questions should be considered, including:

- Does the SEA have the requested date?

- Can the SEA legally provide the requested data to the researcher (e.g., does the request comply with FERPA)?

- Has the researcher successfully completed all required training?

- If this researcher has previously been granted access to data, did he or she adhere to all agency requirements? Were data managed and used appropriately? Has the destruction of previously accessed datasets been certified?

- If the researcher is affiliated with a research organization, such as a university, does the researcher have approval for the project from the organization’s IRB?

- Will any fees be required? If so, have they been paid?

- Does the SEA have the available resources to assemble the data within the timeframe needed by the researcher?

If the answers to these background questions are acceptable, a more robust evaluation of the specific data request may be appropriate. The completed Full Research /Data Request form discussed in Core Practice 2 (and presented in appendix C) should help to assess the following types of questions:

- Does the research have general merit?

- Does the research align with the SEA’s priorities and goals?

- Have data needs been clearly and accurately described?

- Are the requested data appropriate for the stated research question(s)?

- Is the research request limited to the data required for the specified research?

- Is there a more appropriate source for the data?

- What is the impact of data sharing on confidentiality and security assurances?

- Does the topic require approval from other entities in the agency (e.g., the state superintendent or agency legal counsel) or from other agencies (e.g., early childhood or postsecondary institutions)?

Working with Outside Agencies

By linking education data with data from outside agencies, researchers can answer questions about education that go far beyond the classroom. Linkages between K-12, early learning, and social services, for example, can provide insight into the factors beyond the school environment that influence education. Similarly, linkages between K-12 schools, postsecondary institutions, and employers can provide considerable information about post-school outcomes. Thus, requests for data from multiple agencies can inform important and useful research; however, such requests must be carefully managed, especially with respect to protecting the confidentiality of personally identifiable information and adhering to privacy laws such as FERPA and all other applicable laws related to the permissible uses of any of the data to be used. This issue has become even more critical as states establish P-20W data systems, linking data from early childhood programs, through elementary and secondary school, and into postsecondary education and the workforce.[6]

All parties engaging in an MOU are bound to the agreement. For example, an MOU in which an SEA agrees not to share data submitted by a school district means exactly that—the SEA cannot disclose those data without explicit permission from its district partner in the MOU.

When considering requests that involve data from outside agencies, the outside agency should be involved in the review process, and approval from each organization should be confirmed before data are released. Many SEAs have found that the process of collaborating with other agencies is facilitated by the use of MOUs—and written agreements are specifically required when sharing FERPA-protected data. MOUs typically specify the rights and interests of each agency, the expectations for working together, and the responsibilities of each organization. When data from an outside agency are frequently included in requests submitted to SEAs, the establishment of an MOU can standardize methods for ongoing interagency collaboration.[7]

Review Data Requests Strategically: Action Items

- Determine a process for channeling requests to the appropriate office or offices in the agency.

- Assign responsibility for each incoming request, including who is responsible for logging requests, confirming their receipt, and monitoring their progress through the process.

- Standardize a review method or methods, including establishing clear rules for escalating requests throughout the review process.

- Determine the participants and frequency of review team meetings and, if appropriate, publish a schedule of meetings.

- Establish a method for handling requests that cannot wait for a review team meeting.

- Develop guidelines for handling requests that involve data from other agencies. When appropriate, develop MOUs or other agreements in advance to guide the process.

Why Should an SEA Evaluate Data Requests Strategically?

- Research proposals do not always reflect the priorities of the agency or warrant staff time to complete.

- Research projects are not always properly designed.

- The organization may not have data that are appropriate for the proposed research plan.

- Not all data requests can be ethically or legally fulfilled.

Core Practice 4: Manage the Data Request Process Efficiently

Data requests should be managed in a consistent, transparent, and timely manner. To do so, rules and expectations must be clear and explicit, and the researcher must agree to abide by agency policies and positions.

Establish Expectations for Researchers

Policies designed to govern researchers should accurately reflect the priorities and interests of the SEA, clearly establish expectations for researchers, and effectively describe the process of having a data request evaluated. Related guidance should be readily available to potential researchers, either through an agency website or via training materials that are easily accessible.When establishing policies, SEAs should address who is eligible to conduct research with agency data; timelines for data access; fee structures (if any); and expectations for data confidentiality, security, and destruction.

Eligibility to Request Data: Policies established by the data sharing infrastructure should define who is eligible to request agency data and at what level of detail. Data requests from members of the research community may come from a wide range of sources—for example, from an academic researcher who wishes to publish findings in a professional journal, a national or multi-national research organization interested in education trends, or an advocacy group looking to support an education policy or political goal. As such, the SEA must determine whether specific credentials are required for researchers requesting access to particular types of data. Eligibility may vary quite significantly depending on the nature of the data request. For example, access to personally identifiable data on individual students will likely warrant a different level of scrutiny than a request for aggregate data.

SEAs may wish to develop formal relationships with representatives of universities at an institutional level rather than with individual researchers at institutions. Prior to engaging in data sharing with individual researchers, the agency can require that MOUs be signed by representatives of the university or organization with which the researcher is affiliated. These high-level partnerships can expedite the data sharing process and ensure institutional oversight of the research. For example, an MOU between an SEA and a university could

be drafted to ensure that if the researcher leaves his or her post, the university will be responsible for the security and confidentiality of the data.

Timelines for Data Use: Adequate time must be given for the researcher to analyze the data without allowing unlimited, ongoing data access. As such, SEAs should require that a research request include an estimate of the time required to complete the research. Without policies that limit the duration of data use, agencies may find themselves susceptible to researchers who continue to access data over the course of many years— leading to ongoing risk of disclosure without the benefits associated with the completion of a research project. Alterations to a research project, including the timeline, should only be allowed following the submission and approval of an Agreement Modification form (see appendix E).

Fee Structures: Guidelines should also include information on whether the researcher will be charged for data. Fees for fulfilling data requests vary by state and organization. In some agencies, for example, it is illegal to charge any type of fee for data; in other agencies, flat fees are applied to all requests for data. In some agencies, fees are charged only when certain thresholds are met (e.g., requests that take more than one hour of staff time to service). Other agencies charge fees when asked to format data in a specific way, even when the data are otherwise available on a public website.Whatever the situation, SEAs should be transparent about applicable fee structures.

Many recommendations in this document reflect legal requirements originating in the federal Family Educational Rights and Privacy Act (FERPA). Information about FERPA is available through the U.S. Department of Education’s Privacy Technical Assistance Center: nces.ed.gov/programs/ptac

Expectations for Protecting Confidentiality and Security: Agencies engaging in data sharing are responsible for using “reasonable methods” to ensure0 that FERPA-protected data remain confidential. It is essential that researchers be held accountable for meeting the standards of the education agency with regard to technological, procedural, and statistical methods for protecting the confidentiality of student information. Standards for data suppression, redaction, masking, de-identification, and other forms of privacy protection must be explicitly stated in the agency’s data sharing policies, and clearly communicated to researchers. Researchers should also understand their responsibilities with regard to how the data can be accessed and stored, as well as any prohibitions to data matching and other efforts to re-identify records.[8]

Track Data Requests and Use

Good management practices suggest that an agency track the status and progress of all data requests, from the point at which a request is received; through its review, rejection, or approval; the delivery and receipt of data; the publication of research findings (e.g., articles and reports); and the certification of data destruction. The management of this information is greatly facilitated by the use of an automated tracking system that permits authorized users to search records by researcher name, research topic, data access privileges, organizations and affiliations, publication titles and keywords, and other relevant components of data sharing practices. In addition to tracking each data request, it is good practice to keep a copy of any computer code (e.g., SQL code) and original output used to fulfill the request. Such a tracking system also expedites the logging and tracking of a request, escalation and routing of the request, and reporting and communication with individuals who have a stake in the request.While it is good practice to track all requests, it is especially important to track any requests that involve the release of personally identifiable information, since recordation of such requests is a FERPA requirement for SEAs.

Communicate with Researchers

Throughout the request process, researchers should have access to information about the agency’s review of their data request. Timely communication with the researcher regarding the status of the request is appropriate until the request has been either refused or approved and fulfilled. SEAs may also find it advisable to provide researchers with predicted timelines for the data sharing process. The timeline should reflect reasonable estimates for when a researcher can expect to have a request evaluated and, if approved, receive data. SEAs may even wish to let researchers know when modifications to a data request could help the response timeline. For example, it might be possible for an SEA to provide a researcher with a small dataset in a few weeks, whereas a large dataset might take several months. Similarly, applicants should be aware of any deadlines governing the data request process. For example, a review board that meets quarterly may only accept proposals for review up to one month prior to the meeting, whereas any proposals not received by the cutoff date have to wait until the next quarterly meeting for consideration.

Manage the Data Request Process Efficiently: Action Items

- Establish and clearly communicate eligibility criteria for researchers requesting data.

- Develop a searchable system to track requests from the point at which a request is received, through its review, rejection, or approval; the delivery and receipt of data; the publication of research findings (e.g., articles and reports); and the certification of data destruction.

- Determine how the SEA will communicate with researchers and share formative information about the review status of a data request.

- Advise researchers about ways in which the request process can be streamlined (e.g., requests for smaller datasets may not take as long to service as requests for larger datasets).

Why Should an SEA Manage the Data Request Process Efficiently?

- Clear guidelines from the SEA can help researchers to design projects that are in compliance with SEA policies and that incorporate SEA expectations.

- It may be difficult to track the progress of new requests or to efficiently locate old requests without an organized tracking system.

- SEAs are legally accountable through FERPA regulations for recordation of the release of all personally identifiable data.

Core Practice 5: Release Data Appropriately

Once a request has been reviewed and approved, training expectations have been met, and the researcher has certified adherence to all requirements, the data are nearly ready to be released. Data (and relevant metadata) should be provided in a format and media that have been explained to the researcher. Secure delivery and transmission are essential. As such, when releasing data it is important that an SEA appreciate that unexpected events can occur once data have left the secure environment of the agency. For example, it is possible that a disc could be stolen or that electronic delivery could be intercepted. It is also possible that a user could try to improperly combine datasets or otherwise attempt to match one set of data with other data, with the direct or indirect consequence of disclosing personally identifiable information about individual students. Any datasets that will be merged with the SEA’s data should have been disclosed by the researcher in the Full Research Request form (see appendix C) and Data Sharing Agreement (see appendix D), and failure to notify the SEA of efforts to combine datasets should be considered a breach of these usage agreements. In order to protect data from these types of misuse, a wide array of technical and statistical tools have been devised to help protect the privacy and confidentiality of education data even after release. These include, for example

- Suppression.The process of removing sensitive information from a data report so that the report may be distributed to a broader audience without disclosing the sensitive components.

- Masking: The process of obscuring (masking) specific data elements within a dataset to ensure that sensitive data are replaced with realistic, but not real, data so that sensitive information is not accessible to unauthorized users.

- De-identification: Removing or obscuring enough personally identifiable information so that 1) remaining information does not identify an individual, and 2) there is no reasonable basis to believe that the information can be used to identify an individual (although a re-identification code may be securely maintained to link back to the source data in order to add more information about individual students).

- Anonymization: De-identifying data without a re-identification code so that there is not a way to link to original source records.

Because the use of these statistical and presentation methods can affect the analysis and interpretation of data, it is important that researchers be aware of the modified nature of any suppressed, masked, de-identified, and anonymized data they receive.

The type of media on which data are released is also important to data security. For example, email is considered secure only when data are appropriately encrypted and otherwise protected prior to attachment and delivery. Similarly, the exchange of physical media, such as discs and tapes, requires transport by entities that can effectively guarantee safe and secure delivery to authenticated recipients. Traditional file transfer protocols (FTP) were not designed to be a secure mechanism for the safe movement of data, although secure FTP (SFTP) may be appropriate.

To minimize the security risks related to releasing restricted datasets to researchers, some agencies limit data access to safe, highly monitored locations such as research data centers, secure facilities in business or universities, or similar locations (see appendix A). The use of such sites can reduce the risk of confidentiality and security breaches for both researchers and the SEA. It is important to note, however, that researchers usually are not permitted to remove data from these types of secure sites, which can reduce the utility of the data. Depending on the needs of the SEA and its data characteristics, this type of site-based access can be required for all data access requests, or, as is often the case, may only be required for large or particularly sensitive datasets.

Release Data Appropriately: Action Items

- Create a checklist for review prior to releasing data in order to verify that the researcher has completed all mandated training, paid any required fees, and signed all required agreements.

- Establish procedures for providing the researcher with relevant metadata.

- Determine the format and media for releases, engaging in data protection activities when it is possible that released datasets could be matched to publicly available datasets or other data that are available to the researcher.

- Record all disclosures in compliance with expectations set forth in FERPA.

- Establish a standard for determining when data will be released directly to the researcher and when the researcher will be required to access data at an SEA-approved site, such as a research data center, university, or similar location.

Why Should an SEA be Careful to Release Data Appropriately?

- The SEA must ensure the security of data it has shared with third-party researchers, including anticipating the possibility of stolen media, intercepted transmissions, and unethical data matching activities.

- Researchers expect to receive data released to them in a format that is readily usable for their research purposes.

- Without proper explanation, researchers will not understand why released data may have been suppressed, masked, de-identified, anonymized, or otherwise protected— and these modifications can have significant ramifications on the analysis and interpretation of the data.

Core Practice 6: Monitor Data Use

Because an education agency’s responsibility to ensure proper data use does not end when data are released to a researcher, an SEA should commit to monitoring a researcher’s management and use of the data, especially when personally identifiable data have been shared.While monitoring is necessary to ensure adherence to all agreements, the process of monitoring should be pursued as part of a larger effort to encourage clear and ongoing communication between the SEA and the researcher.

Confirm Adherence to Agreements

Data use should be limited to the purposes stated in the Data Sharing Agreement (see appendix D), and should not be used for other research without explicit approval. Monitoring research enables the SEA to confirm that actual data use conforms to proposed use as approved by the agency. However, even well-prepared researchers find that their research plans can change throughout the course of the research process. SEAs should be prepared to consider modifications to the initial Data Sharing Agreement—if such requests are justified and consistent with the overall purpose of the original research plan. Any such meaningful modifications should be formally requested by the researcher and reviewed by the SEA by means of an Agreement Modification form (see appendix E).

Researchers sometimes want to maintain datasets over long periods of time because they might some day need to explain or defend their research. This possibility does not automatically justify the risks associated with indefinite storage outside the authority of the agency. One solution is for agencies to store datasets for researchers, who may request access to the files should they be needed for future use.

An SEA may also choose to conduct security audits to confirm that data are properly managed and protected. If at any time agreed-upon storage and access-control processes are not being observed by the researcher or the security of the data are not sufficiently guaranteed, the agency is obliged to respond. Appropriate responses may range from requiring corrective action that addresses security deficiencies to terminating the project. In the latter case, the agency must retrieve any data previously shared that could disclose personally identifiable information about an individual student, and subsequent destruction of the data should be documented. Such a response should be specified in the agency’s data sharing policies and agreed upon in MOUs, Data Sharing Agreements, and Personal Access Agreements (see appendices).[9]

Review Research Outcomes

At the conclusion of a research project, an agency may wish to review the findings and proposed publications prior to public release in order to prevent the unintended disclosure of personally identifiable information. It should be noted that although this review is a recommended practice, it may be difficult to enforce and can create a substantial burden on agency staff. For example, while it is in the best interest of the agency to review cell sizes in draft publication tables in order to confirm that suppression rules have been upheld, this task is labor-intensive and requires significant statistical expertise. However, whenever possible, an SEA should take the time to review outcomes and offer researchers additional information that may clarify or otherwise improve their research.

Confirm Project Completion and Data Destruction

A considerable amount of excitement accompanies the publication of research results and completion of a research project. It is imperative, however, that researchers engage in proper data destruction practices when a project is complete. Researchers should be informed of appropriate data destruction procedures during data use training, and such guidance should be consistent with all procedures described in the Data Sharing Agreement (see appendix D). Several states have implemented policies requiring researchers to actively certify, via a written email statement, that data have been destroyed as agreed upon. A template for a Certificate of Data Destruction is included in appendix G.

Use Research Findings

SEAs that provide data to researchers have sometimes expressed a general disappointment that, too often, they never hear back from researchers after requested data have been shared. Researchers, in turn, may be surprised to learn that SEAs are interested in knowing about the progress of their research and the results of their studies.While FERPA does not explicitly require SEAs to obtain research findings, the studies exception in particular requires that the researcher be working “for or on behalf of ” the educational entity.Without sharing findings, it is difficult to see how research could be on behalf of an educational entity. If the SEA is thoughtful about specifying the expected benefits of the proposed research during request negotiations, and integrates those expectations into subsequent agreements, then a post project follow-up process is appropriate. In some cases, research results can be adapted or adopted by an agency for policy development, program review and improvement, or the resolution of technical and operational issues.

Build Partnerships with Researchers

SEAs that are interested in developing a research agenda—including encouraging research that serves the goals and priorities of the SEA, using the results of research to improve educational practices, and developing mutually-beneficial partnerships with researchers— should work to build relationships with education researchers. Ongoing communication can benefit both the SEA and the researcher, and lead to better research.

Monitor Data Use: Action Items

- Confirm researcher adherence to agreements through project monitoring and data security audits.

- Consider reasonable modifications to the data sharing agreement when the researcher submits the appropriate forms and justifies any modifications to the original request.

- Ensure that approved modifications are stored and tracked with the original request.

- Review research outcomes to prevent any unintended disclosure of personally identifiable information.

- Confirm project completion and data destruction.

- Use the research findings to inform the work of the SEA.

- Build mutually beneficial partnerships with researchers through ongoing communication.

Why Should an SEA Monitor Data Use?

- Research papers may not accurately present data tables in publications and other public releases, or may inadvertently contain small cell sizes (counts of individuals represented in that cell) that would allow the identification of an individual.

- Researchers may make errors based on incorrect understanding of the data (e.g., if they do not have access to metadata showing that subgroup coding changed at some point during the collection of a longitudinal dataset).

End Notes

1. For more information about statewide longitudinal data systems, visit the IES Statewide Longitudinal Data Systems Grant Program website at http://nces.ed.gov/programs/SLDS/ or see the Forum series Traveling Through Time:The Forum Guide to Longitudinal Data Systems, Books I-IV, available at http://nces.ed.gov/forum/publications.asp

Additional information about data stewardship and governance is available at http://www2.ed.gov/policy/gen/guid/ptac/pdf/issue-brief-data-governance-and-stewardship.pdf. Return to text

2. For more information about data governance and management practices, see Traveling Through Time: The Forum Guide to Longitudinal DataSystems, Book III: Effectively Managing LDS Data, available at http://nces.ed.gov/forum/pub_2011805.asp. Return to text

3. For more information about metadata, see the Forum Guide to Metadata, available at http://nces.ed.gov/forum/pub_2009805.asp. Return to text

4. More information about privacy and confidentiality is available in the following NCES publications:Basic Concepts and Definitions for Privacy and Confidentiality in Student Education Records http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2011601 , Data Stewardship: Managing Personally Identifiable Information in Electronic Student Education Records http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2011602 and Statistical Methods for Protecting Personally Identifiable Information in Aggregate Reporting http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2011603 Return to text

5. For more information about metadata, see the Forum Guide to Metadata, available at http://nces.ed.gov/forum/pub_2009805.asp. Return to text

6. See https://ceds.ed.gov for information about the Common Education Data Standards (CEDS), which is a national collaborative effort to develop voluntary, common data standards for a key set of education data elements that streamline the exchange and comparison of data across education institutions and sectors. Return to text

7. Sharing data between agencies can have implications on student privacy rights. For more information about the Family Educational Rights and Privacy Act (FERPA), visit http://www2.ed.gov/policy/gen/guid/ptac/index.html Return to text

8. Data matching refers to the act of combining datasets to match one set of data with other data, with the possible direct or indirect consequence of disclosing personally identifiable information about individual students. Return to text

9. More information on the security auditing process is available from the U.S. Department of Education’s Privacy Technical Assistance Center at http://nces.ed.gov/programs/ptac. Return to text