National Forum on Education Statistics - Winter Forum 2011 Meeting Notes

National Forum on
Education Statistics

Newsflash

Winter Forum 2011 Meeting Notes

National Forum on Education Statistics
February 21-22, 2011
Austin, TX

Opening Session

Joint Session: Connecting K–12 to Early Childhood Education and the Workforce

Joint Session: Privacy Issues

Joint Session: Public Domain Clearinghouse

National Education Statistics Agenda Committee (NESAC) Meeting Summary

Policies, Programs and Implementation (PPI) Standing Committee Meeting Summary

Technology (TECH) Standing Committee Meeting Summary

Closing Session

Steering Committee Meeting Summary

Opening Session

Monday, February 21, 2011

Welcome and Opening Comments (1.98 MB)
Forum Chair Kathy Gosa (Kansas State Department of Education) welcomed Forum members to the 2011 Winter Forum Meeting in Austin, Texas. She spoke about the contemporary issues facing the education data community and reminisced about the days, just a few years ago, when assigning unique student identifiers was the major challenge, noting the tremendous progress that has been made in recent years. Kathy then introduced this year's Forum officers and welcomed twenty-one new members to the Forum

DeDe Conner, Kentucky Department of Education
Thomas Deadrick, Marion County Schools (WV)
Daniel French, Bennington Rutland Supervisory Union (VT)
Sally Gordon, Minnesota Department of Education
Craig Hoyle, Regional Education Laboratory-Northeast & Islands
Sandy Hyslop, New Hampshire Department of Education
Candy Johnson, Kentucky Department of Education
Joshua Klein, Oregon Department of Education
Douglas Levin, State Educational Technology Directors Association
Judy Merriman, South Dakota Department of Education
Dorice Miller, North Dakota Department of Public Instruction
Esmeray Ozdemir, Nevada Department of Education
Marshall Patton, West Virginia Department of Education
Marilyn Peterson, Nebraska Department of Education
Joyce Popp, Idaho Department of Education
Eli Pristoop, Bill & Melinda Gates Foundation
Geir Solvang, Wyoming Department of Education
Paul Tisdale, Biloxi Public School District (MS)
Brian Townsend, Vermont Department of Education
Ken Wagner, NY State Education Department
David Wu, Hawaii Department of Education

Kathy then reviewed the Forum's mission and announced the release of three new Forum products: Prior-to-Secondary School Course Classification System: School Codes for the Exchange of Data, and the second and third installments of the Traveling Through Time: The Forum Guide to Longitudinal Data Systems series, Book II: Planning and Developing an LDS and Book III: Effectively Managing LDS Data. Kathy also reminded members of the summer release of the The Forum Guide to Data Ethics Online Course. She then reviewed the meeting agenda and website statistics showing that Forum website traffic continues to grow with an average of 8,515 visits per month since June 2007, which spiked this January, when website visits reached nearly 18,000 in one month . During this same period, Forum publications were downloaded as PDFs or were visited on their home pages over 6,619 times per month.

Connecting K–12 to Early Childhood Education and the Workforce
K–12 And Early Childhood Data Connections (3.66 MB)
K–12 and P-20 Workforce Connections (1.54 MB)

Kathy Gosa and co-presenter, Carol Jenner (Washington State Education Research and Data Center), delivered a presentation on the linkage of K–12 data to early childhood education (ECE) and the workforce.

Kansas has made significant progress towards establishing the link to ECE. One of the first challenges was to identify the myriad public and private sector organizations that have established ECE programs - programs that are funded through various sources and serve many different purposes. This reality introduced a secondary challenge stemming around determining which student outcomes that should be measured, and thus, which data elements would need to be collected to support relevant analyses. Other challenges included establishing a shared understanding of "basic" data terminology; convincing ECE staff of the importance of collecting individual, longitudinal student data; and training ECE staff on good data entry and management practices to ensure that the data will be of high quality. The Kansas Early Childhood Comprehensive Systems plan is the current framework encompassing most of the state's ECE programs and stakeholder groups. Through this framework, Kansas has developed eight policy questions for the linkage and use of ECE and K–12 data focused on "determining and supporting the effectiveness of ECE programs in the P-20 continuum." To answer these questions, Kansas assigns student IDs to all ECE students and collects data on enrollment, attendance, and program participation, and will soon collect additional program-specific data. Thus far, Kansas has identified a strong collection of P-20 data both within the education agency's system and in partner organizations. Work is underway to meet the challenge of governing the data, consolidating (or more effectively linking) the information, and sustaining the effort.

Carol Jenner (Washington Education Research and Data Center (ERDC)) discussed her organization's work to connect K–12 and workforce data. The ERDC has partnered with organizations from a number of sectors to share data about students as they pass from early childhood through the workforce. With these data, the ERDC consolidates and shares information with partners; track outcomes; collaborates on P-20 research projects; responds to ad-hoc requests from partners, the legislature, and external parties; provides the legislature with needed information; and creates feedback reports on behalf of other agencies. The linkage to workforce data, which relies on Social Security numbers, is used to study both outcomes and the employment status of current students. The ERDC also looks at information such as employment status, earnings, employment by industry, hours worked, the employment paths of teachers, and training program and baccalaureate degree outcomes. Ms. Jenner also discussed the current and planned governance structure and the process for handling requests, as well as the ERDC's future research plans.

Joint Session: Establishing and Using the Teacher-Student Link

Monday, February 21, 2011

Establishing and Using the Teacher-Student Link
David Weinberger (Yonkers Public Schools, NY) moderated a panel of SEA representatives considering the establishment and use of a link between teacher and student data. Tom Ogle (Missouri Department of Elementary and Secondary Education), Peter Tamayo (Washington State Office of Superintendent of Public Instruction), Sonya Edwards (California Department of Education), and Bob Swiggum (Georgia Department of Education) described many aspects of their state's work on this front. Each panelist provided some background on their state's work and described their current status and issues, including collaboration with other states or organizations (GA was the only state on the panel that is part of the CELT Teacher of Record (TOR) initiative). They also addressed the mechanics of the link, major challenges experienced, uses of the link, LEA involvement, data validation, vendor coordination, and the status of defining TOR. Details were also provided in a handout (48 KB). Following the presentation portion of the session, the audience asked a number of questions on issues such as the frequency of teacher data collections, teacher's ability to validate rosters, teachers' union resistance, and local data burden.

Joint Session: Privacy Issues

Tuesday February 22, 2011

Privacy Issues
Melanie Muenzer (Office of Planning, Evaluation, and Policy Development, ED) joined the Forum to discuss the Department's new initiatives related to privacy, security, and confidentiality. Ms. Muenzer informed the group that a new notice of proposed rulemaking (NPRM) on the Family Educational Rights and Privacy Act (FERPA) is currently with the Office of Management and Budget and is expected to be released in the Federal Register for public comments in March. Melanie discussed the NPRM process in detail and stressed that the Department is interested, specifically, in feedback on how the proposed changes will affect the states. She also focused on several recently launched privacy related products and initiatives at ED. A series of technical briefs have begun to provide guidance on privacy issues such as data stewardship, security, and statistical methods. Three have been released for comment and another four are being developed. These briefs will eventually become non-regulatory guidance. A position for Chief Privacy Officer (CPO) is being created and should be filled in the near future. The CPO will oversee the Family Policy Compliance Office (FPCO) and the Privacy Technical Assistance Center (PTAC), and will serve as adviser to the Secretary on all ED policies and programs related to privacy, confidentiality and data security. Finally, Melanie introduced PTAC, which was recently launched to provide timely and accurate information and guidance about data privacy, confidentiality, and security issues and practices in education and closely related fields; to disseminate this information to the field and the public; and to provide technical assistance to key stakeholders. Follow up discussion occurred in the NESAC, PPI, and TECH standing committees.

Joint Session: Public Domain Clearinghouse

Tuesday February 22, 2011

Public Domain Clearinghouse (2.15 MB)
Jeff Sellers (Statewide Longitudinal Data Systems Program) introduced the Forum to the Public Domain Clearinghouse, a new resource being developed by the SLDS Program at ED to facilitate more efficient sharing of non-proprietary SLDS-related tools and documents. The concept, which was initiated by states, is to help SEAs easily share their work on a centralized platform, helping them to leverage other's solutions to common challenges while saving resources. Initially, the site, which is currently being designed and developed, will be piloted with a small number of products; it will eventually be opened up to a broad audience of states (both grantees and non-grantees) as well as districts. The searchable database of products will be accessible through the SLDS Program's GRADS360° website, which is currently being rolled out to states. The products will include tools (technical and organizational in nature) and sample documents, and will be housed directly on the site for downloading after a user agreement is accepted. NCES will not endorse the products and contributing states will not be held liable for issues experienced. The initial group of tools will include the Colorado Growth Model, Georgia's "Tunnel," and Oregon's DATA Project, as well as over 250 other documents. Users of these products will be asked to provide feedback and will have the opportunity to upload modified versions for others to use. Basic support on PDC products will be provided by the SLDS Program's State Support Team experts, which will serve to lessen the burden on contributing states. Before closing, Jeff described the processes for sharing and using products through the PDC, the PDC governance process, and some preliminary screen designs for the site. Follow up discussion occurred in the NESAC, PPI, and TECH standing committees.

National Education Statistics Agenda Committee (NESAC) Meeting Summary

Monday, February 21, 2011

Morning Session

Welcome and Introductions
NESAC Chair Pat Sullivan (Texas Education Agency) welcomed the committee members to Austin and to the NESAC Subcommittee. Following member introductions, Pat took a moment to remind business partners and vendors that while their attendance in these sessions is appreciated, they may risk disqualification in RFP competitions by being part of sensitive conversations during the subcommittee proceedings. Pat then reviewed the meeting agenda and proceedings of the Summer 2010 meeting.

Connecting K–12, Early Childhood, and Workforce (Follow Up Discussion)
Carol Jenner (Washington State Education Research and Data Center (ERDC)) and Kathy Gosa (Kansas Department of Education) joined the committee for a follow up conversation from the morning's opening session on connecting K–12, early childhood, and workforce data. The morning's presentation was particularly timely as states are in various stages of implementing "P-20W" longitudinal data systems in accordance with the terms of the State fiscal Stabilization Fund requirements.

Ms. Jenner's presentation focused on Washington State's approach to postsecondary and workforce connections, giving Forum members a very real use case for this type of data. Jenner hopes that by showing what states are capable of doing with the appropriate use the Social Security Number (SSN) it might entice policymakers elsewhere to loosen their restrictions on this vital piece of data. A very good use case of the SSN is the ability to link to the Justice System and the entry/exit of prison inmates. Washington will be looking at these data in an attempt to identify the number of high school dropouts who enter the corrections system. Washington State has detailed the process and outcomes of their implementation online; this site includes links to research briefs, visualizations of the data illustrating usage potential, and a listserv subscription for those wishing to receive news and updates from the project. Washington's project started in 2007; staff worked closely with the legislature to gain buy-in and identify what elements were need to make sound policy.

Gosa presented the early childhood side of the equation, providing a look at what Kansas is trying to do to bring these data into their system. A key challenge states and districts are facing in the collection of early childhood data is how to get non-state licensed programs to submit data. Kansas is working toward this but starting by identifying students who are in programs that require a state-issued student ID.

Afternoon Session

Assessment Consortia Overview
To fully meet the dual needs for accountability and instructional improvement, states need assessment systems that are based on standards designed to prepare students for college and the workplace, that more validly measure student knowledge and skills, that better reflect good instructional practices, and that support a culture of continuous improvement in education. The U.S. Department of Education's Comprehensive Assessment Systems grant supports the development of such assessment systems by consortia of states. On September 2, 2010 the Department announced that the Partnership for the Assessment of Readiness for College and Careers (PARCC) and the SMARTER Balanced Assessment Consortium (SMARTER) had earned competitive grants.

Peter Tamayo (Washington Office of the Superintendent of Public Instruction) serves as the technical lead for the SMARTER Consortium; he presented to the committee an overview of the RTT Assessment competition and the implications for data systems. States in both the SMARTER and PARCC consortia will use consortium developed tests as part of their federal accountability and will be required to adopt the Common Core State Standards.

The SMARTER Consortia has 16 governing and 14 advisory states, with Washington serving as the fiscal agent and WestEd providing project management support. PARCC has 13 governing states and 12 participating states. Florida is their fiscal agent and project management is provided by Achieve, Inc. Each consortium will manage their assessments differently. SMARTER is a computer adaptive test using a traditional "end of year" approach. PARCC is a computer based test using a "through-course" model of summative assessments for accountability. Each consortium has slightly differing timelines with implementation set for the 2014-2015 school year. Both consortia are required to build a method of sharing student level data across consortia states in order to improve instruction and share outcomes. Access to "rapid time" data will be an essential element for states in both projects.

Both SMARTER and PARCC are in the beginning stages of building these systems and developing their assessments. As these projects ramp up and begin to implement their systems it will be essential to communicate with all stakeholders. NESAC members wishing to find more information about the consortia can visit their websites. For a summary and side by side comparison visit here.

Crime Violence and Discipline Taskforce
Bill Smith (Sioux Falls School District, SD) reported that the Crime, Violence, and Discipline task force is putting the finishing editorial touches on their document, which should be ready to go to the printer shortly. This task force, sponsored by NESAC, worked diligently to update the Safety in Numbers publication and is very pleased with the outcome.

Defining and Developing Student Transcripts
Tom Howell (Michigan Center for Education Performance Information (CEPI)) and Kristina Martin (Macomb Intermediate School District, MI) joined the committee to talk about Michigan's initiative to track its students longitudinally to ensure all students are college and career ready. Michigan has a number of critical projects underway to work toward this goal:

The Unique ID Extension –Michigan does not have any oversight of their institutions of higher education creating and sharing the state's student ID; this project is an attempt to expand the use of the ID into higher education.
E-Transcripts—Michigan is using Docufide for its transcript service; this was judged to be the most flexible proposal and had the best capability to work with both K–12 and higher education in the state.
Postsecondary student level data collection—through this project Michigan came up with a metadata system that will assist with the completion of America COMPETES elements and SFSF indicators C11 and C12.
National Student Clearinghouse—Michigan is working with NSC to match high school graduates with postsecondary enrollments in other states; NSC will hold on to the Michigan ID in order to continually improve the quality of the data.
Statewide Longitudinal data system—Michigan has an IES SLDS Grant to help support the development of their state data system.

Michigan sets the bar for the term "local control" but has been able to move forward with their transcript initiatives through a healthy combination of communication, relationship building, dedicated training, and building on current infrastructure. The biggest implementation challenge faced in the state has been to get to the right venues at the right times in order to make sure the right information is being to every stakeholder. CEPI was sure to make the tools and process it implemented added value for the LEAs across the state and added value to the institutions of higher education; these projects are not just about compliance data collection but are truly dedicated to providing stakeholders with access to useful data to ensure that Michigan's students are graduating college and career ready. This consistent message has been carried throughout the process from planning and requirements gathering to building, implantation, and use. At the LEA level the implementation of the transcript system proved to be relatively easy, with the design of the system and its standardization lending itself to efficiencies not previously seen.

Monitoring and responding to the Federal Register
Maureen Wentworth (Council of Chief State School Officers) provided NESAC members with an overview of the Federal Register, including where to find information and how to respond to listings. The Federal Register is the U.S. Government's official daily publication that informs citizens of their rights and obligations, opportunities for funding and federal benefits, and actions of federal agencies for accountability to the public (including data collection burden). It was authorized by the Paperwork Reduction Act of 1980; the same act defines both information collection and burden. According to the law, any piece of information being requested from 10 or more people requires a public comment period posted via the Federal Register. This process follows well defined timeline

A program office designs a data collection;
The program office submits OMB 83i, an official form that requires the program office to outline the collection and estimate the burden;
OMB issues the 60 day comment period notice which is posted in the federal register;
Following the close of the comment period the program office must respond to all comments;
OMB the issues a 30 day comment period notice (also posted in the Federal Register) along with the responses to the comments from the 60 day period;
The program office is required to respond again to public comments that come in from the 30 day period; and
Upon evaluation, OMB may approve the collection.

Information from the Federal Register is sent to Forum members regularly via the Forum listserv. This email summarizes each collection that indicates an increased burden for SEAs or LEAs; the weekly email also includes grant opportunities and information from other agencies that may be relevant or of interest. Reviewing and commenting to the Federal Register is the only way to voice an opinion about increasing data collection burden.

Bullying Data and Collecting Incident Level Data
Pat Sullivan (Texas Education Agency) led the committee in a discussion about incident reporting and the collection of data about bullying. Recent news about the devastating effects of bullying have prompted numerous policymakers to ask about data SEAs and LEAs currently have on bullying and what they should be collecting to better identify issues and intervene prior to a tragedy. This data topic presents a number of logistical questions around definitions, identification of incidents, in school and out of school jurisdiction, reporting, and privacy. The quality of these data is often dependent on a victim's comfort level with reporting the incident or a bystander's understanding of the incident. LEAs tend to have a better handle on this information and local authorities have been grappling with these questions for many years. Many LEAs have found that introducing data collections on bullying greatly increases the number of incidents—greater awareness and recognition leads to reporting incidents that might have previously been ignored. Policymakers should be cognizant of this possibility and be prepared to respond accordingly.

Tuesday, February 22, 2011

Morning Session

Public Domain Clearinghouse Follow up
At the morning's general session, Jeff Sellers of IES Statewide Longitudinal Data Systems State Support Team, presented on the SLDS Public Domain Clearinghouse (PDC). Sellers joined the NESAC subcommittee to answer questions and gain input from the members. The PDC is an effort to collect the work and products that are being developed through the SLDS grant program and make them available for use by other states. Currently, SEAs each have a single membership for the PDC, but the project is exploring how to expand access with a system to allow people to request access upon verifying affiliation. PDC currently contains resources such as reporting tools, course coding processes, Arkansas's Hive application, the Colorado Growth Model, and other open source or open code products for use with a longitudinal data system. The PDC will look to expand access and use by LEAs in the future. LEAs looking for documents and best practice are encouraged to check out LDS Share.

Teacher-Student Data Link and the Teacher of Record
The full Forum discussed the teacher student data link during the general session on Monday afternoon. David Weinberger (Yonkers Public Schools, NY) facilitated a continuation of the discussion by NESAC members. Generally speaking, LEAs have informally had teacher-student data links in place for some time. SEAs do not think they have all the answers and often struggle with using these data; however it is now important for the data to be collected at the state level. For example, growth models require at least two years' worth of scores to establish an initial growth assessment, but student mobility means that some data are not available at the LEA level unless SEAs can share it through properly built and implemented state wide longitudinal data systems that track mobile, transient, and migrant students. A robust SLDS will also provide LEAs with more accurate early warning indicators and trend data. Thus, it is important for both the states and locals to have comprehensive data systems. As each state struggles with implementing the teacher student data link it is important to engage LEAs and learn lessons from local implementation as well as from other states. There are a number of issues that need to be addressed when implementing the teacher student data link:

How are you dealing with teacher evaluation? Is this a local function?
How will you handle the collections technically? Is this a weekly collection? Monthly?
How are you defining the teacher of record? In elementary school? For special populations?
Will you allow roster verification?
How are you implementing common course codes?

As states and districts implement these systems care must be taken to engage stakeholders, communicate to vendors, and address local needs.

Privacy Follow-up Discussion
Tuesday morning's general session began with a conversation about privacy issues with Melanie Muenzer (U.S. Department of Education). Ms. Muenzer and Emily Anthony (NCES) joined the subcommittee to further discuss privacy and security. Although the new FERPA relations have not yet been released, the Department is making headway in easing confusion and consolidating resources. Recently, the Department opened the Privacy Technical Assistance Center (PTAC). This program, housed with the SLDS grant program, will create and consolidate materials, best practice briefs, and frequently asked questions. As such, PTAC will be a "one stop" shop for technical assistance around privacy and security. Major projects and resources include:

Training materials
FAQ
Glossary of terms
Security checklist
Issue briefs
On-site technical assistance visits
Help desk
Regional meetings
Presentations

PTAC will address more than just FERPA related questions and will be the coordinating body for all privacy and security related questions. As PTAC establishes itself, it will be looking for feedback, requests, and assistance from Forum members in identifying needs.

Afternoon Session

Common Education Data Standards Initiative
Stuart Kerachsky (NCES) provided an update on the Common Education Data Standards. In September, Version 1 of the CEDS elements were released. Version 2 of the standards will concentrate on postsecondary data elements while continuing to add to the K12 elements. Stuart asked members if they were aware of the CEDS elements and what can this project provide to Forum members. NESAC members requested clarification on issues such as

What is the relationship between NEDM and CEDS?
Can there be some language created to explain the differences between CEDS and other projects?
What is the expectation for SEA adoption of CEDS when building a P-20 system?

Small group breakout: Teacher of Record
The committee split into LEA and SEA groups in order to further discuss the teacher of record issue. Chair Pat Sullivan (Texas Education Agency) facilitated the discussion with the SEAs; and vice chair Cheryl McMurtrey (Mountain Home School District, ID) facilitated the discussion with the LEA representatives. The groups focused on a few key questions:

How has your state/district defined the teacher of record?
What data elements are required to support those definitions?
What stakeholder engagement and communications activities have you engaged in to prepare you schools/teachers for making the teacher student data link?
How are you using this data?

After forty minutes of stimulating discussion, members came back together to report out from their conversations. SEAs noted that they are all in different stages of development and implementation. It was clear that no two states were defining teacher of record in the same way. LEAs struggle with defining teacher of record, pointing out that many are working on it but none were in a position to give a definitive response. NESAC members felt that there needs to be a clearly articulated vision/reason for collecting and using these data—communicated to stakeholders at all levels. There is a lot of unease around the teacher of record definition and the reasons for this type of data collection. SEAs, LEAs, policymakers, and implementers need to be clear about the need for teacher data, how it will be used, and insurances on how it will and will not be used by policymakers.

Steering committee business meeting report
Pat Sullivan (Texas Education Agency) reported back to the committee on the proceedings of the Steering Committee meeting. The steering committee discussed the Sunday and Monday activities and the progress of the current task forces. Two work groups were discussed—one on the teacher student data link and one on succession planning. NCES staff will consider these recommendations in light of available resources. The committee had a stimulating conversation about the directions for the Forum and its value. Steering committee members were adamant that the Forum is more important than ever, especially with respect to activity at the national, state, and local levels around reform and the need for data. The Steering Committee believes that our Forum is uniquely positioned to intelligently discuss issues at all levels and communicate up and down the chain. The value of the Forum publications is immeasurable, as SEAs and LEAs could not have implemented so many new initiative over the last few years with the assistance of these publications.

Planning for next meeting
NESAC members finished their two days of meetings by looking forward to the summer meeting in Bethesda, MD. The committee the following items for the summer agenda

Continued discussion on the teacher student data link and teacher of record (Linda Rocks volunteered to demonstrate Louisiana's roster verification system);
Continued discussions on the assessment consortia and further discussions with both SMARTER Balanced and PARCC;
Discussion around supplemental plans for using technology for developing curricular materials and tools;
The linking of education data with other state agencies and intra-agency data sharing;
FERPA and the Civil Rights Data Collection; and
Updates, processes, experiences, and lessons learned from the Investing in Innovation (I3) Grants and other ARRA grants directed to LEAs.

Policies, Programs and Implementation (PPI) Standing Committee Meeting Summary

Monday, February 21, 2011

Morning Session

Welcome and Introductions
PPI Chair Laurel Vorachek (Anchorage School District, AK) called the PPI committee to order and asked members to introduce themselves.

Agenda Review
Laurel Vorachek outlined the PPI agenda for the summer meeting.

Summer 2010 PPI Meeting Review
Laurel Vorachek reviewed the work PPI accomplished at the Summer 2010 Forum, noting that many of the topics would be addressed as updates at this summer meeting.

Data Use Working Group Update
Kathy Gosa (Kansas State Department of Education) provided the update for the Data Use Working Group. This group has met twice since the summer meeting. It hopes to release its products as a series of briefs, the first for educators, then school and LEA administrators, and finally SEA program directors. The first brief is scheduled to be completed in Summer 2011, the second in Fall 2011 and the final brief and final product in Winter 2012.

Connecting K–12, Early Childhood, and Workforce (Follow Up Discussion)
Carol Jenner (Washington State Education Research and Data Center) and Kathy Gosa (Kansas Department of Education) came to the PPI Committee to answer member questions. PPI members had many questions and discussion points related, for example, to policy decisions informing their P-20 systems, the attitude of private colleges about sharing data, how to collect the missing information, and how these connections can be made without using the SSN?

Communications Subcommittee
Laurel Vorachek discussed a new statement to be read at each committee meetings for non-Forum members. She then read the statement at the start of each section of the meeting.

Laurel gave a short presentation (1.58 MB) on one of the Forum's products from 2006, Accounting for Every Student: A Taxonomy for Standard Student Exit Codes. Members discussed how important the sections in the Forum documents are that discuss the context, caveats, best practice, etc. This led to a discussion about whether it was time to update this document or if it should be archived or re-presented without state specific information. Members felt this conversation applied to all Forum documents. This topic was taken to the steering committee.

Afternoon Session

CRDC Presentation (100 KB)
Rebecca Fitch and Ross Santy (USED) gave an overview of some of the current issues with the CRDC collection at ED, including the schedule and some of the guiding business rules. There is a new CRDC Work Group that is meeting at the Forum for the first time. Members had the following questions and comments for ED consideration:

What about schools that are rented from private organizations?
In some states (e.g., Vermont), technical schools are not considered enrolling organizations so they aren't reported as schools?
What subgroups should be used, AYP or what the state uses?
Can ED notify SEAs when their LEAs change their pre-populated data or directory?
Is ED aware that SEAs are losing staff and this isn't going to be a high priority?

Challenges and Advantages of Statewide Data System Discussion
PPI vice chair Tom Howell led a discussion about statewide data systems, a topic that was started via email before the meeting. Tom reviewed where several states are and several more states discussed their progress towards these systems. The systems included ones in which the state purchased or developed the software for all districts to states that have multiple district systems. Members discussed issues such as implementation costs, funding sources, how LEAs validate the data in these systems, etc.

Tuesday, February 22, 2011

Morning Session

Privacy Follow-up from General Session (120 KB)
Melanie Muenzer and Emily Anthony (USED) came to the PPI Committee to provide members with an opportunity to ask questions and provide feedback on the general session regarding privacy. PPI members discussed the following topics:

CTE and GED environments are not part of K12 in MI, so how do you have the two-way data exchange?
Some traditional pathways in education are have changed in recent years and are hard to track when adhering to strict privacy restrictions (e.g., non-traditional schools, returning adults, students enrolled in high school and college at the same time).
How can earnings and employment be collected while respecting data privacy?
Education data records have to be housed in education agencies, so what happens if you add in data that is beyond education (e.g., data from Labor), and where and how is the mapping handled?
Technical briefs should include the reason why data sharing is important (not just best practice) to help explain to policymakers.
Can PTAC post what is already out there in other states (e.g., MOUs)?
Can the ED work include other federal privacy laws that impact education such as labor or other areas outside ED?

Teacher-Student Linkage Follow-up Discussion
Sonya Edwards (California Department of Education) and Peter Tamayo (Washington Office of Supt. of Public Instruction) discussed the teacher student linkage projects in their states and took questions and comments from PPI members. Discussion focused on how to involve districts in the conversation, the impact of legislation on the linkage, on the percent of the teacher evaluation that is measured by student achievement, and the challenge of defining teacher of record.

Public Domain Clearinghouse Follow-up
Jeff Sellers (Statewide Longitudinal Data Systems Program) came to PPI to follow-up on the general session on the Public Domain Clearinghouse (PDC). Jeff asked members for their feedback on the project and for comments on the role of vendors and user feedback in the Clearinghouse. PPI members discussed their concerns regarding the vendor role in the PDC. Members also wanted all of the current resources to be included in the new site and asked for a key word search to be considered.

Issues from the Floor
Laurel and Tom opened the discussion to issues from the floor. The conversation surrounded the two America Competes Act indicators that require states to report the enrollment of graduates in postsecondary and the number of credits earned by these students. Members discussed the use of the National Student Clearinghouse as an option for reporting this data.

Afternoon Session

SHEEO Strong Foundations (668 KB)
Hans L'Orange (SHEEO) provided an overview of a recent report produced by SHEEO on the status of state data systems with postsecondary data—describing state postsecondary data systems. This resource presents information based on a study conducted by SHEEO that cataloged 59 state-level student unit record (SUR) data systems containing postsecondary data in 44 states and the District of Columbia. The study updates and expands the Critical Connections study conducted by the National Center for Higher Education Management Systems and funded by the Lumina Foundation for Education. The pilot is scheduled to start this summer. Hans will come back in July and report on the progress.

Common Education Data Standards
Stuart Kerachsky (National Center for Education Statistics) provided an update on the Common Education Data Standards. In September, Version 1 of the CEDS elements was released. Version 2 of the standards will concentrate on postsecondary data elements while continuing to add to the K12 elements. Stuart asked members if they were aware of the CEDS elements and what can this project provide to Forum members. The following questions and comments were discussed:

One state noted that they used the CEDS webinar in their state to teach others about it (including postsecondary folks).
Could CEDS work with PTAC to determine which items are subject to FERPA, etc.?
Can CEDS be used to reference translation schemes between states?
CEDS should include items used in EDEN reporting (as a starting point rather than an ending point).
What about items included as part of MSIX?
EDEN is okay but does not include a lot of student outcome data used by states.

Online Data Ethics Course and Assessment (248 KB)
Tom Purwin (Jersey City Public Schools, NJ) provided an overview of a course he created based on the Forum's Data Ethics publication. The course is a series of PDF and video files that present the content of The Forum Guide to Data Ethics. Users get notified when they are registered for access to Moodle, the district's Online Course Management System. Users can sign-on to Moodle and go through the course at their leisure; the videos are typically 3-5 minutes long in 12 sections correlated to publication. Users can review the materials online or download the document and read the book. The last section of the Online Course includes an Online Data Ethics Final Assessment in which users are given three chances to pass the exam. A passing score of 80 earns a Certification via email. Students unable to pass the test after three attempts will be scheduled for a workshop.

Members asked why create this versus use the on-line curriculum that exists. Tom explained that he created this course to be a bit harder than the on-line course. For example, the on-line tool gives you the answer if you get it wrong and it asks the questions in the same sequence. Also, he needed to maintain a database of who had taken the course and completed it and some control over who needed to be pushed to take the course. Tom noted that there is a resource burden on doing this for LEAs.

Steering Committee Business/Report
Laurel provided an update from the steering committee. NESAC had a proposal for a working group on the student-teacher linkage. TECH had a proposal for a possible working group on SEA and LEA data managers. NCES is going to look at what already exists on these topics and suggest a plan for committing to such work or not. Laurel brought the PPI discussion topics on updating publications to the Steering Committee, and PPI will be provided a list of publications and which committee they belong too. Each committee would be responsible for reviewing the documents and bring recommendations to the Steering Committee.

Summer Meeting Planning
PPI members provided suggestions for the summer 2011 meeting agenda:

Early Childhood and DQC follow-up to its survey
FERPA and Privacy
Invite someone from PTAC to review the new technical briefs.
Teacher evaluation based on student achievement (maybe a panel?).
Update from Hans (SHEEO) on the postsecondary pilot study, etc.

Technology (TECH) Standing Committee Meeting Summary

Monday, February 21, 2011

Morning Session

Connecting K–12, Early Childhood, and Workforce: Follow Up Discussion
Carol Jenner, Washington State Education Research and Data Center, and Kathy Gosa, Kansas Department of Education (KSDE) joined the TECH Committee to entertain questions in a follow up to their presentation at the opening session.

	Q) Does KSDE oversee early childhood education in Kansas?
	A) No, early childhood education in Kansas is coordinated by a wide range of organizations.
	Q) What identification systems are used for early childhood education programs in Kansas?
	A) Most programs already have state IDs.
	Q) What other issues are at the forefront of ECE data collection in Kansas?
	A) Kansas is working to help its early childhood programs redefine "days in membership," "seat time," "class size," and related elements. Kansas is also working hard to identify which individuals actually submit the data so that SEA staff can provide appropriate support and training.
	Q) What are the key issues facing Washington state?
	A) WA is hoping to develop a single source for LEAs to get feedback reports on early childhood data. They are also working to identify what items would be most useful for LEAs.
	Q) How critical are ECE students' SSNs for match rates?
	A) Some schools may choose not to collect SSNs but match rates are better with SSNs.
	Q) How is the Washington ERDC organized?
	A) It is intentionally separated from all other state agencies.

TECH members noted that:

	The National Education Data Model at ED incorporates early childhood data elements, which are available to everyone (even though they are not perfectly aligned, they do cross walk to many state data systems).
	CCSSO and the DQC are actively involved in early childhood initiatives and may also be able to provide resources.

Welcome, Introductions, Summer 2010 TECH Meeting Review
TECH Chair Lee Rabbitt (Newport Public Schools, RI) welcomed everyone to the TECH meeting, led the group in introductions, and reviewed proceedings from TECH's meeting at the Summer 2010 Forum.

Winter 2011 Agenda Review
Lee Rabbitt reviewed the agenda for our time together in TECH.

TECH Discussion: Forum Publication Refresher (2.14 MB)
TECH Vice Chair Peter Tamayo (Washington Office of Supt. of Public Instruction) noted that the Forum has produced many useful publications over the years. While we do a good job of describing newly released resources, we also have older publications that are still relevant but not well understood, even by Forum members. Peter shared an overview of some of the older, but still useful, Forum publications, including the Forum Guide to Elementary/Secondary Virtual Education (2006), Forum Guide to Education Indicators (2005), and Forum Unified Education Technology Suite (2005). These publications, while aging, are still relevant and helpful. TECH members agreed, but noted that the Forum Technology Suite may require some updating (e.g., "cloud computing" was not a well-known practice at the time of publication and, therefore, is not in the current version of the TECH Suite).

Tom Purwin (Jersey City Schools, NJ) then shared a presentation (248 KB) about his application of the Forum Ethics Online Course across his school district. Driven by Tom (and a participation requirement issued by the superintendent), each staff member who accesses data in the district is required to view the online course and pass the assessment. Tom has migrated the course and assessment to a Moodle course management system application so that he could change the order of questions and answers in the assessment. Following a pilot in November 2010, over 783 staff have already enrolled in the course and nearly 300 have earned completion certificates. TECH members were very impressed with the use and inquired about replicating it in their agencies. Tom reported that he would be happy to share it.

Following the presentations, TECH members received a handout to help them plan how they might improve their outreach efforts on behalf of the Forum.

Afternoon Session

Teacher of Record Panel & Discussion (884 KB)
Panel members included Bethann Canada, Virginia Department of Education; Charlene Swanson, New York State Education Agency; Tom Purwin, Jersey City Public Schools (NJ); Mike Hopkins, Rochester Schools (NH); Raymond Yeagley, NWEA; and Larry Fruth, SIFA. Panelists reviewed the topic from the perspective of data collectors, data reporters, and data users. Issues included balancing data needs with capacity, vendor communications, linking between separate systems (HR and SIS), mapping to SCED codes, and accounting for multiple teaching roles (e.g., main teacher, team teacher, in-class supplemental teacher, pull-out supplemental teacher, after-school supplemental teacher, in-class aide, pull-out aide, and after-school aide). Other dimensions to be considered included course duration (minutes), assessment tools, enrollment, attendance, and mobility within a single academic year. In some ways, this issue is similar to collection issues related to virtual education. Panelists also mentioned related projects supported by the Gates Foundation in Arkansas, Florida, Georgia, Louisiana, and Ohio. However, in spite of advances, many data systems are not totally ready for this use and vendors will not respond to the issue of teacher-student linkages until there is a stable vision for policies and practices.

A draft Teacher of Record Data Model was presented, as was a list of factors for weighting teacher links to individual students and classes. Panelists also acknowledged numerous factors outside of the teacher-student link that might affect student performance, including after school programs, student attendance, staff attendance, curricula and standards, administrator certification, and instructional strategies (e.g., textbook selection and instructional approaches).

There was a lively debate in TECH about concerns related to the potential use and misuse of the teacher-student link. LEA members suggested that it might be more useful if these data were reported at a program participation level as opposed to the class level. Several members suggested that these data should be used to inform professional development choices rather than for teacher evaluation. There was also broad agreement on concerns about using a single assessment to evaluation students and teachers—not a sound approach logically and, perhaps, not valid or defensible statistically. Because of the many factors to consider and dearth of research to date, the conclusion of the panel was that for many students, assigning responsibility and accountability for learning in a single Teacher of Record fosters an illusion that may be politically satisfying, but is educationally weak.

Because of the concerns raised in TECH, members wondered if the Forum should consider establishing a working group on the topic. While members acknowledged that other organizations were already addressing the issue, many of them appear to have policy positions to advance – whereas the Forum might be able to evaluate the issue more objectively. If a working group is not an option, perhaps the Forum could at least write a letter to voice concerns about validity (e.g., a single assessment), practicality (e.g., a lot of variables in play), purpose (e.g., teacher development versus evaluation), and utility (e.g., program data are more useful).

TECH SEA/LEA Breakout Discussion: SEA-LEA Systems Coordination
TECH members broke into two groups: local representatives (facilitated by chair Lee Rabbitt) and state representatives (facilitated by vice chair Peter Tamayo). We concluded that models for intra-state coordination vary by state, ranging from single-system states to states with virtually no coordination at all as long as disparate systems can map to the state system. While there are pros and cons to all approaches, a concern that arose more broadly was related to perceptions about the role of data managers in education. Many LEAs, for example, seem to not recognize that a data manager is a highly trained professional with data expertise... very different than a database administrator who is in charge of the technology that stores data. For example, twenty years ago when systems were first becoming automated, people thought they now had "data" but the technology only produced numbers without any sense of data quality. Perhaps a "Forum Guide to Education Data Management" would help to present the overarching picture of the role of data managers. It would also help agencies with succession and sustainability planning. Such a document, focused on core competencies of the role in an education agency, would also help inform school administration programs in higher education. Some of this information already exists in other Forum publication (e.g., Forum Guide to Building a Culture of Quality Data: A School and District Resource), but it is currently difficult to access the information from multiple sources.

Tuesday, February 22, 2011

Morning Session

Teacher-Student Linkages: Follow Up Discussion
In a follow up to our Monday afternoon Forum joint session (and TECH discussion on Teacher of Record), TECH vice chair Peter Tamayo led a discussion about teacher-student linkages. Peter noted that the Data Quality Campaign shows that 35 states report a teacher identifier system with the ability to match teachers to students. TECH members discussed the pros and cons of having the SEA or LEA map courses (instructors) to SCED codes. There was general consensus that LEAs should do their own mapping. Although there is burden, doing so helps the LEAs build capacity and ensures that they know what, why, and how these important decisions are being made. Another question dealt with whether a teacher would receive any useful data from teacher-student linkages. Consensus suggested that teachers should receive "actionable data" to help with instruction. After all, SEAs can provide a lot of analytics and comparisons that would be helpful to local instructors. One outstanding obstacle, however, is still the issue of "instructional time." TECH members were not aware of robust theoretical models for quantifying teacher value (there are simply too many variables in play) although there has been progress of late with respect to some growth models.

SLDS and the Public Domain Clearinghouse: Follow Up Discussion
In a follow up to our Tuesday morning Forum joint session, Jeff Sellers received questions about the SLDS Public Domain Clearinghouse (PDC). He began by stressing that "we want the PDC to be useful and accessible" as a tool for sharing lessons learned, documentation, etc. TECH members asked whether PDC resources will require licensing. Jeff replied that any organization that submits a resource to the PDC will need to agree that it is appropriate for the "public domain." When asked about resources for LEAs, Jeff reported that school districts are very much on the radar as a PDC audience, but the effort would begin with a focus on SEAs. TECH members suggested that code also be available via the PDC, perhaps using the example of CodePlex, an open source project community. TECH also suggested that the EDEN data model and business rules be posted. In the near future, the PDC will be managed by the SST team using the GRADS 360 tool. The SLDS team invites feedback from stakeholders about the utility of the PDC.

Privacy: Follow Up Discussion
Melanie Muenzer (Office of Planning, Evaluation, and Policy Development, U.S. Department of Education) and Emily Anthony (NCES) visited TECH to follow up on the Tuesday morning Forum joint session on privacy initiatives at ED. They noted that the Privacy Technical Assistance Center (PTAC) is a new initiative at ED that will provide "one stop" technical assistance on best practices related to data privacy and security. The initial target audience is SEAs developing longitudinal data systems, but the vision extends to school districts as well. In addition to offering onsite technical assistance and hosting regional meetings and lessons learned forums, PTAC is developing resources such as FAQs, a glossary of terms, training materials (technical briefs, webinars, etc.), security checklists, and a help desk.

TECH members felt that PTAC would be a very useful resource and suggested the following topics of focus: P-20-workforce data sharing agreements, FRPM support, MSIX support, and DQC coordination. TECH members felt strongly that the best way for PTAC to be useful is to focus on use cases/scenarios in plain, understandable language (not legalese). Additionally, LEAs would benefit greatly from online courses about privacy so staff could participate in PTAC professional development. Similarly, OGC training for state AGs would be very helpful – in many states, whatever the AG says is all that matters.

Closing Session

Tuesday February 22, 2011

Overview and Agenda (860 KB)

NCES Update (20 KB)
Jack Buckley, the new Commissioner of NCES, visited with the Forum for the first time to provide an update on his organization's recent and planned activities. He began by expressing NCES' appreciation for the work of the Forum and its members and reinforced NCES' commitment to the organization. Jack then stressed some of his core priorities, which include the integrity of NCES and the data it collects, the relevance and timeliness of the information, the rigor of its research methodologies, and the need for NCES to continue to be an innovative organization. Commissioner Buckley acknowledged the growing complexity of the education data environment over the past decade with state K–12 assessment programs and data warehouses increasing from a handful of states to nearly all—and a scope that is quickly expanding to P-20. Policymaker demand for these data to inform decisionmaking and evaluate policies and programs has never been higher. He asserted that NCES, IES, and states must proactively work to meet this growing need. The Commissioner then reviewed a number of key initiatives that NCES is leading or involved in:

U.S. Education Dashboard—This past year the Department engaged NCES to help create a dashboard of key education indicators as part of an initiative to ensure transparency by increasing access to national and state data. Senior Department officials released the Version 1.0 at the Education Stakeholders Forum on January 24, 2011.
2011 NAEP-TIMSS Linking Study— In order to enable states to measure their NAEP performance against international benchmarks, NCES is currently administering TIMSS mathematics and science items side-by-side with NAEP items in NAEP-style booklets to a national sample of eighth-graders. This will be followed in the spring by administration to state-representative student samples in eight states.
High School Longitudinal Study of 2009–1st Follow-up in 2012—The new High School Longitudinal Study of 2009 is a nationally representative, longitudinal study of 9th graders who will be followed through secondary and postsecondary experiences to examine their learning in algebra and their decision-making about courses, college, early work, and careers.
Measuring Certificates and Certifications—NCES recently collaborated with colleagues at Census, the Bureau of Labor Statistics, the Council of Economic Advisors, and the Office of Management and Budget to develop new questionnaire items designed to measure the prevalence of educational certificates and industry-recognized certification in the US adult population.
Statewide Longitudinal Data Systems (SLDS) Grant Program—With the most recent round of SLDS grants, NCES is investing in state systems that will bridge the linkages from early childhood through workforce data systems. With this funding, NCES has launched an aggressive Technical Assistance program for all states, regardless of grant status. Through onsite and online technical assistance, the program is facilitating the exchange of best practices, providing expert feedback to states, and disseminating state-developed documents and applications.
Privacy Technical Assistance Center (PTAC)—NCES, in close cooperation with OPEPD, FPCO, the Office of the General Counsel, and other key offices at ED, recently established PTAC, which is intended to be a one-stop site for information about privacy, data security, and confidentiality. In a complementary effort, NCES' Statistical Standards Program is producing a set of technical briefs to provide confidentiality-related "best practices" to states as they build their data systems.
Common Education Data Standards (CEDS)—The second year of the CEDS project will continue the first year's K–12 work while adding a parallel focus on postsecondary education and early childhood. A contract has just been awarded to support this effort and the Commissioner stressed that establishing common standards for administrative data is a central priority of his time at NCES.
Common Core State Standards (CCSS)—With the growing momentum behind the CCSS Initiative, a development that the Commissioner sees as the "right policy for the nation," NCES is faced with the challenge of transitioning to alignment of some of the NAEP standards with the CCSS, while maintaining the decades-old Long Term Trend NAEP standards to allow for continued time series analysis.

Standing Committee Progress Reports

Recognition of Completed Projects and Forum Officers
Ghedam Bairu presented plaques to the members of the Prior-to-Secondary Course Classification Working Group for their completion of the new SCED code document, and acknowledged the release of two additional releases from the Longitudinal Data Systems Task Force.

Closing
Kathy thanked the Forum for participating in another great meeting and looked forward to seeing everyone again in the summer.

Meeting Evaluations
Forum members shared their opinions on the Winter 2011 Forum Meeting by completing evaluations.

Steering Committee Meeting Summary

Monday February 21, 2011

Welcome and Agenda Review
Chair Kathy Gosa (Kansas State Department of Education) opened the Steering Committee meeting and welcomed members to the 2011 Winter Forum Meeting.

Review of Sunday Events
The Data Use Working Group meeting led to significant redirection of the group's vision for the resources it hopes to develop. It now expects to release its products as a series of briefs, the first for educators, then school and LEA administrators, and finally SEA program directors. The first brief is scheduled to be completed in Summer 2011, the second in Fall 2011 and the final brief and final product in Winter 2012.

The Communications Subcommittee met and advanced several topics of importance to Forum communications with members and externally, including updates to the Policies and Procedures Manual, Forum thank you letters, website revisions, the Forum Voice, a kiosk for use at meetings, the Forum Overview PPT, and publication/product announcements.

Review of Monday Events
Members thought the Opening Session went very well, including the early childhood/workforce presentation and discussions it spawned in standing committees. The teacher link panel was also very well received.

Next, Standing Committee Chairs reported on the day's discussions:

	NESAC
	PPI
	TECH

Marie Stetser (NCES) then inquired about the role of the Forum as support for federal initiatives. After a lively discussion, Steering Committee members felt strongly that the Forum plays a unique role in the federal-state-local education data dialogue and that Forum publications were tremendously helpful to SEA and LEA staff looking to improve data systems, data quality, data capabilities, and other issues that improve the overall quality of federal data.

Tuesday February 22, 2011

Review of Tuesday Events
Members thought the Privacy and Public Domain Clearinghouse joint session was right on target—both providing important and useful information for Forum members.

Next, Standing Committee Chairs reported on the day's discussions were rich, balanced, and a good use of members' time:

	NESAC
	PPI
	TECH

NCES Commissioner Jack Buckley thanked the Steering Committee for the good work of the Forum and pledged his continued support of this productive and highly respected partnership between NCES, state education agencies, local education agencies, and other partners in national organizations and federal agencies.

Summer 2011 Forum Planning
Ghedam will send information about our next planning call. At that time, we'll discuss possible ideas for a pre-meeting professional development speaker.

Meeting Notes

	How does/not FERPA apply to MSIX?
	What type of MOUs did DODD use for its voc ed follow up study?
	Can best practices also be written for the LEA level? If so, focus on policy more than technology
	Can PTAC focus on what is allowable with respect to FERPA? In the past, FPCO has focused heavily on what cannot be done.
	What happens when a parent records a school even and then posts it on Youtube, Facebook, etc.?

	One state has established a district-state partnership to create a model that is efficient and valuable to state legislators.
	One state is creating a federated system in which multiple agencies contribute to sustaining a shared system.
	One state is hiring limited duration staff to backfill regular staff during SLDS development.

	TECH members voiced a request to change the format of the standing committee agendas. In the future, could the agenda include all events in the day, including lunch and joint sessions, so that readers don't need to flip back and forth to the full Forum agenda to know where they need to be.
	There are several national issues that affect states (e.g., e-transcripts, college enrollment data links, etc.). Can the Forum help coordinate a national response to these issues by working with ED, CCSSO, DQC, etc. For example, multiple organizations are trying to address the e-transcript problem (e.g., Docufide and the Clearinghouse). How can the Forum help?

Winter Forum 2011 Meeting Notes

National Forum on Education Statistics February 21-22, 2011 Austin, TX

Contents

Opening Session

Monday, February 21, 2011

Joint Session: Establishing and Using the Teacher-Student Link

Monday, February 21, 2011

Joint Session: Privacy Issues

Tuesday February 22, 2011

Joint Session: Public Domain Clearinghouse

Tuesday February 22, 2011

National Education Statistics Agenda Committee (NESAC) Meeting Summary

Monday, February 21, 2011

Tuesday, February 22, 2011

Policies, Programs and Implementation (PPI) Standing Committee Meeting Summary

Monday, February 21, 2011

Tuesday, February 22, 2011

Afternoon Session

Technology (TECH) Standing Committee Meeting Summary

Monday, February 21, 2011

Tuesday, February 22, 2011

Closing Session

Tuesday February 22, 2011

Steering Committee Meeting Summary

Monday February 21, 2011

Tuesday February 22, 2011

National Forum on Education Statistics
February 21-22, 2011
Austin, TX