Winter Forum 2009 Meeting Notes

National Forum on Education Statistics
February 16-17, 2009
Seattle, WA

Contents

Opening Session

Monday, February 16, 2009

Welcome and Opening Comments Zip File  (953 KB)
Forum Chair Bruce Dacey (Delaware Department of Education) welcomed Forum members to the 2009 Winter Forum Meeting in Seattle, WA. Bruce introduced this year's Forum officers and announced the release of three new Forum products – Managing an Identity Crisis: The Forum Guide to Implementing New Federal Race and Ethnicity Categories , Every School Day Counts: The Forum Guide to Collecting and Using Attendance Data, and the Education Data Model Version 1 (PK-12). He then reviewed the meeting agenda and reported that Forum website traffic continues to grow with an average of 6,821 visits per month since June 2007. During this same period, Forum publications were downloaded as PDFs or were visited on their home pages nearly 6,000 times per month. The Chair then welcomed fourteen new members to the Forum:

  • Steve Canavero, Nevada Department of Education
  • Sheila Corey, Alaska Department of Education and Early Development
  • Tom Howell, Michigan Center for Educational Performance and Information
  • Gayle Johnson, Illinois State Board of Education
  • Irma Jones, Tennessee Department of Education
  • Frank "Steve" Snow, North Dakota Department of Public Instruction
  • Randolph Thomas, U.S. Virgin Islands Department of Education
  • Gary West, South Carolina Department of Education
  • Edward Eiler, Lafayette School Corporation (IN)
  • David Feliciano, Savannah-Chatham County Public Schools (GA)
  • David Person, Maine School Administrative District #54
  • Hans P. L'Orange, State Higher Education Executive Officers (SHEEO)
  • Christopher Lohse, Council of Chief State School Officers (CCSSO)
  • Nancy J. Smith, Data Quality Campaign (DQC)

Welcome to Washington
Peter Tamayo (Washington State Office of Superintendent of Public Instruction) welcomed the Forum to Washington for the Winter 2009 Meeting.

Data Systems and the Data Informed District Zip File  (18 KB)
Dr. Jeffrey C. Wayman, Assistant Professor at the University of Texas at Austin, presented on data systems and the "data-informed district" (DID). Dr. Wayman began by considering the types of student data that are commonly collected as well as the rationale for collecting them. He then reviewed the concept of the DID, which is characterized by:

  • a focus on and common understanding of teaching and learning achieved through the process he calls "calibration,"
  • broad integration and alignment of data systems,
  • emphasis on professional development to "advance" data use in addition to just "increasing" it,
  • data use as a part of educators' normal routines, and
  • a central focus on technology (while realizing that technology alone, is not sufficient).

Three main types of data systems that are commonly used relative to teaching and learning were compared, noting the benefits and limitations of each:

  • Student Information Systems provide real-time accounting of daily school functions, with no historic capacity;
  • Assessment Systems rapidly organize and analyze frequent benchmark assessments; and
  • Data Warehouse and Reporting Systems provide access to historical data of all types, with user-friendly interfaces, though there is a lag time in populating the system with recent data.

Dr. Wayman listed a number of issues that should be considered when choosing a data system including: alignment with district goals and context, total and complete integration, interoperability, user-friendly access and navigation. He then argued that the focus of data use should broaden from just compliance and simple monitoring to encompass deeper issues of teaching and learning.

Next, Dr. Wayman reviewed findings of a case study from a Wyoming district in which he and colleagues studied data use practices and the data system environment. Though district leaders were supportive of data use, school level support was inconsistent, there was no clear vision for data use, and actual use of the data, especially for teachers, was challenging and inefficient. Following the study, the researchers made recommendations for improving the district's data use practices, including calibration (e.g., getting stakeholders to collectively define their goals and improvement strategies), acquisition of an integrated data system that provides easy access to a central data warehouse, and implementation of a district-wide data initiative.

Finally, Dr. Wayman offered his views on where data systems are headed in the future, forecasting that issues of importance will include portals, integration and interoperability, teacher evaluation, and a focus on using data to inform better education practices. Dr. Wayman is currently working on a prototype for system reviews by means of funding from the Dell Foundation. And, with support from the Spencer Foundation, Dr. Wayman will begin to study the effects of the DID on student performance and specific challenges that arise in district efforts to become "data- informed."

See some of Dr. Jeffrey Wayman's publications for further information.

LEA Breakout with Jeffrey C. Wayman
Monday, February 16, 2009

After his Opening Session presentation and a round of visits with all three standing committees, Dr. Jeff Wayman led a follow-up discussion with LEAs to delve deeper into his research on data use for school improvement and to take questions from district representatives.

Dr. Wayman began by discussing some interesting practices he has seen in districts over the years. He told the story of a principal who set up a system of "non-threatening triangulation," an approach to data-driven decisionmaking conducted using three data points, one of which had to be teacher judgment. In another instance, a district used triangulation to determine eligibility for its gifted program, using multiple measures and student histories to decide which students were chosen for the program, rather than relying on a single data point to place students. And another school's teachers posted their data use goals on their classroom doors to increase transparency and focus their data use efforts.

The discussion then shifted to the concept of calibration, which is an open process in which a broad range of stakeholders convene to conduct a needs assessment and create a common vision for school or district improvement. In convening calibration discussions, Dr. Wayman suggested that ground rules be established early on to guide the process in an orderly fashion. Participants should define their problem and then figure out how to go about solving it. Re-calibration should take place on an annual basis to reflect progress and align with shifting visions.

Dr. Wayman was then asked to review his findings from a one-year-later perspective on the Wyoming district he and his colleagues studied to assess its data use practices and data system environment. He reported that it took six months before momentum started to build behind his team's recommendations. A new superintendent set specific, measurable achievement goals, an amended calibration process was undertaken, and a new SIS was purchased (despite the recommendation to start with a data warehouse before assessing the other systems). However, though progress had been made, Dr. Wayman felt that administrators were not truly invested in the process.

In response to another question, Dr. Wayman said that a data initiative can modify the way people work (e.g., teacher pedagogy), but it should not completely change the way they do business. A shift in culture in terms of the way people think about and use data is necessary, and data use should be built into the daily routine to inform work. Dr. Wayman also said that in many cases, especially in districts where paper data use and analysis are common, an electronic data system can dramatically reduce burden and increase efficiency.

SEA Breakout on Privacy and FERPA Guidelines
Monday, February 16, 2009

LeRoy Rooker's recent departure from the Family Privacy Compliance Office (FPCO), the USED office that implements FERPA, and his withdrawal from the Forum Meeting left a void in the schedule. To fill this need, Helene Bettencourt (Massachusetts Department of Elementary and Secondary Education) and Bruce Dacey (Delaware Department of Education) moderated an SEA breakout session about privacy issues and the new FERPA guidance, which went into effect on January 8, 2009. In preparation for the session, a request for questions about FERPA was sent out to Forum members. The moderators presented a list of these questions and spent much of the session refining them and developing new questions to submit to FPCO. Questions they produced included:

  • Were there changes to the regulations that will impact our manner of providing data to researchers that requires masking of small numbers?
  • Do the regulations allow additional sharing of individual student records between LEAs?
  • Must SEAs and LEAs still take into account the ability for requestors outside the LEAs and SEAs to unmask data with multiple data requests?
  • Under what circumstances can a state release data to a school that no longer has a student? For example, under what circumstances can the state release Fall 6th Grade state assessment results for a student to a K-5 school in a different district? If the state needs to have legislation rules that grant authority for this, what does that authority have to look like? Can you provide examples or guidance?
  • Do you have guidelines for best practices in regards to recordation?
  • Can you provide examples where disclosures are allowed to non-education state auditors?
  • Please provide examples of legitimate educational interests, and non-legitimate educational interests.
  • Do journalists have a legitimate reason for student information?

These questions will be refined before submission to FPCO. The plan is to use them as talking points in a session with a FPCO representative at the Summer 2009 Forum Meeting.

Joint Session

Tuesday, February 17, 2009

PK-20 Data Sharing: Lessons Learned Panel Zip File  (748 KB)
Hans L’Orange (State Higher Education Executive Officers (SHEEO)), Michael Sessa (Postsecondary Electronic Standards Council (PESC)), and Nancy Smith (Data Quality Campaign (DQC)) presented to the Forum on PK-20 data sharing.

Dr. L'Orange began by introducing the work of SHEEO, which serves its members (statewide coordinating and governing boards) in developing and sustaining excellent systems of higher education. From a "30,000 ft." policy level perspective, he discussed the importance of aligning the PK-12 and higher education data systems, asserting that while P-12 and higher education systems should remain independent, they must work together to increase student success.

  • Dr. L'Orange reviewed several reasons why the postsecondary level is interested in sharing data with P-12, as well as some of the questions alignment can help address.
  • Some of the barriers to PK-20 alignment were reviewed, such as lack of system coordination, lack of a common identifier and resources, and FERPA interpretations. (Here he offered the observation that while higher education often sees the law as a simple a procedural concern, PK-12 seems more concerned about protecting the privacy of its younger students.) To overcome these and other challenges, Dr. L'Orange offered several steps to alignment including identifying shared benefits, reconciling technical difficulties, assuring student privacy, designing a usable system, and planning for sustainability.
  • A number of approaches that have been taken to share data between PK-12 and higher education were reviewed, both technical (e.g., requiring a unique number on transcripts, e-transcripts, and feedback reports) and political (e.g., P-20 councils and changes to state laws).
  • The status of postsecondary state-level unit record databases was discussed. While these databases are operational in 42 states, only 11 of these are linked to PK-12 databases.
  • A number of challenges facing higher education in its P-20 efforts were listed (e.g., data quality, lack of analytical capacity, and reporting of non-credit activity).
  • Fifteen suggested characteristics of a responsive state postsecondary data system were reviewed, covering key student and course data characteristics, as well as operational and data governance characteristics.

Michael Sessa (PESC) introduced his organization and provided the "under the microscope" view of PK-20 data sharing, focusing on technical standards issues.

  • He reviewed why PK-12 and higher education have historically been and continue to be compartmentalized (e.g., differences in funding, oversight and governance, data and systems providers/vendors, policies, etc.). This compartmentalization has hampered efforts to evaluate student progress through the entire education pipeline. As a consequence of these issues, the need for longitudinal data systems has grown and increasing emphasis has been put on establishing national, technical data standards.
  • Mr. Sessa asserted that business interoperability and technology are equally important in the PK-20 alignment effort.
  • Too often, data enter a system through manual data entry rather than automated data exchange between systems. A simple example (the coding of Mary Smith's name) was provided to illustrate the very basic technical issues that continue to impede data sharing efforts. While the state may mandate certain formats to facilitate intrastate data sharing, national standards have not been established. Mr. Sessa noted that a lack of such standards increases costs, data entry and programming burden, and response time, and can threaten data quality.
  • Electronic data sharing has increased in recent years in terms of e-transcript sharing and online Free Application for Federal Student Aid (FAFSA) processing.
  • The "hub and spoke" model of data sharing was reviewed and promoted, contrasting it with the "peer-to-peer" model. The latter requires the development of multiple data matching processes, while the former offers "plug and play" usability. The hub and spoke model, which is also used in the airline and banking industries, has been used to integrate PK-12 and postsecondary educational institutions, both public and private, in many states. Mr. Sessa argued for the development of a national hub or network that would allow interstate data sharing via a national set of technical standards.
  • In closing, Mr. Sessa discussed the collaboration between PESC and the Schools Interoperability Framework Association (SIFA) to establish interoperability among educational software systems via a common set of objects/core components.

Nancy J. Smith of the DQC updated the Forum on the DQC's new survey results, focusing on progress towards P-20 alignment (#9 of the DQC's 10 Essential Elements).

  • As of last year's survey, 28 SEAs reported having the ability to share data with postsecondary (up from just 12 in 2005), and 17 more say they have plans to develop this capability in the future.
  • Barriers to alignment include lack of resources, lack of a common identifiers or crosswalks, lack of coordination, incompatible systems, and FERPA. Ms. Smith also noted other obstacles, including interpersonal matters like "turf issues," a lack of trust that may arise at the operational level between or within organizations, technical concerns such as the lack of a common definition for the term "higher education" across the states (e.g., anything after 12th grade? community colleges?), and data nuances, such as conflicting definitions for a term like "retention" (a good thing in higher education but a bad thing in P-12).
  • Ms. Smith stressed that policy discussions on data use must include the input of data staff who know the technical issues such as capacity.
  • Sustainability of these data sharing systems has been an ongoing issue, and while Ms. Smith noted the importance of the continued and growing support from the federal government in developing P-20 longitudinal data systems (e.g., IES SLDS grants), she warned that states should not rely solely on Washington to fund their systems.
  • To wrap up, several P-20 action steps were offered to guide efforts to achieve PK-20 alignment. The DQC continues to spread its message to a broad audience including policymakers. For more information on P-20 and the DQC, visit www.DataQualityCampaign.org.

National Education Statistics Agenda Committee (NESAC) Standing Committee

Monday, February 16, 2009

Morning Session

Welcome, Introductions, and Agenda Review
NESAC Chair, Helene Bettencourt (Massachusetts Department of Education), called the committee to order, welcomed everyone to Seattle, and reviewed the agenda. The committee went around the table for introductions, each sharing a piece of “positive news.” While reviewing the proceedings from the Summer Forum meeting, a few ideas were identified as potential topics of discussion during the meeting, including discipline data and mobility data.

Attendance Task Force Update
Bill Smith (Sioux Falls School District, SD) shared an update with the committee on the Attendance Task Force. A PDF version of Every School Day Counts: The Forum Guide to Collecting and Using Attendance Data is now available online and the brochure was distributed during the meeting; the print document will be available shortly. Bill recognized the members of the task force and thanked them for their hard work.

The task force focused on several major areas, weaving them into a four chapter publication. The first task was to make the case for the importance of accurate attendance data, and illustrate how high quality attendance data can truly make a difference in schools. The publication presents a taxonomy of data elements and defines the differences between the terms “present” and “absent.” The document includes several case studies and examples of the benefits of good attendance data with an aim of addressing commonly arising problems at the district and state levels.

SLDS Grant Update
Tate Gould (NCES) gave the committee a quick update of the IES Statewide Longitudinal Data Systems (SLDS) grant program. The program will add to its current ranks with a third round of grantees expected to be announced by the end of the month. To date, 27 states have been awarded federal funds to begin developing or enhancing their longitudinal data system. The amount of money awarded in the third round will be largely dependent on the final passage of federal FY09 appropriations.

Also, the American Recovery and Reinvestment Act of 2009 includes an additional $250 million for the further development of these systems. The exact process for allocating this money had yet to be determined, but it can be assumed that there will be an additional competitive grant process under the auspices of IES/NCES. The money allocated in the stimulus package will be coupled with the requirements outlined in the America Competes Act with an emphasis on P-20 linkages.

With the growth of the IES SLDS program, Tate intends to broaden the scope and create a more transparent program. Lessons learned, knowledge sharing, and usable products will be shared with the wider LDS audience. The hard work of the grantee states will be leveraged to assist and inform states that have not received federal funding.

Opening Session Discussion (Jeffrey Wayman: School Improvement, Collaboration, and Professional Development)
The Forum's opening session included a lively presentation by University of Texas professor, Jeffery Wayman. A well known researcher in the field education data collection and use, Dr. Wayman stressed the importance of the data-driven, or "data-informed," district, using collaboration and calibration in all areas of data use, and making sure all district staff receive professional development in the proper use and utility of data. Dr. Wayman joined the committee for a follow-up conversation and more in-depth discussion.

Using a question and answer format, the committee delved into issues surrounding SEA level initiatives, collaboration, how practice should inform policy, professional development, and training in higher education.

  • When asked what role the state agencies play in the data-driven district, Dr. Wayman concluded that SEA should play a very supportive role. The state should facilitate district collaboration and help them navigate state policy to best serve their students' needs. Dr. Wayman was uncertain what the best model for this support was, but stressed the need for the state to broker relationships and regional collaboration.
  • Dr. Wayman stressed that collaboration involves everyone and being as inclusive as possible. Collaboration should extend both vertically and horizontally within an organization: between and across grades, subjects, and staff roles. Data are an excellent way to start and foster this collaboration. Data use needs to occur at every level and be supported from top to bottom.
  • Policy ends in the principal's office and does not always resemble what occurs in education. If data use is going to be integrated into the day-to-day life of a teacher and a school, the principal must be on board with that culture.
  • When asked about the statewide data warehouse, Dr. Wayman had a number of concerns. While state-level data warehouses can cut costs and increase efficiency, it is almost impossible for a statewide data warehouse to meet the needs of every district. District-level data that are not used for compliance (e.g., formative instructional data) sometimes are treated less formally and viewed as convenient rather than necessary.
  • The credentialing process that occurs in higher education and the professional development process in districts and/or schools are essential for creating a data-driven district. All staff should be fluent in data use; all staff should understand the data they need to use on a day-to-day basis. As such, there needs to be a minimum level of data competence for teachers and administrations graduating from a state's higher education system.
  • The importance of professional development cannot be overlooked. Professional development should be focused on a single purpose and on seeing how data use can contribute to that purpose. Too often, districts conduct professional development in isolation. If we continue down the path of having to recreate the professional development wheel each year and in each district, we will never catch up.
  • Asked to illustrate and contrast "shallow" and "deep" data use, Dr. Wayman said that while shallow data use might rely on scattered assessment data (e.g., to identify struggling kids or to address general subject deficiencies), deep data use may use student histories and regular, detailed assessments to identify specific problems that individual students are having (e.g., Johnny adds both the numerator and the denominator) for individual support. Dr. Wayman encouraged people to think outside the box in terms of how to use data and who the data can be used by.

Afternoon Session

Interstate Compact on Educating Military Children
Tom Ogle (Missouri Department of Elementary and Secondary Education) shared information about the Interstate Compact on Educating Military Children. The Council of State Governments began this project in response to problems encountered by the highly mobile population of children of military parents. The average child of military personnel will change schools around nine times, often with little warning and across state lines. The compact allows this special population of students to waive some of the requirements usually required of a transferring student. For example:

  • The student will have up to 30 days to meet the immunization standards for the state;
  • The student will automatically be allowed to continue in the grade from which he or she was enrolled previously;
  • A student may be enrolled by a legal guardian designated with power of attorney;
  • Any tuition assigned by the new school will be waived;
  • The student will be allowed to continue in any extracurricular activities from the previous school (the exact details of this provision have yet to be worked out);
  • The course placement of the student will be honored to the best of the incoming school's ability;
  • Senior transfer students will be allowed to waive any graduation and exit exam requirements and graduate on time with his or her designated cohort.

The compact became active with the passage of legislation in ten states. In establishing the compact, these states will be required to set up a Council with in the state and attend national meetings. Concerns about this legislation include the lack of specificity, the possibility of different data standards and/or needed elements, and a duplication of effort with such projects as MSIX.

E-transcripts Discussion
Derek Howard (Utah Department of Education) joined the committee to share and lead a discussion about what Utah is doing with e-transcripts. Currently, Utah is working on their Utah eTranscript and Record Exchange (UTREx) system of transcripts and records exchange. The project started in late 2007 and should be finished and operational in a year. The system will use a statewide data warehouse of transcript level data to allow sharing across the state and to send e-transcripts to higher education institutions both in and out of the state. Utah is using the DigitalSAMS, Student Achievement Management System, for the state's data warehouse. DigitalSAMS will also be made available to any district that would like it; an organization that declines the DigitalSAMS will be able to access the state data warehouse with limited functionality through SIF capabilities. The system will house student-level data by state, including name, address, demographics, test scores, classes, etc. Utah is building a student interface that will allow graduating high school seniors to send their transcripts to the college/university of their choice. Currently, information is not flowing back to the K-12 system from higher education, but Utah is working on that in the next phase.

National Education Data Model
In 2008, the Forum's PK-12 Data Model Task Force finished its work on the Education Data Model Version 1 (PK-12). At the completion of this project, Forum members expressed an interest in seeing this work continued through additional versions and capabilities. The Office of Education Technology, USED, will further develop this work through the Council of Chief State School Officers. Ty Mapp (CCSSO) and Beth Young (QIP) joined the committee to give an update on the progress of that work. A comprehensive team, including CCSSO, QIP, SIFA, and Kforce, has begun work on future versions of what is now called the National Education Data Model (NEDM). The team will work with an Advisory Group from local, state, federal, association, and vendor organizations, as well as a PK-20 steering committee. NESAC asked the speaker to consider how PK will be involved in the development of the NEDM.

Data Quality Curriculum
In 2004, the Forum released The Forum Guide to Building a Culture of Quality Data: A School and District Resource. From that effort, the Forum initiated the development of the Forum Curriculum for Improving Education Data: A Resource for Local Education Agencies (2007) and a series of online courses hosted by the SIF Association on SIFA University. Michael Derman (Central Susquehanna Intermediate Unit, PA) was instrumental in the development of the curriculum and its online course. The intent of these resources was to improve the quality data in the local education agencies. The publication has two main audiences: "all staff" (part I) and "data stewards/coordinators" (part II). Each part addresses the issues of accuracy, timeliness, security, and quality for both the broader audience and the more focused data collector/user.

The Central Susquehanna Intermediate Unit has been selected by the Pennsylvania Department of Education to work with their SLDS project on issues of data quality at the LEA level. Pennsylvania, through its intermediate units, will implement data analysis and data quality trainings in the LEAs. The 29 intermediate units will utilize the Forum's data quality curriculum in a train-the-trainer model for LEAs. In Michigan, a similar session was held for 200 district level data collectors. The curriculum is highly adaptable for a state or district's needs and can be delivered in a variety of ways based on audience need.

To date, use of the online tool has been sparse. Feedback from the committee included questions about why SIF-U is hosting the tool.  Moreover, SIF email is being sent to people who register at SIFA-U to use the tool. Despite this concern, feedback from those who have used the tools for district-wide trainings has been overwhelmingly positive. The curriculum is well thought out, easy to use, and very helpful. The committee suggests rethinking the online portion of this tool, particularly removing any registration step and indicating an estimated time commitment at the outset.

Tuesday, February 17, 2009

Morning Session

Joint Session Review and Discussion: P-20 Data Sharing
Tuesday morning's joint session featured presentations from Hans L'Orange (SHEEO), Michael Sessa (PESC), and Nancy Smith (DQC). Following the session, Nancy Smith and Aimee Guidera of the DQC joined the committee to further discuss issues, challenges, and opportunities involved in P-20 data sharing. The mission of the DQC is to inform policymakers about the utility of quality education data and create a ground swell of political will to support the building and maintenance of these systems. For the past three years, the DQC has been pushing its mission through the "10 Essential Elements," an estimation of what state data systems need to include in order to be robust and effective. In the next phase of the DQC, the focus will shift from data capabilities to data use.

Sharing data across the spectrum from P-20 presents a number of issues that policy and practice need to address in order to reach the full utility of longitudinal data systems.

  • The postsecondary system is disjointed and presents difficulties for K-12 systems trying to glean useful information from it.
  • Course codes and grade point averages rarely have standard definitions, making it hard to compare these data across schools, districts, or states.
  • PK and early childhood data elements generally lack standards as well and are rarely collected by the K-12 system.
  • Data on teacher preparation and professional development isn't usually available to the K-12 system in a meaningful way that permits program evaluation.

The DQC sees itself as a spotlight that shines the light on these issues. Since the start of the DQC survey, six states have reported having all of the 10 Essential Elements, and nearly all states have made tremendous progress in implementing a robust LDS.

FERPA Discussion
In December, the U.S. Department of Education released final changes to its FERPA regulations. The new regulations address a number of the issues that states had concerning FERPA and the development and use of longitudinal data systems. Additional discussions about FERPA will be included at the Summer Forum meeting when a representative from USED can join the committee to discuss the new regulations in detail.

Implementing the new Race and Ethnicity codes—Where are states now?
Since the passage of the new codes for the collection of racial and ethnic data, there has been a great deal of discussion about how the change will affect states and districts. The Forum released Managing an Identity Crisis: The Forum Guide to Implementing New Federal Race and Ethnicity Categories in the summer of 2008. It includes significant guidance on the new codes and how to implement them prior to mandatory implementation in the 2010-11 school year. LEA committee members reported that they are still not receiving much guidance from their SEAs. For example, there is still uncertainty about how the switch will affect AYP reporting.

Publications Review
The committee reviewed recent, and older, publications released by the Forum. The Safety in Numbers publication developed by NESAC in 2002 will be reviewed and possibly updated by the U.S. Department of Education. This effort would include clarification on issues resulting from the Uniform Management Information and Reporting System (UMIRS).

Afternoon Session

EIMAC CRDC Workgroup Update
Linda Rocks (Bossier Parish Schools) gave the committee a quick update on the Civil Rights Data Collection Workgroup. The workgroup was formed in April 2008 by the Education Information Management Advisory Consortium (EIMAC) to give the U.S. Department of Education suggestions for improving the collection. The initial plan for the CRDC is to use EDFacts data to prepopulate certain sections of the collection. The workgroup provided feedback on a number of issues, including the instructions for filling out the survey, questions regarding certain data elements, communication, and the website. The CRDC has been put on hold until passage of FY 2009 appropriations.  

Boston's Data Warehouse Project
Arleen Thompson (Boston Public Schools) offered a presentation about Boston's data warehouse project. Massachusetts bought a statewide license for data warehouses through Cognos, but has been slow in rolling it out to districts—and larger districts such as Boston Public Schools (BPS) have continued to advance with their own projects. In the spring of 2007 BPS was awarded a state grant to help with their efforts. Access to the BPS systems is currently at the school level and does not yet reach the teacher/classroom. Boston's first priority for this project was to look closely at the attendance data. Attendance was seen as an essential first step because it is used for compliance and for evaluating AYP. The data collected in the report is very useful at the school level. BPS has faced a number of challenges with this project including:

  • staffing and resources, including hiring a good contractor from Cognos;
  • keeping the team united and on target;
  • ensuring senior management that the project is progressing;
  • fine tuning and response time;
  • selecting a well-balanced, functional review team;
  • security issues; and
  • staying on schedule.

The next steps for the project include putting together a school-based review, issuing technical support to teachers, reviewing the state scheme for assessments and entering them into the system, and doing this all on a possibly slim budget due to severe cuts.

State Education Data Center Update
Maureen Matthews (CCSSO) provided an update on the State Education Data Center (SEDC), a project of the Council of Chief State School Officers to make education data readily available through the website www.SchoolDataDirect.org. In September of last year, the U.S. Department of Education issued a contract to CCSSO to support and enhance the utility of the SEDC. There are five major deliverables for this contract:

  1. To design, install, and improve the link between EDFacts and schooldatadirect.org, and have the file sharing in place with data suppression through the N size in the accountability workbook;
  2. To improve the data download tool so users are able to customize their data download;
  3. To create a Technical Advisory Panel to assist with web development;
  4. To put in place a Help Desk Manager to assist with the day-to-day issues involving the website; and
  5. To create the National Education Data Model, and to map the SEDC to the data model, the Common Data Ask (CDA), and the NCES Handbooks.

EDFacts Update and Technical Amendments
Ross Santy (USED) joined the committee to provide a few updates on EDFacts and the EDEN submission system. There have been a few modifications to the EDEN data set. In particular, changes have been made to the optional data groups for 08-09 and 09-10 reporting year:

  • The Department added a data group to collect the adjusted cohort graduation rate as specified in Title I regulation; the Department will also collect any optional extended cohort rates that are collected by the state.
  • The Department will introduce differentiated accountability pilots—state specific definitions for schools flagged for needing improvement. This is not being done yet, but the Department is taking a look at assessment data and how to collect IDEA assessment data and NCLB assessments in one file spec.
  • The Department is reconsidering the 2004 policy on assessment participation and students with medical emergencies, and will possibly modify the file specification for assessment participation information to add this variable.
  • The Department restored a field it had removed regarding Limited English Proficiency; they had removed the field to take account of students taking LEP assessment for the first time – that field will be replaced.

All of this information can also be found on the EDFacts Communities site on the CCSSO website.

American Recovery and Reinvestment Act
In the last weeks and months, there has been growing interest in the American Recovery and Reinvestment Act. At the time of this meeting, the bill had just been signed into law. Exact allocations and rules regarding the stimulus money had not been made clear. The committee discussed how to access information about this initiative and possible strategies for using this money. It is clear that money will be allocated in three different ways:

  1. Allocations will be made for existing formula grant programs (Title I, IDEA, etc); this money will need to be moved through the Department quickly, perhaps in as little as 30 days.
  2. The state fiscal stabilization fund will be administered by the USED. State governors will apply for this money.
  3. Additional money is being allocated to discretionary programs, including $250 million for the IES SLDS grant program. This money will also have to be distributed quickly, possibly in as little as 90 days.

Meeting Review and Ideas for Summer
The meeting concluded with a review and discussion of what should possibly be on the agenda for the summer meeting. The committee expressed an interest in discussing more PK data issues (including interfacing with K-12), teacher professional development data, mobility rates, and a follow up on the stimulus package.

Policies, Programs and Implementation (PPI) Standing Committee

Monday, February 16, 2009

Morning Session

Introductions and Agenda Review
PPI Chair James Haessly (Waukesha School District, WI) called the PPI committee to order, and asked for members to talk about one aspect of their job. Jim also reviewed the Winter Meeting agenda.

Crisis Data Management Task Force Update
Linda Rocks (Bossier Parish School System, LA) provided an update on behalf of the Crisis Data Management Task Force. The group has met six times since the Summer 2007 Forum. The group's product will focus on the data items and management issues surrounding the displacement of students due to a crisis. Linda led a discussion on the topic and fielded questions. PPI members asked about the field length of the student indicator, whether the indicator could be used to track military movement, the importance of the indicator to help track any possible behavior problems, etc. The task force plans to have a draft for review in the next few months with possible approval by July.

EIMAC Civil Rights Data Collection (CRDC) Workgroup Update
Jim Parsons (Richardson Independent School District, TX) gave an update on the EIMAC Civil Rights Data Collection (CRDC) Workgroup. The workgroup consists of LEA and SEA members who provide the USED with suggestions for the collection. Jim said the workgroup has made many suggestions, specifically related to improved documentation. The Office of Civil Rights has responded positively and made changes based on their comments. The Workgroup will be meeting again during the conference.

PPI Discussion: SEAs Reporting Data for LEAs
PPI members broke into LEA and SEA groups to discuss the pros and cons of SEAs reporting data for LEAs.  These conversations were in response to the recent work being done by the CRDC (see above).

Afternoon Session

Opening Session Discussion (Jeffrey Wayman: Implementation of Data Driven Decision Making Through Professional Learning Communities)
Following up on his Opening Session presentation, University of Texas at Austin's Dr. Jeffrey Wayman joined PPI to discuss a number of data use issues related to his work. The session took a question and answer format.

  • Professional Learning Communities (PLCs) were discussed in great depth. This is the education community's version of communities of practice in which educators collaborate to improve their work. PLCs may be established through a mandate, but policymakers should not dictate the details of how these communities work. These groups are sustained through ongoing support and guidance, and leaders of PLCs can receive training on effective leadership of such groups. PLCs should tackle one small issue at a time (e.g., improving a teacher's strategy for teaching a specific student math). PLC members should use data to identify which teachers' students are performing well in an area and draw lessons from their colleagues' effective practices.
  • The process of "calibration" should involve an early dialogue in which goals are clearly defined and agreed upon by the range of stakeholders. This is an important step to ensure that all stakeholders are on the same page (whatever page it is that they agree to be on) in a data use improvement effort. It also helps to create a broad sense of ownership.
  • With the support of the Spencer Foundation, Dr. Wayman will soon begin research on the implementation and effects of the Data-Informed District (DID) model in three districts. Similar to his study in Casper, Wyoming, Dr. Wayman will study the districts' data use practices and data system environments, make recommendations for data use improvements, help implement those recommendations, and find out what happens when the DID model is implemented.
  • Professional development on data use for both teachers and administrators is critical to the success of a data initiative. Without it, there will only be local pockets of success. The combination of a prepared administrator with prepared teachers is a recipe for successful data use. On May 10, three of Dr. Wayman's students will publicly defend their dissertations related to training administrators and teachers about data use.
  • A collaborative data use culture is established first by planning the structure. Then, a dialogue and culture shift should follow. Teachers should collaborate around instructional improvement and lesson plans. Policymakers should mandate the what (e.g., PLCs), but not the details of how (e.g., topics for PLC dialogue).

National Education Data Model
In 2008, the Forum's PK-12 Data Model Task Force finished its work on the Education Data Model Version 1 (PK-12). At the completion of this project, Forum members expressed an interest in seeing this work continued through additional versions and capabilities. The Office of Education Technology, USED, will further develop this work through the Council of Chief State School Officers. Ty Mapp (CCSSO) and Beth Young (QIP) joined the committee to give an update on the progress of that work. A comprehensive team, including CCSSO, QIP, SIFA, and Kforce, has begun work on future versions of what is now called the National Education Data Model (NEDM). The team will work with an Advisory Group from local, state, federal, association, and vendor organizations, as well as a PK-20 steering committee.  PPI members had the following concerns/ suggestions for the speakers:

  • Time and resources are barriers to using the data model.
  • Some more value added tools would help members know how to describe the model to others in their LEA and SEAs.
  • The faster the P-20 integration the better. National best practice is needed.

Tuesday, February 17, 2009

Morning Session

Joint Session Review and Discussion: P-20 Data Sharing with Hans L'Orange
PPI members discussed issues surrounding P-20 data sharing with Hans L'Orange (SHEEO).  They discussed the following P-20 data sharing topics: student privacy, individual versus aggregate information, whether the data is necessary versus just "nice to know", value-added documents for P-20, where to house a P-20 data system, and the impact of scholarships on data transfer.

SLDS Grant Update
Tate Gould (National Center for Education Statistics) reported on the Statewide Longitudinal Data Systems (SLDS) grants. The FY09 applications were reviewed and USED is currently waiting on a budget decision before awarding the grants. The economic stimulus package includes an additional $250 million for state grants. Information on the new grants will be forthcoming; they have a focus on P-20 data sharing. The SLDS program is also working on a website to have all states share in the lessons learned by grantee states. States that have submitted unsuccessful FY09 applications will receive comments before the new grant request for applications is released.

State Education Data Center (SEDC)
Ross Santy (USED) gave a quick update on the State Education Data Center and the two-way link between EDEN and the SEDC. They are currently working on the website's data download capabilities, as well as forming a technical group as an offshoot of their Advisory Group. SEDC is also expanding its help desk and providing a map between the NEDM and SEDC. About half of the data needed for SEDC are already collected through EDEN. PPI members wanted to make sure that there was a cap on the data requested by Standards and Poor.

Afternoon Session

FERPA Publication Discussion
Levette Williams (Georgia Department of Education) led a discussion on previous publications that PPI has produced about FERPA. Because of the issuance of new FERPA regulations, some of the publications need to be reconsidered. The group agreed that both the smaller guide, Forum Guide to the Privacy of Student Information: A Resource for Schools (2006) and the larger product, the Forum Guide to Protecting the Privacy of Student Information: State and Local Education Agencies (2004) should be redone. A working group will be formed to develop a plan for these revisions. The subcommittee discussed a few issues with these revisions.

  • The comprehensive guide should include information on new privacy issues that have arisen since the last releases, including P-20 data sharing, SLDS, HIPPA, recordation, sharing with those outside education, crisis data management, etc.
  • Be careful about trying to do too much with one document.
  • LEAs have the responsibility to guard the student's data.
  • FERPA staff should be part of the group updating the publications.
  • Can we ask the FERPA office to review the current publications?

State Education Data Center (SCED) Codes for Elementary and Middle School Courses
Lee Hoffman (NCES) gave an update of the current SCED codes project. They are thinking about expanding the codes to the elementary and middle school levels and Lee wanted feedback from the group on this idea. The members supported the expansion of the codes down to the elementary and middle school levels. Lee asked for volunteers to sit on a working group for these new codes.

Implementation of New Racial/Ethnic Codes
PPI Chair James Haessly (Waukesha School District, WI) asked the members to discuss how their organizations were implementing the new racial/ethnic codes. Most states said they would be at full implementation in the 2010-11 school year. Some states said they would use both sets of codes the first year of implementation.

Technology (TECH) Standing Committee

Monday, February 16, 2009

Morning Session

Welcome and Introductions
Chair Tom Purwin (Jersey City Public Schools, NJ) called the meeting to order.

Opening Session Discussion (Jeffrey Wayman: School Improvement through Analysis of Data)
Dr. Jeffrey C. Wayman visited TECH following his Opening Session presentation to speak about data use and analysis issues in more depth. The discussion took the form of a question and answer session. Topics included:

  • Discussing his earlier statement that while he sees data use "increasing," it does not seem to be "advancing," Dr. Wayman argued that better professional development was necessary to help educators make the leaps from data to information, and from information to knowledge and changes in practices. Furthermore, he offered a cyclical view of data use in which we start with knowledge, collect data based on that knowledge, extract information from those data, and then use that new information to enhance our knowledge, before we return to the beginning and refine our data collection choices.
  • The status of teacher education on data use is not very encouraging and graduate school programs on data use are not common. Dr. Wayman also stressed the importance of training administrators (e.g., principals and superintendents) on data use so they can guide their schools and districts from the top.
  • Though Dr. Wayman currently has no hard evidence that his Data-Informed District model works, he outlined his future research plans that will implement the model and study its effects on student achievement over the next several years.
  • Sources of scientific research on what programs work were discussed including the Center for Data Driven Reform in Education (CDDRE), the Best Evidence Encyclopedia (BEE), and the What Works Clearinghouse (WWC). It was suggested that educators should look at what researchers are analyzing to inform their own work and local data analysis.
  • Vendors need to be told by states and districts what data to collect. This guidance is far more important in an RFP than detailed technical specifications. Vendors know the technical nuts and bolts, but need to be told what data to collect in order to answer state and local education questions.

Introductions and Agenda Review
Members and attendees introduced themselves and reviewed the TECH agenda and Chair Purwin reviewed the work TECH accomplished at the Summer 2008 Meeting in Bethesda, MD. This included discussions and activities related to FERPA, task orders, task forces, e-transcripts, Section 508 accessibility standards, data environment security, the Migrant Student Records Exchange Initiative, Schools 2.0, longitudinal data systems, and TECH publications.

Introductions and Agenda Review
Patsy Eiland (Alabama Department of Education) was elected TECH Vice-Chair for 2009 to replace Corey Chatis (Tennessee Department of Education), who resigned from the Forum.

Task Force and Working Group Updates
Data Ethics Task Force: Task force chair Tom Purwin (Jersey City Schools, NJ) updated TECH members on the status of the group's work. The task force has completed the document's content and expects that a final product will be available at the Summer 2009 Forum.

Metadata Task Force: Task force chair Tom Ogle (Missouri Department of Elementary and Secondary Education) reported that the document is with the editor. It will be sent for final Forum approval in the next couple of weeks and should be available at the Summer 2009 Forum.

Longitudinal Data Systems Task Force: Task force chair Bruce Dacey (Delaware Department of Education) reported that the group continues to advance a draft of its product. The task force has completed several chapters and hopes to have a final product by the Winter 2010 Forum.

Race/Ethnicity Task Force: Task force chair Bethann Canada (Virginia Department of Education) reported that the task force's final product was released in September 2008 and is available in print (via GPO order) and electronic form.

PK-12 Data Model Task Force: Task force chair Jeff Stowe (Arizona Department of Education) announced the release of the Education Data Model Version 1 (PK-12), which catalogues and describes the relationships among data used in PK-12 education. Version 1 is available online. At the Summer 2008 Forum (and as the project neared completion), Forum members expressed an interest in seeing this work continued through additional versions and capabilities. The Office of Education Technology, USED, will further develop this work through the Council of Chief State School Officers. Ty Mapp (CCSSO) and Beth Young (QIP) joined TECH to share an update on how that work is progressing. A comprehensive team, including CCSSO, QIP, SIFA, and Kforce, has begun work on future versions of what is now called the National Education Data Model (NEDM). The team will be working with an Advisory Group of staff from local, state, federal, association, and vendor organizations, as well as a PK-20 steering committee. TECH was reminded about the SEA focus of the NEDM, and members asked about user tools being developed to show the SEA perspective. It was agreed that since the TECH Committee was the home of the original Data Model task force, the current NEDM group will update the committee on NEDM status at each Forum meeting.

Task Order Update
Chair Kathy Gosa provided TECH with a final update on Kansas' FY 2007 State Cooperative System Task Order Award: Improving the Quality of Data Submitted by Kansas LEAs. The data quality certification program focuses on data coordinators and data entry staff. Its goals are to: 1) help users develop basic skills; 2) share best practices at a grass-roots level; and 3) recognize the hard work done by staff by awarding them with certification. Kansas continually works to expand and improve this program.

Afternoon Session

TECH Professional Development: Data Analysis and Mining Tools Zip File  (239 KB)
Larry Fruth (Schools Interoperability Framework Association) led a team of presenters, including Forum member Glenn McClain (Platte Valley School District, Weld RE-7, CO), to discuss the nomenclature, quality issues, basic tenets, leadership issues, references, and examples of data analysis and mining tools. See the attached PPT.

TECH Discussion: Recordation
Following the presentation, TECH members divided into SEA and LEA breakout groups to compare and contrast data analysis needs, strategies, and trends. LEA members concluded that: SEAs drive a lot of data mining and analysis activities; intermediate agencies can be very helpful in sharing tools and expertise with LEAs; and reconciling records (e.g., transfers) is a role for states. SEA discussions identified numerous related issues, including: it is an SEA's role to provide data analysis and mining tools that "keep a level playing field" within and across states; professional development needs are critical; oversight and governance are also critical; leadership issues and "support from the top of the organization" is necessary but not sufficient for effective data mining and analysis efforts; and SEAs need to prove the utility of such efforts to LEAs.

Additionally, TECH members concluded from this conversation that the committee should facilitate these types of conversations between SEA and LEA representatives. More specifically, SEA members want to hear more about what state agencies can offer to local agencies. This information is necessary if SEAs are going to be able to provide assistance to their counterparts in LEAs.

TECH Publications
TECH will conduct a final review of its previously retired technology publications: Weaving a Secure Web Around Education: A Guide to Technology Standards and Security (2003), Technology in Schools: Suggestions, Tools and Guidelines for Assessing Technology in Elementary and Secondary Education (2002), and Safeguarding Your Technology: Practical Guidelines for Electronic Education Information Security (1998). Together, these publications have been accessed online over 31,000 times in the last 18 months. TECH believes that these are useful resources and that having a small group update them will not take more than a few months of work. Doing so would be a good use of Forum resources and good for the education data community given how often these publications are being accessed. Bethann Canada (Virginia Department of Education) and Tom Purwin (Jersey City Public Schools, NJ) volunteered to contribute to these efforts.

TECH also had several ideas for new task forces. Potential topics included Section 508 Accessibility Standards and data release policies and procedures. TECH decided to proceed with the Section 508 request for now. Lee Rabbitt (Newport Public Schools, RI) volunteered to chair a working group. Additionally, TECH would like the Forum to investigate various methods states and districts use to incorporate AP and SAT results from the College Board. Perhaps it might be possible for the Forum to write a letter requesting changes in the College Board's approach to supplying these test data.

Tuesday, February 17, 2009

Morning Session

Technology Issues in SEAs and LEAs
Patsy Eiland (Alabama Department of Education) facilitated an open discussion about how technology issues affect SEAs and LEAs. Strands of discussion included:

  • The necessity of senior staff commitment – New ways of doing business often present many challenges. In addition to assuring resources, leaders can establish a culture to support data quality and usage activities.
  • Communications – Even email is a difficult issue in some states and districts. States must be careful about how much email is sent to LEAs or else messages might be disregarded when there is too much being sent (i.e., when it begins to feel like spam). Along these same lines, identifying the appropriate point of contact is also important. Sending a data-related message to a superintendent may adhere to the chain of command, but doesn't always ensure that information gets into the right hands at the district level.
  • Change Histories – There is often tension between data quality and consistent reporting. For example, a state may release a report and then be told at a later date that one of its districts needs to change the data it submitted for that report. Does the state re-release the report? Probably not, but the data in the report are not correct, so there is a concern. This simple anecdote occurs quite often and raises questions about how long states permit previously submitted data to be revised. No one wants to publish inaccurate data but, at some time, once a report is released it can be difficult or impossible to retract it.
  • Resource Sharing at the Forum – Most Forum members have tools they have built (or purchased at a relatively low cost) to help deal with everyday issues that arise in the field of education data. Participants asked that TECH collect a list of such tools and share a summary via the listserv so that members can contact each other to exchange/share resources. TECH will attempt to do so this spring.
  • Mega-IT Units in SEAs – Some states have consolidated IT services at the state level. In other words, SEAs lose their IT departments in order to be serviced by a mega-IT unit that serves the entire state government. In theory, efficiencies of scale drive this, but, in practice, mega-IT units rarely can align services with the strategic vision and goals of otherwise independent agencies. This appears to be a growing problem and TECH may wish to keep it on the agenda for discussion.

Data Quality Campaign
Nancy Smith and Aimee Guidera asserted that the 10 Essential Elements collection they publish is not an accountability instrument but, rather, a "conversation starter." Forum members appreciated this distinction but also noted that there may be some concern about the quality of the self-reported data. For example, if an SEA receives one file per year with college enrollment data about the previous year's high school graduates, should they answer "yes" to the question about "Can you exchange data between P-12 and postsecondary education?" Issues surrounding the ten essential elements, longitudinal data systems, e-transcripts, and P-20 data sharing will continue to arise over the next few years, so TECH hopes to advance this dialogue with the DQC.

Afternoon Session

Technology Issues in SEAs and LEAs (Continuation of Morning Discussion)
Forum members don't believe that P-20 data exchange is really a two-way conversation at this point. Higher education simply isn't returning data to P-12, even though a single annual report might be enough to serve P-12's research needs (e.g., Do our students finish college in four years? Do they need remediation?). The connection to labor force statistics/research is also very compelling.

SLDS Update
Tate Gould (National Center for Education Statistics) gave an overview of the IES Statewide Longitudinal Data Systems (SLDS) grant program. FY 2009 awardees have been identified, although they will not formally receive grants until the federal budget is finalized. NCES will attempt to make future SLDS activities more transparent to the education data community. For example, WebEx's will be publicized and open to anyone who wants to participate.

In addition to SLDS funds, the American Recovery and Reinvestment Act includes $250M for improvements to state education data systems. A request for applications has been prepared in the likelihood that this money will be appropriated. It is possible that applications and awards will occur on a short timeline. All states will be eligible to apply for these funds, regardless of whether they have already received an SLDS grant.

Resources and Tools to Use Data to Improve Learning
Tom Purwin facilitated a discussion about data use to improve learning. TECH discussed important aspects of this issue, including: How do you know when teacher professional development has worked? What metrics can be used? How do data contribute to school improvement? In the past, district administrators might have assessed effectiveness through data such as "I trained 100 teachers" without any measure of how those teachers changed or how learning was improved. There are a lot of variables to consider in this type of analysis, but this is a direction for future research and consideration.

Following the presentation, TECH members broke into SEA and LEA groups to discuss these issues from their different perspectives. SEA members acknowledged that there is a large gap between the utility of the administrative and instructional data that they collect. While it is possible that statisticians could help teachers use data more effectively, this is not the situation in most education settings. Currently, we are much better at collecting administrative data for compliance and resource management purposes. LEA members noted that program data are rarely meaningful to LEA or school staff (i.e., that they aren't particularly useful for instructional decisionmaking).

Lessons Learned
Patsy Eiland (Alabama Department of Education) led a discussion about "lessons learned" by TECH members that might be transferable to colleagues in other states and districts. For example, it is unwise for data staff to assume that vendors understand their needs. Unless RFAs are clear and explicit, vendors rarely meet unspoken expectations. TECH will try to exchange RFA language as a part of its new "sharing resources/tools" initiative.

Meeting Review/Summer 2009 Planning

  • FERPA guidance – TECH would like to hear from an authority on FERPA this summer. Given recent changes to FERPA, members really need guidance from a voice of authority at ED.
  • Data release policies and procedures – Kathy Gosa (Kansas Department of Education), Bethann Canada (Virginia Department of Education), Bertha Doar (Rockwood School District, MO), and Laurel Krsek (Napa Valley Unified School District, CA) agreed to prepare a professional development session on "data release policies and procedures" for the Summer 2009 TECH meeting.
  • Outsourcing data responsibilities – More and more data responsibilities are being outsourced beyond U.S. borders. TECH members were uncertain about potential liabilities associated with this practice and would like to hear more about the topic.
  • SIF implementation at the message level – TECH would like to learn more about the nuts and bolts of using SIF agents at the message level rather than at the more broad "data model" level as is usually discussed.
  • Data usage – The Forum does a lot of work related to data collection, synthesis, analysis, and reporting, but very little about data usage. Do we need to consider how people use data to improve student achievement? We know we use data to develop policies, but how are data used to improve teaching and encourage learning? Admittedly, this extends beyond the Forum's traditional focus on data system development, and it appears that there may be a limited foundation of knowledge to inform these discussions.
  • SLDS progress – Can TECH get an update on what, precisely, has been achieved by SLDS grant recipients? This is not an accountability concern; we are truly interested in hearing details about the good things that have occurred as a result of the grants and SEA focus on these projects.

Closing
Chair Tom Purwin thanked the TECH Committee for its hard work at the meeting.

Closing Session
Tuesday, February 17, 2009

Overview and Agenda Zip File  (41 KB)
NCES Update
Stuart Kerachsky, Acting Commissioner of the National Center for Education Statistics (NCES), updated Forum members on recent business and coming plans at NCES and the Institute of Education Sciences (IES).

  • Commissioner Kerachsky began his remarks by highlighting the promise of the new "USED regime" led by Secretary Arne Duncan, an active data user. He predicted that under this new leadership, data-driven decisionmaking would make its way to the forefront of the national education dialogue, the federal government would continue to support state work to establish PK-20 data systems, and NCES would advance its mission.
  • The National Assessment of Educational Progress (NAEP) will include new science assessments, including a hands-on section. An online administration system is also under development. Several new NAEP tools will be available soon, including the NAEP Explorer, question tools, a mapping instrument, a state comparison tool, and state profiles. In general, the focus is on improving access to NAEP data.
  • The Schools and Staffing Survey (SASS) survey will include a new questionnaire about the status of principals (e.g., are they still at the school, transferred, or retired).
  • The Early Childhood Longitudinal Study, Kindergarten Class of 2010-11 (ECLS-K:11) has improved its methodology and now includes measures of developmental gains. School administrators will be contacted soon for a field test and the study will start in the fall of 2010.
  • The results of the 2007 Trends in International Mathematics and Science Study (TIMSS) are now available. The US ranked sixth in 8th grade math, and generally trails behind developing Asian countries and, in some assessments, some European countries. Massachusetts and Minnesota paid for their own state-level analyses and both rank near the top of the lists. The Program for International Student Assessment (PISA) measures skills more holistically than other tests. States should contact NCES if they would like to be analyzed at the state level on TIMSS or PISA.
  • The third round of the IES Statewide Longitudinal Data Systems (SLDS) grants will be announced soon. Additionally, the American Recovery and Reinvestment Act, signed on the day of Dr. Kerachsky's update, includes $250 million for the SLDS grant program. A new grant competition should be announced soon, and the time frame for writing proposals will be shorter than usual. Grants should be awarded within several months. The focus will be on improving education rather than on accountability, and on linking K-12 systems with early childhood, postsecondary, and workforce information. To qualify for the grants, states will have to demonstrate progress towards the elements in the America Competes Act, which deal with collecting and using longitudinal data. The grant proposals should emphasize how SLDSs will help teachers teach, policymakers improve policy, and, ultimately, give students an edge.
  • Several other NCES projects were also mentioned:

In closing, Commissioner Kerachsky asserted that end users should be involved in data system design and said that the federal government and the states can and should do a better job of using data to improve education.

Task Force/Working Group Progress Reports

Standing Committee Progress Reports

Meeting Evaluations
Forum members shared their opinions on the Winter 2009 Forum Meeting by completing evaluations.

Steering Committee

NCES Winter Forum 2009

Monday, February 16, 2009

Welcome and Opening Comments
Chair Bruce Dacey (Delaware Department of Education) opened the Steering Committee meeting and welcomed participants to the 2009 Winter Forum Meeting.

Orientation Session
Steering Committee members felt that the morning Orientation Session went very well. Members agreed that the sooner new members get involved, the better; and joining a task force is a great way to learn how the Forum really works. The mentoring program seems to be working as well. Getting new members up-to-speed quickly is important. We are doing a much better job of this than we did five years ago, but that doesn't mean we can afford to change our focus on this important ongoing task. There will be a full orientation at the Summer 2009 Forum.

Opening Session
The Steering Committee was happy with the Opening Session presentation by Dr. Jeffrey Wayman. He managed to stay out of the "ivory tower" from which he came. Having said that, Dr. Wayman may have underestimated the expertise of his Forum audience. We will need to see how the post-meeting evaluations assess the presentation.

Standing Committee Time
All three standing committees had a productive day (see NESAC, PPI, and TECH notes). The following issues from these sessions were raised at the Steering Committee meeting:

NESAC: NESAC members were concerned about the Forum Curriculum for Improving Education Data being maintained at the Schools Interoperability Framework Association (SIFA) website. Although everyone agreed that SIFA was a valued Forum partner, people believed that personal data submitted when registering for the online course was being used to send unsolicited email about SIFA activities. There were also questions about whether it made sense for a Forum product to exist somewhere other than the Forum or NCES website. The Steering Committee agreed to: 1) request usage statistics for the curriculum to see whether the SIFA site is a better "marketing" location than the Forum site; and 2) ask that SIFA not use curriculum registration data to populate electronic mailing lists.

PPI: PPI members asked about NCES plans for managing cooperative system contracts. Some state representatives noted that they wouldn't be able to attend the Forum if NCES returns to the old task order funding mechanism. NCES said that states just need to let NCES and/or Coffey Consulting (the logistics contractor) know what their needs are with respect to participation support.

SEA/LEA Breakout Time
SEA representatives met during breakout time to discuss FERPA guidelines. While this session had good potential, two problems arose: 1) USED's FERPA representative was not able to attend the meeting, and the absence of an "authoritative" voice severely limited the value of the discussion; and 2) the room was too big to be conducive to this type of discussion, which detracted from the effectiveness of the session.

LEA representatives met with Jeffrey Wayman. Steering Committee members believed that the session went very well.

Tuesday, February 17, 2009

Standing Committee Time
TECH chair, Tom Purwin, reported that TECH established a Section 508 Accessibility working group. NESAC and PPI were encouraged to contribute members to the group.

PPI believes that recent changes to FERPA necessitate updates to the Forum Guide to the Privacy of Student Information: A Resource for Schools (2006) and the Forum Guide to Protecting the Privacy of Student Information: State and Local Education Agencies (2004). PPI will establish a working group to deal with this important issue. PPI also received a request from NCES to contribute to the development of a course codes standardization project for elementary and middle schools (parallel to the Secondary School Course Classification System: School Codes for the Exchange of Data (SCED) (2007). PPI will work with Lee Hoffman at NCES to support this effort as requested.

NESAC and TECH both raised concerns about the quality of self-reported SEA data to the Data Quality Campaign.

PK-20 Data Sharing
The Steering Committee felt that the panel went fairly well, although people wondered how much new information was shared. There was a general sense that very little progress is being made on this front at the national level.

Closing Session
The Steering Committee felt that NCES Acting Commissioner Stuart Kerachsky made a very strong showing at his first Forum. His participation throughout the meeting was very much appreciated and he gave a good update of NCES activities at the Closing Session. Between the SLDS grant program and the proposed stimulus bill, it is clear that a lot is happening at the federal level concerning education data systems.

Summer 2009 Forum Planning
The Forum has traditionally focused on administrative data. Perhaps it is time to extend our view to include more about how data can be (or are being) used to support teaching and learning, and improve achievement in general. Could the Forum, for example, use its data expertise to develop a professional development module that helps teachers become better data users? Is there a speaker out there who could help us advance our knowledge in this area?

The Forum will plan sessions in which SEAs and LEAs discuss what support, specifically, LEAs would like to receive from SEAs. It will be good to get this type of dialogue going, either at the standing committee level or in a joint session.

A suggestion was made that mentors meet with new members at the end of Forum meetings (in addition to or prior to the meetings). The Steering Committee thought this was a good idea given how many questions come up for new members during the course of a meeting. Ghedam will follow up with mentors about this suggestion in time for the Summer 2009 Forum.

Next Conference Call
The next Steering Committee conference call is scheduled for March 10, 2009.

Meeting Notes
IES    
NCES
National Center for
Education Statistics
Explore the Institute of Education Sciences