Monday, February 16, 2009
Welcome and Opening Comments
Forum Chair Bruce Dacey (Delaware Department of Education) welcomed Forum members to the 2009 Winter Forum Meeting in Seattle, WA. Bruce introduced this year's Forum officers and announced the release of three new Forum products – Managing an Identity Crisis: The Forum Guide to Implementing New Federal Race and Ethnicity Categories , Every School Day Counts: The Forum Guide to Collecting and Using Attendance Data, and the Education Data Model Version 1 (PK-12). He then reviewed the meeting agenda and reported that Forum website traffic continues to grow with an average of 6,821 visits per month since June 2007. During this same period, Forum publications were downloaded as PDFs or were visited on their home pages nearly 6,000 times per month. The Chair then welcomed fourteen new members to the Forum:
Welcome to Washington
Peter Tamayo (Washington State Office of Superintendent of Public Instruction) welcomed the Forum to Washington for the Winter 2009 Meeting.
Data Systems and the Data Informed District
Dr. Jeffrey C. Wayman, Assistant Professor at the University of Texas at Austin, presented on data systems and the "data-informed district" (DID). Dr. Wayman began by considering the types of student data that are commonly collected as well as the rationale for collecting them. He then reviewed the concept of the DID, which is characterized by:
Three main types of data systems that are commonly used relative to teaching and learning were compared, noting the benefits and limitations of each:
Dr. Wayman listed a number of issues that should be considered when choosing a data system including: alignment with district goals and context, total and complete integration, interoperability, user-friendly access and navigation. He then argued that the focus of data use should broaden from just compliance and simple monitoring to encompass deeper issues of teaching and learning.
Next, Dr. Wayman reviewed findings of a case study from a Wyoming district in which he and colleagues studied data use practices and the data system environment. Though district leaders were supportive of data use, school level support was inconsistent, there was no clear vision for data use, and actual use of the data, especially for teachers, was challenging and inefficient. Following the study, the researchers made recommendations for improving the district's data use practices, including calibration (e.g., getting stakeholders to collectively define their goals and improvement strategies), acquisition of an integrated data system that provides easy access to a central data warehouse, and implementation of a district-wide data initiative.
Finally, Dr. Wayman offered his views on where data systems are headed in the future, forecasting that issues of importance will include portals, integration and interoperability, teacher evaluation, and a focus on using data to inform better education practices. Dr. Wayman is currently working on a prototype for system reviews by means of funding from the Dell Foundation. And, with support from the Spencer Foundation, Dr. Wayman will begin to study the effects of the DID on student performance and specific challenges that arise in district efforts to become "data- informed."
See some of Dr. Jeffrey Wayman's publications for further information.
After his Opening Session presentation and a round of visits with all three standing committees, Dr. Jeff Wayman led a follow-up discussion with LEAs to delve deeper into his research on data use for school improvement and to take questions from district representatives.
Dr. Wayman began by discussing some interesting practices he has seen in districts over the years. He told the story of a principal who set up a system of "non-threatening triangulation," an approach to data-driven decisionmaking conducted using three data points, one of which had to be teacher judgment. In another instance, a district used triangulation to determine eligibility for its gifted program, using multiple measures and student histories to decide which students were chosen for the program, rather than relying on a single data point to place students. And another school's teachers posted their data use goals on their classroom doors to increase transparency and focus their data use efforts.
The discussion then shifted to the concept of calibration, which is an open process in which a broad range of stakeholders convene to conduct a needs assessment and create a common vision for school or district improvement. In convening calibration discussions, Dr. Wayman suggested that ground rules be established early on to guide the process in an orderly fashion. Participants should define their problem and then figure out how to go about solving it. Re-calibration should take place on an annual basis to reflect progress and align with shifting visions.
Dr. Wayman was then asked to review his findings from a one-year-later perspective on the Wyoming district he and his colleagues studied to assess its data use practices and data system environment. He reported that it took six months before momentum started to build behind his team's recommendations. A new superintendent set specific, measurable achievement goals, an amended calibration process was undertaken, and a new SIS was purchased (despite the recommendation to start with a data warehouse before assessing the other systems). However, though progress had been made, Dr. Wayman felt that administrators were not truly invested in the process.
In response to another question, Dr. Wayman said that a data initiative can modify the way people work (e.g., teacher pedagogy), but it should not completely change the way they do business. A shift in culture in terms of the way people think about and use data is necessary, and data use should be built into the daily routine to inform work. Dr. Wayman also said that in many cases, especially in districts where paper data use and analysis are common, an electronic data system can dramatically reduce burden and increase efficiency.
LeRoy Rooker's recent departure from the Family Privacy Compliance Office (FPCO), the USED office that implements FERPA, and his withdrawal from the Forum Meeting left a void in the schedule. To fill this need, Helene Bettencourt (Massachusetts Department of Elementary and Secondary Education) and Bruce Dacey (Delaware Department of Education) moderated an SEA breakout session about privacy issues and the new FERPA guidance, which went into effect on January 8, 2009. In preparation for the session, a request for questions about FERPA was sent out to Forum members. The moderators presented a list of these questions and spent much of the session refining them and developing new questions to submit to FPCO. Questions they produced included:
These questions will be refined before submission to FPCO. The plan is to use them as talking points in a session with a FPCO representative at the Summer 2009 Forum Meeting.
Tuesday, February 17, 2009
PK-20 Data Sharing: Lessons Learned Panel
Hans L’Orange (State Higher Education Executive Officers (SHEEO)), Michael Sessa (Postsecondary Electronic Standards Council (PESC)), and Nancy Smith (Data Quality Campaign (DQC)) presented to the Forum on PK-20 data sharing.
Dr. L'Orange began by introducing the work of SHEEO, which serves its members (statewide coordinating and governing boards) in developing and sustaining excellent systems of higher education. From a "30,000 ft." policy level perspective, he discussed the importance of aligning the PK-12 and higher education data systems, asserting that while P-12 and higher education systems should remain independent, they must work together to increase student success.
Michael Sessa (PESC) introduced his organization and provided the "under the microscope" view of PK-20 data sharing, focusing on technical standards issues.
Nancy J. Smith of the DQC updated the Forum on the DQC's new survey results, focusing on progress towards P-20 alignment (#9 of the DQC's 10 Essential Elements).
Monday, February 16, 2009
Welcome, Introductions, and Agenda Review
NESAC Chair, Helene Bettencourt (Massachusetts Department of Education), called the committee to order, welcomed everyone to Seattle, and reviewed the agenda. The committee went around the table for introductions, each sharing a piece of “positive news.” While reviewing the proceedings from the Summer Forum meeting, a few ideas were identified as potential topics of discussion during the meeting, including discipline data and mobility data.
Attendance Task Force Update
Bill Smith (Sioux Falls School District, SD) shared an update with the committee on the Attendance Task Force. A PDF version of Every School Day Counts: The Forum Guide to Collecting and Using Attendance Data is now available online and the brochure was distributed during the meeting; the print document will be available shortly. Bill recognized the members of the task force and thanked them for their hard work.
The task force focused on several major areas, weaving them into a four chapter publication. The first task was to make the case for the importance of accurate attendance data, and illustrate how high quality attendance data can truly make a difference in schools. The publication presents a taxonomy of data elements and defines the differences between the terms “present” and “absent.” The document includes several case studies and examples of the benefits of good attendance data with an aim of addressing commonly arising problems at the district and state levels.
SLDS Grant Update
Tate Gould (NCES) gave the committee a quick update of the IES Statewide Longitudinal Data Systems (SLDS) grant program. The program will add to its current ranks with a third round of grantees expected to be announced by the end of the month. To date, 27 states have been awarded federal funds to begin developing or enhancing their longitudinal data system. The amount of money awarded in the third round will be largely dependent on the final passage of federal FY09 appropriations.
Also, the American Recovery and Reinvestment Act of 2009 includes an additional $250 million for the further development of these systems. The exact process for allocating this money had yet to be determined, but it can be assumed that there will be an additional competitive grant process under the auspices of IES/NCES. The money allocated in the stimulus package will be coupled with the requirements outlined in the America Competes Act with an emphasis on P-20 linkages.
With the growth of the IES SLDS program, Tate intends to broaden the scope and create a more transparent program. Lessons learned, knowledge sharing, and usable products will be shared with the wider LDS audience. The hard work of the grantee states will be leveraged to assist and inform states that have not received federal funding.
Opening Session Discussion (Jeffrey Wayman: School Improvement, Collaboration, and Professional
The Forum's opening session included a lively presentation by University of Texas professor, Jeffery Wayman. A well known researcher in the field education data collection and use, Dr. Wayman stressed the importance of the data-driven, or "data-informed," district, using collaboration and calibration in all areas of data use, and making sure all district staff receive professional development in the proper use and utility of data. Dr. Wayman joined the committee for a follow-up conversation and more in-depth discussion.
Using a question and answer format, the committee delved into issues surrounding SEA level initiatives, collaboration, how practice should inform policy, professional development, and training in higher education.
Interstate Compact on Educating Military Children
Tom Ogle (Missouri Department of Elementary and Secondary Education) shared information about the Interstate Compact on Educating Military Children. The Council of State Governments began this project in response to problems encountered by the highly mobile population of children of military parents. The average child of military personnel will change schools around nine times, often with little warning and across state lines. The compact allows this special population of students to waive some of the requirements usually required of a transferring student. For example:
The compact became active with the passage of legislation in ten states. In establishing the compact, these states will be required to set up a Council with in the state and attend national meetings. Concerns about this legislation include the lack of specificity, the possibility of different data standards and/or needed elements, and a duplication of effort with such projects as MSIX.
Derek Howard (Utah Department of Education) joined the committee to share and lead a discussion about what Utah is doing with e-transcripts. Currently, Utah is working on their Utah eTranscript and Record Exchange (UTREx) system of transcripts and records exchange. The project started in late 2007 and should be finished and operational in a year. The system will use a statewide data warehouse of transcript level data to allow sharing across the state and to send e-transcripts to higher education institutions both in and out of the state. Utah is using the DigitalSAMS, Student Achievement Management System, for the state's data warehouse. DigitalSAMS will also be made available to any district that would like it; an organization that declines the DigitalSAMS will be able to access the state data warehouse with limited functionality through SIF capabilities. The system will house student-level data by state, including name, address, demographics, test scores, classes, etc. Utah is building a student interface that will allow graduating high school seniors to send their transcripts to the college/university of their choice. Currently, information is not flowing back to the K-12 system from higher education, but Utah is working on that in the next phase.
National Education Data Model
In 2008, the Forum's PK-12 Data Model Task Force finished its work on the Education Data Model Version 1 (PK-12). At the completion of this project, Forum members expressed an interest in seeing this work continued through additional versions and capabilities. The Office of Education Technology, USED, will further develop this work through the Council of Chief State School Officers. Ty Mapp (CCSSO) and Beth Young (QIP) joined the committee to give an update on the progress of that work. A comprehensive team, including CCSSO, QIP, SIFA, and Kforce, has begun work on future versions of what is now called the National Education Data Model (NEDM). The team will work with an Advisory Group from local, state, federal, association, and vendor organizations, as well as a PK-20 steering committee. NESAC asked the speaker to consider how PK will be involved in the development of the NEDM.
Data Quality Curriculum
In 2004, the Forum released The Forum Guide to Building a Culture of Quality Data: A School and District Resource. From that effort, the Forum initiated the development of the Forum Curriculum for Improving Education Data: A Resource for Local Education Agencies (2007) and a series of online courses hosted by the SIF Association on SIFA University. Michael Derman (Central Susquehanna Intermediate Unit, PA) was instrumental in the development of the curriculum and its online course. The intent of these resources was to improve the quality data in the local education agencies. The publication has two main audiences: "all staff" (part I) and "data stewards/coordinators" (part II). Each part addresses the issues of accuracy, timeliness, security, and quality for both the broader audience and the more focused data collector/user.
The Central Susquehanna Intermediate Unit has been selected by the Pennsylvania Department of Education to work with their SLDS project on issues of data quality at the LEA level. Pennsylvania, through its intermediate units, will implement data analysis and data quality trainings in the LEAs. The 29 intermediate units will utilize the Forum's data quality curriculum in a train-the-trainer model for LEAs. In Michigan, a similar session was held for 200 district level data collectors. The curriculum is highly adaptable for a state or district's needs and can be delivered in a variety of ways based on audience need.
To date, use of the online tool has been sparse. Feedback from the committee included questions about why SIF-U is hosting the tool. Moreover, SIF email is being sent to people who register at SIFA-U to use the tool. Despite this concern, feedback from those who have used the tools for district-wide trainings has been overwhelmingly positive. The curriculum is well thought out, easy to use, and very helpful. The committee suggests rethinking the online portion of this tool, particularly removing any registration step and indicating an estimated time commitment at the outset.
Tuesday, February 17, 2009
Joint Session Review and Discussion: P-20 Data Sharing
Tuesday morning's joint session featured presentations from Hans L'Orange (SHEEO), Michael Sessa (PESC), and Nancy Smith (DQC). Following the session, Nancy Smith and Aimee Guidera of the DQC joined the committee to further discuss issues, challenges, and opportunities involved in P-20 data sharing. The mission of the DQC is to inform policymakers about the utility of quality education data and create a ground swell of political will to support the building and maintenance of these systems. For the past three years, the DQC has been pushing its mission through the "10 Essential Elements," an estimation of what state data systems need to include in order to be robust and effective. In the next phase of the DQC, the focus will shift from data capabilities to data use.
Sharing data across the spectrum from P-20 presents a number of issues that policy and practice need to address in order to reach the full utility of longitudinal data systems.
The DQC sees itself as a spotlight that shines the light on these issues. Since the start of the DQC survey, six states have reported having all of the 10 Essential Elements, and nearly all states have made tremendous progress in implementing a robust LDS.
In December, the U.S. Department of Education released final changes to its FERPA regulations. The new regulations address a number of the issues that states had concerning FERPA and the development and use of longitudinal data systems. Additional discussions about FERPA will be included at the Summer Forum meeting when a representative from USED can join the committee to discuss the new regulations in detail.
Implementing the new Race and Ethnicity codes—Where are states now?
Since the passage of the new codes for the collection of racial and ethnic data, there has been a great deal of discussion about how the change will affect states and districts. The Forum released Managing an Identity Crisis: The Forum Guide to Implementing New Federal Race and Ethnicity Categories in the summer of 2008. It includes significant guidance on the new codes and how to implement them prior to mandatory implementation in the 2010-11 school year. LEA committee members reported that they are still not receiving much guidance from their SEAs. For example, there is still uncertainty about how the switch will affect AYP reporting.
The committee reviewed recent, and older, publications released by the Forum. The Safety in Numbers publication developed by NESAC in 2002 will be reviewed and possibly updated by the U.S. Department of Education. This effort would include clarification on issues resulting from the Uniform Management Information and Reporting System (UMIRS).
EIMAC CRDC Workgroup Update
Linda Rocks (Bossier Parish Schools) gave the committee a quick update on the Civil Rights Data Collection Workgroup. The workgroup was formed in April 2008 by the Education Information Management Advisory Consortium (EIMAC) to give the U.S. Department of Education suggestions for improving the collection. The initial plan for the CRDC is to use EDFacts data to prepopulate certain sections of the collection. The workgroup provided feedback on a number of issues, including the instructions for filling out the survey, questions regarding certain data elements, communication, and the website. The CRDC has been put on hold until passage of FY 2009 appropriations.
Boston's Data Warehouse Project
Arleen Thompson (Boston Public Schools) offered a presentation about Boston's data warehouse project. Massachusetts bought a statewide license for data warehouses through Cognos, but has been slow in rolling it out to districts—and larger districts such as Boston Public Schools (BPS) have continued to advance with their own projects. In the spring of 2007 BPS was awarded a state grant to help with their efforts. Access to the BPS systems is currently at the school level and does not yet reach the teacher/classroom. Boston's first priority for this project was to look closely at the attendance data. Attendance was seen as an essential first step because it is used for compliance and for evaluating AYP. The data collected in the report is very useful at the school level. BPS has faced a number of challenges with this project including:
The next steps for the project include putting together a school-based review, issuing technical support to teachers, reviewing the state scheme for assessments and entering them into the system, and doing this all on a possibly slim budget due to severe cuts.
State Education Data Center Update
Maureen Matthews (CCSSO) provided an update on the State Education Data Center (SEDC), a project of the Council of Chief State School Officers to make education data readily available through the website www.SchoolDataDirect.org. In September of last year, the U.S. Department of Education issued a contract to CCSSO to support and enhance the utility of the SEDC. There are five major deliverables for this contract:
EDFacts Update and Technical Amendments
Ross Santy (USED) joined the committee to provide a few updates on EDFacts and the EDEN submission system. There have been a few modifications to the EDEN data set. In particular, changes have been made to the optional data groups for 08-09 and 09-10 reporting year:
All of this information can also be found on the EDFacts Communities site on the CCSSO website.
American Recovery and Reinvestment Act
In the last weeks and months, there has been growing interest in the American Recovery and Reinvestment Act. At the time of this meeting, the bill had just been signed into law. Exact allocations and rules regarding the stimulus money had not been made clear. The committee discussed how to access information about this initiative and possible strategies for using this money. It is clear that money will be allocated in three different ways:
Meeting Review and Ideas for Summer
The meeting concluded with a review and discussion of what should possibly be on the agenda for the summer meeting. The committee expressed an interest in discussing more PK data issues (including interfacing with K-12), teacher professional development data, mobility rates, and a follow up on the stimulus package.
Monday, February 16, 2009
Introductions and Agenda Review
PPI Chair James Haessly (Waukesha School District, WI) called the PPI committee to order, and asked for members to talk about one aspect of their job. Jim also reviewed the Winter Meeting agenda.
Crisis Data Management Task Force Update
Linda Rocks (Bossier Parish School System, LA) provided an update on behalf of the Crisis Data Management Task Force. The group has met six times since the Summer 2007 Forum. The group's product will focus on the data items and management issues surrounding the displacement of students due to a crisis. Linda led a discussion on the topic and fielded questions. PPI members asked about the field length of the student indicator, whether the indicator could be used to track military movement, the importance of the indicator to help track any possible behavior problems, etc. The task force plans to have a draft for review in the next few months with possible approval by July.
EIMAC Civil Rights Data Collection (CRDC) Workgroup Update
Jim Parsons (Richardson Independent School District, TX) gave an update on the EIMAC Civil Rights Data Collection (CRDC) Workgroup. The workgroup consists of LEA and SEA members who provide the USED with suggestions for the collection. Jim said the workgroup has made many suggestions, specifically related to improved documentation. The Office of Civil Rights has responded positively and made changes based on their comments. The Workgroup will be meeting again during the conference.
PPI Discussion: SEAs Reporting Data for LEAs
PPI members broke into LEA and SEA groups to discuss the pros and cons of SEAs reporting data for LEAs. These conversations were in response to the recent work being done by the CRDC (see above).
Opening Session Discussion (Jeffrey Wayman: Implementation of Data Driven Decision Making Through Professional Learning Communities)
Following up on his Opening Session presentation, University of Texas at Austin's Dr. Jeffrey Wayman joined PPI to discuss a number of data use issues related to his work. The session took a question and answer format.
National Education Data Model
In 2008, the Forum's PK-12 Data Model Task Force finished its work on the Education Data Model Version 1 (PK-12). At the completion of this project, Forum members expressed an interest in seeing this work continued through additional versions and capabilities. The Office of Education Technology, USED, will further develop this work through the Council of Chief State School Officers. Ty Mapp (CCSSO) and Beth Young (QIP) joined the committee to give an update on the progress of that work. A comprehensive team, including CCSSO, QIP, SIFA, and Kforce, has begun work on future versions of what is now called the National Education Data Model (NEDM). The team will work with an Advisory Group from local, state, federal, association, and vendor organizations, as well as a PK-20 steering committee. PPI members had the following concerns/ suggestions for the speakers:
Tuesday, February 17, 2009
Joint Session Review and Discussion: P-20 Data Sharing with Hans L'Orange
PPI members discussed issues surrounding P-20 data sharing with Hans L'Orange (SHEEO). They discussed the following P-20 data sharing topics: student privacy, individual versus aggregate information, whether the data is necessary versus just "nice to know", value-added documents for P-20, where to house a P-20 data system, and the impact of scholarships on data transfer.
SLDS Grant Update
Tate Gould (National Center for Education Statistics) reported on the Statewide Longitudinal Data Systems (SLDS) grants. The FY09 applications were reviewed and USED is currently waiting on a budget decision before awarding the grants. The economic stimulus package includes an additional $250 million for state grants. Information on the new grants will be forthcoming; they have a focus on P-20 data sharing. The SLDS program is also working on a website to have all states share in the lessons learned by grantee states. States that have submitted unsuccessful FY09 applications will receive comments before the new grant request for applications is released.
State Education Data Center (SEDC)
Ross Santy (USED) gave a quick update on the State Education Data Center and the two-way link between EDEN and the SEDC. They are currently working on the website's data download capabilities, as well as forming a technical group as an offshoot of their Advisory Group. SEDC is also expanding its help desk and providing a map between the NEDM and SEDC. About half of the data needed for SEDC are already collected through EDEN. PPI members wanted to make sure that there was a cap on the data requested by Standards and Poor.
FERPA Publication Discussion
Levette Williams (Georgia Department of Education) led a discussion on previous publications that PPI has produced about FERPA. Because of the issuance of new FERPA regulations, some of the publications need to be reconsidered. The group agreed that both the smaller guide, Forum Guide to the Privacy of Student Information: A Resource for Schools (2006) and the larger product, the Forum Guide to Protecting the Privacy of Student Information: State and Local Education Agencies (2004) should be redone. A working group will be formed to develop a plan for these revisions. The subcommittee discussed a few issues with these revisions.
State Education Data Center (SCED) Codes for Elementary and Middle School Courses
Lee Hoffman (NCES) gave an update of the current SCED codes project. They are thinking about expanding the codes to the elementary and middle school levels and Lee wanted feedback from the group on this idea. The members supported the expansion of the codes down to the elementary and middle school levels. Lee asked for volunteers to sit on a working group for these new codes.
Implementation of New Racial/Ethnic Codes
PPI Chair James Haessly (Waukesha School District, WI) asked the members to discuss how their organizations were implementing the new racial/ethnic codes. Most states said they would be at full implementation in the 2010-11 school year. Some states said they would use both sets of codes the first year of implementation.
Monday, February 16, 2009
Welcome and Introductions
Chair Tom Purwin (Jersey City Public Schools, NJ) called the meeting to order.
Opening Session Discussion (Jeffrey Wayman: School Improvement through Analysis of Data)
Dr. Jeffrey C. Wayman visited TECH following his Opening Session presentation to speak about data use and analysis issues in more depth. The discussion took the form of a question and answer session. Topics included:
Introductions and Agenda Review
Members and attendees introduced themselves and reviewed the TECH agenda and Chair Purwin reviewed the work TECH accomplished at the Summer 2008 Meeting in Bethesda, MD. This included discussions and activities related to FERPA, task orders, task forces, e-transcripts, Section 508 accessibility standards, data environment security, the Migrant Student Records Exchange Initiative, Schools 2.0, longitudinal data systems, and TECH publications.
Introductions and Agenda Review
Patsy Eiland (Alabama Department of Education) was elected TECH Vice-Chair for 2009 to replace Corey Chatis (Tennessee Department of Education), who resigned from the Forum.
Task Force and Working Group Updates
Data Ethics Task Force: Task force chair Tom Purwin (Jersey City Schools, NJ) updated TECH members on the status of the group's work. The task force has completed the document's content and expects that a final product will be available at the Summer 2009 Forum.
Metadata Task Force: Task force chair Tom Ogle (Missouri Department of Elementary and Secondary Education) reported that the document is with the editor. It will be sent for final Forum approval in the next couple of weeks and should be available at the Summer 2009 Forum.
Longitudinal Data Systems Task Force: Task force chair Bruce Dacey (Delaware Department of Education) reported that the group continues to advance a draft of its product. The task force has completed several chapters and hopes to have a final product by the Winter 2010 Forum.
Race/Ethnicity Task Force: Task force chair Bethann Canada (Virginia Department of Education) reported that the task force's final product was released in September 2008 and is available in print (via GPO order) and electronic form.
PK-12 Data Model Task Force: Task force chair Jeff Stowe (Arizona Department of Education) announced the release of the Education Data Model Version 1 (PK-12), which catalogues and describes the relationships among data used in PK-12 education. Version 1 is available online. At the Summer 2008 Forum (and as the project neared completion), Forum members expressed an interest in seeing this work continued through additional versions and capabilities. The Office of Education Technology, USED, will further develop this work through the Council of Chief State School Officers. Ty Mapp (CCSSO) and Beth Young (QIP) joined TECH to share an update on how that work is progressing. A comprehensive team, including CCSSO, QIP, SIFA, and Kforce, has begun work on future versions of what is now called the National Education Data Model (NEDM). The team will be working with an Advisory Group of staff from local, state, federal, association, and vendor organizations, as well as a PK-20 steering committee. TECH was reminded about the SEA focus of the NEDM, and members asked about user tools being developed to show the SEA perspective. It was agreed that since the TECH Committee was the home of the original Data Model task force, the current NEDM group will update the committee on NEDM status at each Forum meeting.
Task Order Update
Chair Kathy Gosa provided TECH with a final update on Kansas' FY 2007 State Cooperative System Task Order Award: Improving the Quality of Data Submitted by Kansas LEAs. The data quality certification program focuses on data coordinators and data entry staff. Its goals are to: 1) help users develop basic skills; 2) share best practices at a grass-roots level; and 3) recognize the hard work done by staff by awarding them with certification. Kansas continually works to expand and improve this program.
TECH Professional Development: Data Analysis and Mining Tools
Larry Fruth (Schools Interoperability Framework Association) led a team of presenters, including Forum member Glenn McClain (Platte Valley School District, Weld RE-7, CO), to discuss the nomenclature, quality issues, basic tenets, leadership issues, references, and examples of data analysis and mining tools. See the attached PPT.
TECH Discussion: Recordation
Following the presentation, TECH members divided into SEA and LEA breakout groups to compare and contrast data analysis needs, strategies, and trends. LEA members concluded that: SEAs drive a lot of data mining and analysis activities; intermediate agencies can be very helpful in sharing tools and expertise with LEAs; and reconciling records (e.g., transfers) is a role for states. SEA discussions identified numerous related issues, including: it is an SEA's role to provide data analysis and mining tools that "keep a level playing field" within and across states; professional development needs are critical; oversight and governance are also critical; leadership issues and "support from the top of the organization" is necessary but not sufficient for effective data mining and analysis efforts; and SEAs need to prove the utility of such efforts to LEAs.
Additionally, TECH members concluded from this conversation that the committee should facilitate these types of conversations between SEA and LEA representatives. More specifically, SEA members want to hear more about what state agencies can offer to local agencies. This information is necessary if SEAs are going to be able to provide assistance to their counterparts in LEAs.
TECH will conduct a final review of its previously retired technology publications: Weaving a Secure Web Around Education: A Guide to Technology Standards and Security (2003), Technology in Schools: Suggestions, Tools and Guidelines for Assessing Technology in Elementary and Secondary Education (2002), and Safeguarding Your Technology: Practical Guidelines for Electronic Education Information Security (1998). Together, these publications have been accessed online over 31,000 times in the last 18 months. TECH believes that these are useful resources and that having a small group update them will not take more than a few months of work. Doing so would be a good use of Forum resources and good for the education data community given how often these publications are being accessed. Bethann Canada (Virginia Department of Education) and Tom Purwin (Jersey City Public Schools, NJ) volunteered to contribute to these efforts.
TECH also had several ideas for new task forces. Potential topics included Section 508 Accessibility Standards and data release policies and procedures. TECH decided to proceed with the Section 508 request for now. Lee Rabbitt (Newport Public Schools, RI) volunteered to chair a working group. Additionally, TECH would like the Forum to investigate various methods states and districts use to incorporate AP and SAT results from the College Board. Perhaps it might be possible for the Forum to write a letter requesting changes in the College Board's approach to supplying these test data.
Tuesday, February 17, 2009
Technology Issues in SEAs and LEAs
Patsy Eiland (Alabama Department of Education) facilitated an open discussion about how technology issues affect SEAs and LEAs. Strands of discussion included:
Data Quality Campaign
Nancy Smith and Aimee Guidera asserted that the 10 Essential Elements collection they publish is not an accountability instrument but, rather, a "conversation starter." Forum members appreciated this distinction but also noted that there may be some concern about the quality of the self-reported data. For example, if an SEA receives one file per year with college enrollment data about the previous year's high school graduates, should they answer "yes" to the question about "Can you exchange data between P-12 and postsecondary education?" Issues surrounding the ten essential elements, longitudinal data systems, e-transcripts, and P-20 data sharing will continue to arise over the next few years, so TECH hopes to advance this dialogue with the DQC.
Technology Issues in SEAs and LEAs (Continuation of Morning Discussion)
Forum members don't believe that P-20 data exchange is really a two-way conversation at this point. Higher education simply isn't returning data to P-12, even though a single annual report might be enough to serve P-12's research needs (e.g., Do our students finish college in four years? Do they need remediation?). The connection to labor force statistics/research is also very compelling.
Tate Gould (National Center for Education Statistics) gave an overview of the IES Statewide Longitudinal Data Systems (SLDS) grant program. FY 2009 awardees have been identified, although they will not formally receive grants until the federal budget is finalized. NCES will attempt to make future SLDS activities more transparent to the education data community. For example, WebEx's will be publicized and open to anyone who wants to participate.
In addition to SLDS funds, the American Recovery and Reinvestment Act includes $250M for improvements to state education data systems. A request for applications has been prepared in the likelihood that this money will be appropriated. It is possible that applications and awards will occur on a short timeline. All states will be eligible to apply for these funds, regardless of whether they have already received an SLDS grant.
Resources and Tools to Use Data to Improve Learning
Tom Purwin facilitated a discussion about data use to improve learning. TECH discussed important aspects of this issue, including: How do you know when teacher professional development has worked? What metrics can be used? How do data contribute to school improvement? In the past, district administrators might have assessed effectiveness through data such as "I trained 100 teachers" without any measure of how those teachers changed or how learning was improved. There are a lot of variables to consider in this type of analysis, but this is a direction for future research and consideration.
Following the presentation, TECH members broke into SEA and LEA groups to discuss these issues from their different perspectives. SEA members acknowledged that there is a large gap between the utility of the administrative and instructional data that they collect. While it is possible that statisticians could help teachers use data more effectively, this is not the situation in most education settings. Currently, we are much better at collecting administrative data for compliance and resource management purposes. LEA members noted that program data are rarely meaningful to LEA or school staff (i.e., that they aren't particularly useful for instructional decisionmaking).
Patsy Eiland (Alabama Department of Education) led a discussion about "lessons learned" by TECH members that might be transferable to colleagues in other states and districts. For example, it is unwise for data staff to assume that vendors understand their needs. Unless RFAs are clear and explicit, vendors rarely meet unspoken expectations. TECH will try to exchange RFA language as a part of its new "sharing resources/tools" initiative.
Meeting Review/Summer 2009 Planning
Chair Tom Purwin thanked the TECH Committee for its hard work at the meeting.
Overview and Agenda
Stuart Kerachsky, Acting Commissioner of the National Center for Education Statistics (NCES), updated Forum members on recent business and coming plans at NCES and the Institute of Education Sciences (IES).
In closing, Commissioner Kerachsky asserted that end users should be involved in data system design and said that the federal government and the states can and should do a better job of using data to improve education.
Task Force/Working Group Progress Reports
Standing Committee Progress Reports
Forum members shared their opinions on the Winter 2009 Forum Meeting by completing evaluations.
NCES Winter Forum 2009
Monday, February 16, 2009
Welcome and Opening Comments
Chair Bruce Dacey (Delaware Department of Education) opened the Steering Committee meeting and welcomed participants to the 2009 Winter Forum Meeting.
Steering Committee members felt that the morning Orientation Session went very well. Members agreed that the sooner new members get involved, the better; and joining a task force is a great way to learn how the Forum really works. The mentoring program seems to be working as well. Getting new members up-to-speed quickly is important. We are doing a much better job of this than we did five years ago, but that doesn't mean we can afford to change our focus on this important ongoing task. There will be a full orientation at the Summer 2009 Forum.
The Steering Committee was happy with the Opening Session presentation by Dr. Jeffrey Wayman. He managed to stay out of the "ivory tower" from which he came. Having said that, Dr. Wayman may have underestimated the expertise of his Forum audience. We will need to see how the post-meeting evaluations assess the presentation.
NESAC: NESAC members were concerned about the Forum Curriculum for Improving Education Data being maintained at the Schools Interoperability Framework Association (SIFA) website. Although everyone agreed that SIFA was a valued Forum partner, people believed that personal data submitted when registering for the online course was being used to send unsolicited email about SIFA activities. There were also questions about whether it made sense for a Forum product to exist somewhere other than the Forum or NCES website. The Steering Committee agreed to: 1) request usage statistics for the curriculum to see whether the SIFA site is a better "marketing" location than the Forum site; and 2) ask that SIFA not use curriculum registration data to populate electronic mailing lists.
PPI: PPI members asked about NCES plans for managing cooperative system contracts. Some state representatives noted that they wouldn't be able to attend the Forum if NCES returns to the old task order funding mechanism. NCES said that states just need to let NCES and/or Coffey Consulting (the logistics contractor) know what their needs are with respect to participation support.
SEA/LEA Breakout Time
SEA representatives met during breakout time to discuss FERPA guidelines. While this session had good potential, two problems arose: 1) USED's FERPA representative was not able to attend the meeting, and the absence of an "authoritative" voice severely limited the value of the discussion; and 2) the room was too big to be conducive to this type of discussion, which detracted from the effectiveness of the session.
LEA representatives met with Jeffrey Wayman. Steering Committee members believed that the session went very well.
Tuesday, February 17, 2009
Standing Committee Time
TECH chair, Tom Purwin, reported that TECH established a Section 508 Accessibility working group. NESAC and PPI were encouraged to contribute members to the group.
PPI believes that recent changes to FERPA necessitate updates to the Forum Guide to the Privacy of Student Information: A Resource for Schools (2006) and the Forum Guide to Protecting the Privacy of Student Information: State and Local Education Agencies (2004). PPI will establish a working group to deal with this important issue. PPI also received a request from NCES to contribute to the development of a course codes standardization project for elementary and middle schools (parallel to the Secondary School Course Classification System: School Codes for the Exchange of Data (SCED) (2007). PPI will work with Lee Hoffman at NCES to support this effort as requested.
NESAC and TECH both raised concerns about the quality of self-reported SEA data to the Data Quality Campaign.
PK-20 Data Sharing
The Steering Committee felt that the panel went fairly well, although people wondered how much new information was shared. There was a general sense that very little progress is being made on this front at the national level.
The Steering Committee felt that NCES Acting Commissioner Stuart Kerachsky made a very strong showing at his first Forum. His participation throughout the meeting was very much appreciated and he gave a good update of NCES activities at the Closing Session. Between the SLDS grant program and the proposed stimulus bill, it is clear that a lot is happening at the federal level concerning education data systems.
Summer 2009 Forum Planning
The Forum has traditionally focused on administrative data. Perhaps it is time to extend our view to include more about how data can be (or are being) used to support teaching and learning, and improve achievement in general. Could the Forum, for example, use its data expertise to develop a professional development module that helps teachers become better data users? Is there a speaker out there who could help us advance our knowledge in this area?
The Forum will plan sessions in which SEAs and LEAs discuss what support, specifically, LEAs would like to receive from SEAs. It will be good to get this type of dialogue going, either at the standing committee level or in a joint session.
A suggestion was made that mentors meet with new members at the end of Forum meetings (in addition to or prior to the meetings). The Steering Committee thought this was a good idea given how many questions come up for new members during the course of a meeting. Ghedam will follow up with mentors about this suggestion in time for the Summer 2009 Forum.
Next Conference Call
The next Steering Committee conference call is scheduled for March 10, 2009.
Publications of the National Forum on Education Statistics do not undergo the formal review required for products of the National Center for Education Statistics. The information and opinions published here are the product of the National Forum on Education Statistics and do not necessarily represent the policy or views of the U.S. Department of Education or the National Center for Education Statistics.