Skip Navigation
small NCES header image

Winter Forum 2010 Meeting Notes

National Forum on Education Statistics
March 1-2, 2010
Phoenix, AZ

Contents


Opening Session


Monday, March 1, 2010

Welcome and Opening Comments Zip File (2 MB)
Forum Chair Linda Rocks (Bossier Parish Schools, LA) welcomed Forum members to the 2010 Winter Forum Meeting in Phoenix, Arizona. Linda introduced this year's Forum officers and announced the release of three recently released Forum products—Crisis Data Management: A Forum Guide to Collecting and Managing Data about Displaced Students, the Forum Guide to Data Ethics, and the Forum's new outreach brochure, Shared Knowledge for Shared Success. Linda then reviewed the meeting agenda and reported that Forum website traffic continues to grow with an average of 7,343 visits per month since June 2007. During this same period, Forum publications were downloaded as PDFs or visited on their home pages an average of 6,575 times per month. The Chair then welcomed twelve new members to the Forum:

  • Tim Anderson, Washington State Office of Superintendent of Public Instruction
  • Frank Cernik, North Carolina Department of Public Instruction
  • Craig Hoyle, Regional Education Laboratory–Northeast & Islands
  • Candy Johnson, Kentucky Department of Education
  • Janice Johnson, Maryland State Department of Education
  • Brent Kay, Orange Southwest Supervisory Union (VT)
  • Marilyn King, Bozeman Public Schools (MT)
  • Douglas Levin, State Education Technology Directors Association
  • Judy Merriman, South Dakota Department of Education
  • Dorice Miller, North Dakota Department of Public Instruction
  • Tom Olson, South Carolina Department of Education
  • Ken Wagner, New York State Department of Education

Welcome to Arizona
Jeff Stowe (Arizona Department of Education) welcomed Forum members to his warm and sunny home state.

Top

Common Data Standards Initiative Zip File (703 KB)
Nancy Smith (NCES) provided an overview of the Common Data Standards (CDS) Initiative. The Initiative is currently building a voluntary, common vocabulary to help education agencies collect key data that are comparable, interoperable, and portable.

  • Comparable so that valid comparisons of student characteristics and performance can be made across district and state lines, and between the K–12 and postsecondary sectors.
  • Interoperable so data can be seamlessly transferred between systems and software applications.
  • Portable so records can easily follow students as they transition through the educational system, either predictably (e.g., progression from one school level to the next) or unpredictably (e.g., relocation or displacement from one school, district, or state to another).

The CDS Initiative is working to identify and define standards (i.e., element definitions, code sets, business rules, and technical specifications) for this set of key data. Broad use of the CDS standards could also reduce the collection burden on districts by increasing automation and reducing the need for cross-walks and other time-consuming activities. The CDS standards will at first be limited to a key subset of K–12 areas (e.g., demographics, program participation, course information) and K12-to-postsecondary education transition variables, though this scope may expand to early childhood education and the workforce in later phases of the project.

In addition to highlighting the goals and potential benefits of the CDS Initiative, Nancy also sought to clear up some misconceptions about the effort:

  1. Adoption of any or all of the CDS standards will be completely voluntary (not required).
  2. CDS is not a data collection.
  3. CDS is not a federal unit record system, but a model for data standardization to enable sharing between data systems.
  4. The CDS is not solely an ED undertaking. It is a collaborative effort that includes SEAs, districts, state public higher education entities, and national non-profit organizations. Furthermore, this initiative is not for the benefit of the federal government, but rather for the districts, states and postsecondary organizations who can use this common language.

The CDS Initiative follows two major pieces of legislation: The Education Science Reform Act of 2002 gave NCES the authority to determine voluntary standards and guidelines to assist state education agencies in developing SLDSs. And in 2009, the American Recovery and Reinvestment Act provided $250 million in competitive grants to help states implement and use SLDSs that include not only education data for elementary and secondary students, but also early childhood, postsecondary, and workforce information.

NCES is facilitating the CDS Initiative with the help of a Technical Working Group (TWG) consisting of representatives from local and state education agencies, state public higher education entities, and non-profit organizations. Nancy's presentation included a detailed timeline for the TWG's work over the coming years, as well as a sample of work to date. The Data Quality Campaign (DQC), Council of Chief State School Officers (CCSSO), and the association of State Higher Education Executive Officers (SHEEO) are working with the TWG, but also have their own grants to lead a communications and adoption arm of the Initiative. In this effort, they will market the standards to target audiences and encourage adoption of the standards.

Effective communications across institutions in the K–12 and postsecondary sectors will require the use of a common language. The CDS Initiative is working to provide a common vocabulary that can be adopted by the education community. A handout Adobe Acrobat (662 KB) was shared to provide an overview of the effort.

Top

Linking Early Childhood and Postsecondary Education to K–12 Zip File (2.2MB)

Elizabeth Laird (DQC) took the podium to discuss the linkage between early childhood education (ECE) and K–12 data. The need for high quality early childhood education data has grown in importance recently, and has been highlighted in a number of pieces of federal legislation. While 44 states reported to the DQC that they had linked some data between ECE and K–12, the degree of this data sharing is probably not very extensive. Elizabeth summarized the types of ECE programs that states link with K–12, the types of identifiers or methods used to link from ECE to K–12, and the points in time at which unique student IDs are assigned.

Ms. Laird considered the fundamental yet persistently challenging question, "What is 'P'?" (i.e., pre–kindergarten). While some states include prenatal care as a component of early childhood services, other definitions are far more limited in scope. Other challenges to linking K–12 to ECE data stem from the facts that:

  1. There is not a unified governance structure that connects early education programs and services.
  2. There is not a set of common data standards used across ECE programs and services.
  3. There is not a unified or consistent system for collecting even basic data about ECE programs and services.

For these reasons, it is often difficult for ECE programs and services to know basic facts like where children receive services or which children are enrolled in more than one program. Other barriers to making the link include lack of resources, common identifiers, coordination between the ECE and K–12 sectors, incompatible systems, and privacy concerns.

The DQC is part of an Early Childhood Data Collaborative (see PPT for details) that is working to 1) identify current efforts by states to collect and use ECE data and 2) help guide the way to establishing ECE data systems that are linked to K–12. Elizabeth's presentation detailed the key data elements and capabilities that characterize good systems, and listed some of the policy questions that these systems could help us to answer. Ms. Laird closed by previewing some of the collaborative's future activities, which will focus on building stakeholder partnerships, communicating policy questions and elements, sharing news of states' progress, and providing policy and advocacy support.

Hans L'Orange (SHEEO) then discussed the linkage of postsecondary and K–12 education data. He provided an overview of how postsecondary data systems are changing, how the data are being used to understand the flow of students through the education pipeline, and the factors that impact their success. Mr. L'Orange also discussed variation among P–20 councils not only in terms of structure, but also effectiveness.

According to Mr. L'Orange, the ultimate goals of connecting data across the P–20 spectrum are to coordinate the efforts and expectations of the many separate players in the system, and to better prepare students for success within and beyond the educational system. Hans then reviewed a number of pieces of federal legislation that encourage and assist states in making the connection between K–12, postsecondary, and workforce data. Mr. L'Orange reviewed recommendations for an ideal postsecondary education data system (e.g., flexibility, transparency, utility, and the ability to show longitudinal trends) and listed the challenges faced in establishing such systems (e.g., funding limitations/constraints, governance, data ownership issues, and privacy concerns). In closing, he reinforced the need for common data standards that will improve data usage across P–20.

Top

Tribute to Lee Hoffman
Lee Hoffman (NCES) will be retiring in the spring. Linda Rocks (Bossier Parish Schools, LA) invited Kathy Gosa (Kansas State Department of Education), Patti High (Oklahoma State Department of Education), Tom Ogle (Missouri Department of Elementary and Secondary Education), and Bob Beecham (Nebraska Department of Education) to take a few moments to wish Lee a happy retirement and thank her for all of her hard work and support over the years. Their remarks included many kind observations about their colleague, mentor, and friend: Lee is deeply knowledgeable, Professional ("with a capital 'P'"), consistent and steady through change, gracious, approachable, caring, reassuring, and a great tour guide. The Forum will miss Lee Hoffman.

Top

Joint Session: Linking Student and Teacher Data


Monday, March 1, 2010

Linking Student and Teacher Data Zip File (585 KB)

George Noell (Louisiana Department of Education) joined the Forum to share information about Louisiana's experiences connecting student and teacher data, and using those linked data to assess the value added by teachers and to evaluate teacher preparation programs. This work, which ultimately seeks to improve teacher quality and student performance, has gained national attention.

Dr. Noell began with an overview of how Louisiana established its student-teacher link. Relying on in-house programmers, the state built upon its existing data system and implemented a classroom identifier system that allowed the state to connect students with their teachers. Obstacles to making these links included the timing of collections, keeping up with changes in student course enrollment, using uniform course codes, special situation classes (e.g., Special Education), and data quality concerns. Dr. Noell asserted that strong SEA-LEA relationships as well as a single statewide student-level data system were very important components of this effort. He acknowledged, however, that in reality most districts within states use a multitude of different student information systems, which would complicate these types of links in many states.

Louisiana applied this information to the area in which it believes it has the greatest influence in terms of improving teacher quality and student performance: teacher preparation programs. A value-added model, which Dr. Noell described as "regression on steroids," was used to measure teacher effects on student performance. The model considers a number of variables, including the school in which a teacher works, a host of demographics data, several years of prior student assessment data, current year student performance, student attendance, and student disability status. Dr. Noell reviewed the state's assessment approach and discussed some methodological issues that have arisen (e.g., scalability of assessments, data element definitions, reliability of identifiers, etc.). To ensure accurate student-teacher linkages, prior to the analyses, the state gave teachers the opportunity to verify their rosters via a portal to resolve any matching errors.

Dr. Noell reported that the state's linkage and analyses have provided some "game-changing" insights that have altered the conversation around teacher quality. Many new teachers, for instance, are out-performing veteran teachers. And, the state has found that the preparation program a teacher comes from is a good predictor of student performance. The state's research has also supported the notion that certification in the content area that educators teach has a substantial impact on student performance. The 2009 report was well-received and researchers have received more requests to consider additional variables and examine finer details. In closing, Dr. Noell said that while states may have a lot of data, without quality and linkages, those data are limited in utility. Without good information, we are "flying blind."

Following the presentation, Forum members had a chance to ask questions of Dr. Noell. To accurately collect data on Special Education and link multiple teachers to individual students, Dr. Noell said states should adopt thorough code lists, be flexible, include both classes and services, and allow multiple connections to be made. The overall performance of a school is incorporated into the regression model in an effort to account for the effects of school leadership, classroom resources, and other school-specific factors. Finally, Louisiana makes all of its data on teacher preparation program effectiveness publicly available.

Top

Joint Session: State Fiscal Stabilization Fund (SFSF) Metrics and Race to the Top Panel


Tuesday, March 2, 2010

State Fiscal Stabilization Fund (SFSF) Metrics and Race to the Top (RTTT) Panel Zip File (9.9 MB)

Cathy Solomon (Office of the Deputy Secretary, ED) led the panel by discussing the American Recovery and Reinvestment Act (ARRA), SFSF metrics, and RTTT from the federal government's perspective. To provide some background, Ms. Solomon reviewed the Presidential Memorandum of Transparency and Open Government, which has raised the bar for federal government transparency, requiring ED, for example, to share certain detailed information about how funding is distributed and used across the country. Ms. Solomon encouraged the audience to submit ideas for further increasing transparency transparency at ed.gov/open.

Next, the speaker gave a brief overview of ARRA. Ms. Solomon described the funding phases and the ways in which the stimulus dollars will help support education programs and innovation. She described the federal government's website for tracking and sharing ARRA funds (recovery.gov), including how it is managed and key features such as downloadable state- and program-level reports that are updated on a weekly basis. This tool has shown that ARRA funds for education have already saved over 300,000 jobs across the nation. The recipient reporting process was also summarized, including the required elements, challenges faced (e.g., the shortcomings of the one-size-fits-all approach to reporting), and the purposes for which the data are being used.

Ms. Solomon then discussed the SFSF metrics, which are being collected to measure states' progress towards the four reform goals articulated in ARRA. The speaker broke down the list of 37 metrics under the four ARRA reform areas, 15 of which are new items that states have been asked to collect. These metrics are being used, in part, to understand how (or if) certain data are being collected by the states. For example, are elements being collected on teachers' use of data to improve instruction (e.g., are teachers being provided with student growth data?)?

Finally, Ms. Solomon provided an overview of RTTT, covering the competition's goals and priorities, the amount of available funding, the two phases in which it will be awarded, the system used to score applications for funding, and the unprecedented effort to make the process as transparent as possible. She called it, "the cleanest competition ever."

Next, Kathy Gosa (Kansas State Department of Education), Irma Jones (Tennessee Department of Education), and Dianne Bailey (Wyoming Department of Education) shared their state's experiences with RTTT and the monthly and quarterly collection and reporting of SFSF metrics. The speakers shared many similar state- and district-level challenges, including difficulty handling the extra collection burden without additional funding, gathering data not previously collected by districts, defining new elements and other associated metadata, submitting the data within a short turnaround period, and making the complex information easy to understand as called for by the legislation. Kansas and Tennessee have both faced recent budget cuts and personnel limitations that have further limited SEA capacity.

Kansas found that conducting web meetings helped districts to provide the data more effectively. The SEA also pre-populated much of the districts' data to save time and effort at the local level. Tennessee purchased a commercial reporting tool to help districts populate quarterly reports and has plans to partner with a vendor to handle this collection in the future. The state found that it already had some of the data required for SFSF, leveraging, for example, a long-standing collection on teacher effects and struggling schools. Tennessee also developed new legislation to deal with RTTT. Wyoming is the only state that has reported using a third party to collect and report its SFSF metrics. Ms. Bailey described her state's ability to automate ARRA reporting, highlighting the fact that it took the state only 23 minutes to report on the ARRA awards.

Top

National Education Statistics Agenda Committee (NESAC)


Monday, March 1, 2010

Morning Session

Welcome, Introductions, Agenda Review
Chair David Weinberger (Yonkers Public Schools, NY) called the committee to order and welcomed new and old members. The committee members introduced themselves by providing an example of a productive challenge encountered over the past year. Many common themes ran through the committee, including teacher-student data links, reduced funding and the need to find creative ways to use funding, reduced staffing and increased work, analysis of student behavior and discipline data, alignment issues, data quality, and data governance.

Summer 2009 NESAC Meeting Review
In July of 2009, NESAC convened at the annual Summer Forum Meeting in Bethesda, MD. Over the course of this meeting, the committee addressed several hot topics and lessons learned. Chair David Weinberger (Yonkers Public Schools, NY) reminded the committee about the work conducted over the summer and set the stage for continuing that good work here.

Top

National Education Data Model
Hugh Walkup (ED), and Alex Jackl, a consultant with the Council of Chief State School Officers (CCSSO), joined the committee to provide an update on the National Education Data Model (NEDM) work. The NEDM work originated in the Forum as a task force that concluded with the release of the National Education Data Model, Version 1, Pk–12—a conceptual data model. ED has taken up this work through the office of Educational Technology, with support from CCSSO, the Schools Interoperability Framework Association (SIFA), and several other partners, to expand and enhance the model to include early childhood, post secondary, and workforce.

The NEDM website includes the workable model, which can assist states, districts, and vendors in designing and implementing data systems. The model began by defining connections and relationships that are essential to the operation of a school district, and has expanded to include connections and relationships with Health and Human Services, Early Childhood Education, Postsecondary, and transcript standards. The website also includes several reports including:

  • A "Standard Comparison Report" that captures and compares critical standards, specifications, and data sets available in the P-20 data space;
  • A "State Core Report" that lists the data elements (including definitions and code sets) currently present in the LDSs of 19 states, and indicates the necessity of those elements for EDFacts reporting, the fulfillment of key ARRA requirements of ARRA, and more.
  • An "Early Childhood Report" under development that will reflect standards currently available in early childhood education (such as those associated with Head Start);
  • Transcript reports comparing the core elements with the SIF, PESC, and XML standards.

The National Education Data Model is not a data dictionary. It is an inventory of elements, definitions, and standards that currently operate in education organizations. The model is impartial with respect to which standards should be employed in a state or district.

Top

Elementary/Middle School Course Code Classification Working Group Update
Helene Bettencourt (Massachusetts Department of Education) presented an update on the Elementary and Middle School Course Code Classification Working Group. The group's charge is to create course codes for elementary and middle schools based on the SCED code model. In adopting the SCED code structure for the earlier grades, several modifications have been made, including the replacement of course credit with a minimum or maximum grade option. The group is aware that their work needs to find middle ground to accommodate a range of needs at the LEA and SEA levels.

To date, the workgroup has met twice and is hoping to publish the Elementary and Middle School Course Codes in the summer of 2010.

Crime, Violence, and Discipline Working Group Update
At the summer 2009 Forum Meeting, NESAC was presented with an opportunity to establish a working group that would update and revise previous work on discipline data. The Forum has a long history with discipline data beginning in the mid 1990s with the publication of the first discipline handbook. In early 2002 the Forum produced Safety in Numbers, a guide to reporting discipline data. Since that publication, several changes have occurred that require the document to be updated. Bill Smith (Sioux Falls School District, SD), the Chair of that group, presented an update on this work. The Crime, Violence, and Discipline Working Group will analyze a cross-walk to Special Education, Office for Civil Rights and the Office of Safe and Drug Free Schools data collections produced by ED. This cross-walk will be integrated into the update of Safety in Numbers. The working group is aiming to publish its update in July of 2010.

Top

EIMAC CRDC Workgroup Update Adobe Acrobat (1.2 MB)

In April of 2008 CCSSO's Education Information Management Advisory Consortium (EIMAC) formed a workgroup to discuss with and advise ED on implementation of the Civil Right Data Collection (CRDC). Linda Rocks (Bossier Parish Schools, LA), a member of the workgroup, and Rebecca Fitch (ED Office for Civil Rights) joined NESAC to provide an update on this work and the CRDC collection for this year.

The Office for Civil Rights has used the feedback given by the workgroup to retool the collection and collection process. Notable changes include:

  • Data that are already submitted by states into the EDFacts system (e.g., Special Education data) will be pre-populated in the CRDC in an effort to reduce some of the burden.
  • EDFacts will also provide the CRDC with child count and placement data with the expectation that states will pay special attention to these data.
  • Race/ethnicity data will be accepted in either the five or seven category formats depending on where the SEA/LEA stands in its implementation of the new categories.
  • The collection will be made in two parts in order to address significant confusion over the school year of reporting. Part one of the collection will be on March 29th. Part two will open in October 2010.

More information can be found on the CRDC website. The CRDC provides important information to the Office for Civil Rights and the public in ensuring equity in public education.

Top

Afternoon Session

Educational Alignment for Systemic Excellence Zip File (3.5 MB)

Educational Alignment for Systemic Excellence is a CCSSO project in collaboration with the Center for Education Leadership and Technology (CELT) and the University of Virginia's Darden and Curry Schools Partnership for Leaders in Education. The EASE project grew out of the Decision Support Architecture Consortium (DSAC) and the utilization and expansion of this framework by former chief Valerie Woodruff in the state of Delaware.

Rick Rozzelle, Jim Goodell, and Nancy Wilson of CELT presented an overview of the work to date. EASE is a framework to assist states in aligning services, processes, and tools, including Statewide Longitudinal Data Systems, among a state, local districts, and individual schools in order to provide an environment for continuous improvement. EASE is less about the technical architecture and more about the policy and processes in place at all levels. To date, different systems across states have been built with various funding streams and limited forethought into how the systems work together. This framework attempts to align systems, policies, and processes at the state and district levels so that a system can be designed to allow LEAs and SEAs to work in tandem.

Top

Data Governance Stories of Success Zip File (331 KB)

NESAC spent the remainder of Monday afternoon discussing the topic of data governance. With the massive amounts of data being collected by schools, districts, and states, a process for managing the collection, use, and dissemination of that information is essential. Examples of best and current practices were provided by Pat Sullivan, (Texas Education Agency) and Jim Harrington (Hillsboro Schools, OR).

The Texas Education Agency (TEA) has a long history of student level collections dating back to the 1989-1990 school year. With the new collection burden placed on the districts, TEA saw a need for the creation of a governance policy committee. In 1991, the Policy Committee on Public Education Information (PCPEI) was formed. This committee governs the collection of four major district data collections on organization, finance, personnel and student data. The committee meets regularly and includes representatives from different levels to ensure all needs are met. Changes to the data collection must be requested through a formal written form and are then vetted by the Information Task Force, a technical subcommittee of the PCPEI. TEA also convenes the Data and Information Review Committee (DIRC). This is an internal committee that monitors requests to districts, eliminates duplication of data requests, and advises the agency on data and information policy, among other things. Both of these groups report to the Information Policy Committee, an executive level committee that coordinates overall data management.

Oregon uses a federated model within the state to collect, maintain, and manage data. The Hillsboro School district works in partnership with other area districts in a data warehouse project. With several districts working together to collect and house student level data, a governance structure became necessary. The partner districts have a two-tiered governance structure separating roles at the regional and local levels. Moreover, every staff member in the district has a responsibility to be engaged in data governance and quality. The business practices in place are meant to assist in utilizing the data to have the greatest impact on students.

Top

LEA/SEA Breakout on Data Governance
NESAC then broke into SEA and LEA groups to further discuss the issue of data governance from each unique perspective. It is clear that governance is not systemic at all levels. In both LEAs and SEAs, governance needs to play an important role. In reality, however, this kind of formal process is not always adhered to correctly. Most often, it seems that good data governance grows out of a crisis or force of will. At the state level, the SLDS grant can be a catalyst for a more formal process. With great data comes great responsibility.


Tuesday, March 2, 2010

Morning Session

Common Data Standards Panel
In Monday's Opening Session of the full Forum, Nancy Smith (NCES) gave a presentation on the Common Data Standards (CDS) Initiative followed by presentationsby Hans L'Orange (SHEEO) on postsecondary data linkages, and Elizabeth Laird (DQC) on the work being done on early childhood data. All three joined NESAC Tuesday morning for a more robust discussion around standards and the P-20 pipeline.

The CDS work is anticipated to be a three year process starting with a minimal set of K–12 data and focusing on transition data elements into post secondary, including high school feedback reports. Years two and three will expand this core set to include early childhood data elements as well as additional K–12 and postsecondary elements. To date, we've built 52+ LDS systems trying to meet federal and state needs with the idea that we need to support student learning. We need to find commonality so when students move across state lines we can have a core set of information that is able to move with each child.

A working website will be launched in April to begin to share the work the of the ED-funded CDS Technical Working Group and harvest feedback about the first draft of CDS. There is a concern that the CDS will not reduce burden but, instead, shift it. Perspectives on this type of concern would be valuable feedback that can be gathered through the use of the website. Another key element of the website will be an FAQ document and use case scenarios for LEAs and SEAs.

With respect to early childhood data, it was suggested that the DQC and the partners work more closely with LEAs. Local districts may be a good resource for finding the types of elements necessary to define the 'P' in PK. DQC partners are currently struggling with determining which data are appropriate to collect at the state level, and which should be left to the local level.

Top

IES Statewide Longitudinal Data Systems Grant Program Update
Tate Gould (NCES) presented an update on the Statewide Longitudinal Data Systems (SLDS) Grant Program. The program began with a first round of grant funding in late 2005. Since then, the program has expanded from a K–12 focus to include the P-20 pipeline. The FY10 grants currently are being considered and an announcement of awards should be made in the next few months. The process of reviewing and awarding the SLDS grants is often confusing, so Tate spent his time with the committee reviewing how the grants are scored and funded.

The FY 2010 grant applications were submitted on or before December 4, 2009 through the E-grants system. The Institute for Education Sciences (IES), the fiscal agent of the grants, selected a team of reviewers after the submission deadline to ensure that the review panel did not include any potential grantees. The reviewers come from a range of backgrounds representing academic researchers, technical experts, and state and local data systems architects. In January, the reviewers independently scored each application—a process that took about 6 to 8 weeks. Once the scores were submitted, reviewers met in Washington, DC to discuss each application individually. If the reviewers had any questions about the application, the program team was responsible for finding the answers. The review criteria relied on a 1 to 7 scale where 7 is the highest and 1 is the lowest score. Applications also receive an overall score on a 1 to 5 scale where 1 is the best. Once the award decisions are made, the list of awardees will be shared with Congress. Members of Congress are given the opportunity to announce grant awards in their state prior to the official IES announcement.

The reviewers were allowed to award partial funding to states. For example, if the reviewers found only 9 out of the 10 outcomes listed by the state to be worth funding and a state agreed to remove the 10th outcome, that application could still be eligible for funding.

Top

SFSF Metrics and Race to the Top
Cathy Solomon (Office of the Deputy Secretary, ED) has been focusing on ARRA, including the implementation of Race to the Top (RTTT) and the State Fiscal Stabilization Fund (SFSF). Ms. Solomon visited with NESAC to follow up on the morning's Joint Session on SFSF metrics and RTTT and answer questions.

A great concern among committee members is the slowness with which funds have been distributed (i.e., the speed of funding distribution has not matched the urgency of need). While formula funds tend to be distributed quickly, SFSF has taken a bit longer because it is a new formula funding stream. In comparison, the distribution of discretionary funds, such as RTTT and the SLDS Grants, is relatively slower due to the arduous peer review and OMB processes they entail.

ARRA has injected a great deal of money into the economy through various funding streams and agencies. To help ensure that this money is used in a manner that will truly stimulate the economy, great emphasis has been placed on audits. Audits conducted by the Inspector General, which is independent of ED, involve a list of expectations that do not always match the rules put forth in the regulations. In effect, the burden placed on program offices and title programs may be increased. As issues with audit timing and/or requirements arise, states and districts should feel free to contact the Department directly. ED has developed guidance Adobe Acrobat (1.1 MB) to clarify questions related to recordkeeping, documentation, and reporting related to SFSF.

ED understands the position states and districts find themselves in during this time of financial uncertainty. Facing its own fiscal challenges, the Department is currently operating on its lowest staffing levels ever while having 150 percent more funding to distribute and monitor. ED also understands that the shortfalls in states and districts are greater than anticipated.

Beyond RTTT and SFSF, the Department's number one priority has been reauthorization of the Elementary and Secondary Education Act (ESEA), which ED hopes will be a bipartisan process. The Act will continue to focus on accountability with an added emphasis on flexibility.

Top

NESAC Discussion of Gates Foundation Associate Membership
Chair David Weinberger (Yonkers Public Schools, NY) informed NESAC of the Bill & Melinda Gates Foundation's request to become an associate member of the Forum and led a short discussion on the process for establishing associate membership. Based on a Steering Committee decision on Monday evening, the request would be brought to the full Forum for a vote during Tuesday's Closing Session.

Afternoon Session

Teacher-Student Linkages
NESAC members engaged in a follow-up discussion on teacher-student linkages. Louisiana's teacher-student linkage program has provided several lessons learned. For instance, the existence of a statewide course code system proved to be essential in making teacher-student connections in Louisiana. In addition, the state's staff collection has benefited from a coding structure that includes each type of staff member (e.g., teacher, principal, assistant teacher, etc.). The collection is also strengthened by a number of edit checks. For example, if the code for a teacher is entered, a class identifier must be attached. Teachers in Louisiana are still adjusting to the system and responding to the new tools.

It is clear that a number of policy issues associated with the linkage of student and teacher data still need to be resolved. For instance:

  • While teacher identifiers are essential, how will they be implemented? Policymakers need to weigh the merits of using SSNs, staff identifiers, certification IDs, or license numbers.
  • How long does a teacher have to be matched with a student in order to be held accountable for the student's progress?
  • Who should be held accountable for a migrant or highly transient student?
  • What is the goal of establishing this link? Is it truly to assist in teaching and learning or will it be used for punitive purposes?
  • What burden does the establishment of this link place on LEAs?
  • What policies and technical capabilities and features need to be in place at the local and state levels to facilitate the accurate matching of students with their teachers?

Top

SFSF Reporting Metrics
SFSF reporting burden falls almost exclusively on the SEA. But, while the SFSF burden is high now, members looked to past experiences for consolation. They recalled that when NCLB and Graduation Rate requirements first came out, similar burdens were levied and similar reactions were seen from states. But after three years, these burdens subsided into business as usual. Members expressed anticipation for three years down the line.

Discussion of Potential Working Group: Data Use
NCES staff recently suggested the formation of a Forum working group on data use—a hot topic in the education community of late. A Forum best practice guide could offer excellent guidance on how to use data and for what purposes. There are several outstanding questions about this proposed effort in terms of scope, audience, etc. A sign-up sheet was distributed to solicit volunteers to serve on the working group.

EDFacts Update for 2010–11 through 2012–13
Pat Sherrill (ED) provided the committee with an update on the EDFacts collection and a preview of the next few collection years. As a requirement of the Paperwork Reduction Act, the EDFacts data collection must undergo a full review under the Office of Management and Budget (OMB) process every three years. This review will include only minimal changes to the data requirements, including the addition of the Perkins consolidated report, certain accountability data, and the CRDC collection.

The clearance package—released Wednesday, March 3rd, includes several attachments:

  • Attachment B1 describes the legacy collections;
  • Attachment B2 provides an overview of the EDFacts collection (this is a must read for anyone not familiar with the collection);
  • Attachment C outlines all the changes to the collection; and
  • Attachment D provides ED's responses to the previous collection notice.

The OMB clearance process includes a 60-day comment period, which opened on March 3rd. After that period, ED is required to respond to all submitted comments. Those comments will then be posted along with any changes made and the collection notice will be re-released for a 30-day comment period. This clearance package will cover the EDFacts collection over the next three years. Any major changes resulting from ESEA reauthorization will be submitted for separate clearance.

Top

Policies, Programs and Implementation (PPI) Committee


Monday, March 1, 2010

Morning Session

Welcome and Introductions
Chair Vince Meyer (Wyoming Department of Education) called the PPI committee to order. He then asked for members to introduce themselves and describe their past experiences with the Forum.

Agenda Review
Chair Vince Meyer outlined the PPI agenda for the Winter Forum Meeting, noting a few changes. Vince reported that the elected Vice-Chair, David Fringer (Council Bluffs Community School District, IA), was no longer able to participate in the Forum. Therefore, PPI would need to elect a new Vice-Chair during this meeting.

Summer 2009 PPI Meeting Review
Chair Vince Meyer reviewed the work accomplished by PPI at the Summer 2009 Forum Meeting, noting which projects would also provide updates over the next two days.

Education Privacy Working Group Update
Levette Williams (Georgia Department of Education) updated PPI on the work of the Education Privacy Working Group. The group met on Sunday to continue its work to create a single web document that will update the Forum's earlier publications on privacy issues (The Forum Guide to Protecting the Privacy of Student Information: State and Local Education Agencies and The Forum Guide to the Privacy of Student Information: A Resource for Schools). The Family Policy Compliance Office, the ED office that implements FERPA, will review the product before it is released. The working group hopes to complete its work by the Summer 2010 Meeting. Subsequently, additional smaller print documents may be published to assist LEA staff.

Common Data Standards Initiative & Linking Early Childhood and Postsecondary Education to PK-12
Nancy Smith (NCES), Hans L'Orange (SHEEO), and Elizabeth Laird (DQC) visited with PPI to answer follow-up questions after the Opening Session. PPI members inquired when Common Data Standards would be available and how they could be accessed—the first CDS public review website is expected to be released in April 2010. Members also asked for use cases and other marketing materials for the Initiative's various audiences.

Top

Afternoon Session

Elementary/Middle School Course Classification Working Group Update
Helene Bettencourt (Massachusetts Department of Elementary and Secondary Education) provided the update for the new Elementary/Middle School Course Classification Working Group, which met on Sunday. A consultant (MPR) and CCSSO are supporting this effort and have developedstructure. The final product will include a broad overview as well as a detailed course code system. Helene summarized some of the challenges the group has been working through. They expect the next draft to be ready in April and final publication this summer.

National Education Data Model
Hugh Walkup (ED) gave an update on the National Education Data Model (NEDM). Hugh reviewed the Forum's work on the project from two years ago and presented the tool as it is today. The demonstration included EDEN files, data sets, and standard comparison reports. Sonya Edwards (California Department of Education) asked if the original Forum policy questions could be added to NEDM with maps to the data items needed.

PPI Discussion: State Higher Education Executive Officers (SHEEO) and P-20 Data Connections
Hans L'Orange (SHEEO) talked about the work undertaken by the postsecondary community since the Summer 2009 Forum Meeting. The postsecondary language in the SLDS grant program is a very big deal in the postsecondary community although many of its members are maintaining a "wait and see" attitude. A "state of the state" report will be coming out of the postsecondary sector soon.

PPI Vice Chair Elections
PPI members nominated Laurel Vorachek (Anchorage School District, AK) as the Vice-Chair. The nomination was seconded and approved unanimously.

Top


Tuesday, March 2, 2010

Morning Session

SFSF Metrics for Race to the Top Joint Session Follow-up
Cathy Solomon (ED) provided PPI members with an opportunity to ask questions to follow up on her joint session presentation. PPI members asked questions regarding the relationship between SFSF and EDFacts reporting, the timing of the SFSF funding approval, and the requirements for P-20 systems.

EIMAC Civil Rights Data Collection (CRDC) Workgroup Update
Rebecca Fitch (ED) provided an update on the CRDC. The EIMAC workgroup consists of LEA and SEA members that provide ED with suggestions for the collection. The 2009–10 collection has undergone a lot of changes based on recommendations from the EIMAC workgroup. This upcoming collection will come in two parts and data will be taken from EDFacts when possible in order to reduce collection burden. There were some questions from PPI members regarding the difference between the five race/ethnicity categories reported by districts and the seven categories reported by states to EDFacts.

PPI Discussion: SFSF Data Burden Discussion
Diane Bailey (Wyoming Department of Education), one of the panelists from the morning's Joint Session, joined PPI for a follow-up discussion on the burden of SFSF reporting. PPI members discussed the data burden in terms of having to leave some other work undone in order to complete this new federal reporting. Members also discussed the difference between data audits and data validations, as well as the item in the America Competes ActAdobe Acrobat (1.1 MB) that requires a data audit system as an element of a P-16 data system.

PPI Discussion: Teacher-Student Linkages
Chair Vince Meyer (Wyoming Department of Education) led a discussion on states' linkage of teacher and student data. Vince, along with Sam Pernici (Louisiana Department of Education), Tom Howell (Michigan Department of Management and Budget), and Ken Gu (Rhode Island Department of Elementary and Secondary Education) provided a quick update on their states' activities and progress in this area. PPI members agreed with the general session speaker that collecting the data is easy. Deciding what to do with them is the hard part. Members discussed course coding classification systems and the problems associated with using grades for teacher evaluation purposes.

Top

Afternoon Session

IES Statewide Longitudinal Data Systems (SLDS) Grant Program Update
Tate Gould (NCES) reported on the SLDS Grant Program. Tate provided an overview of how the grant process works, including the decisionmaking process and the timeline behind the awarding of the grants. To date, 54 SLDS grants have been awarded to 42 states. A fourth set of grants totaling $245 million will be made available through ARRA, with awards anticipated within the next several months. Applications for this latest round are closed and are now going through the review process, which Tate described in detail.

EDFacts Update
Pat Sherrill and Deborah Newby (ED) provided an update on the EDFacts Program. Currently, the EDEN collection is going through the OMB review process. ED has asked several pointed questions to reviewers and would like to get input from the Forum. Pat urged PPI members to review the clearance package and submit comments. There are no new items related to RTTT, ARRA, or SFSF included in EDEN. The Civil Rights Data Collection (CRDC) is a part of this OMB packet. The speakers also wanted everyone to know that the Office of the Secretary is an avid user of the EDFacts data.

LEA-SEA Breakout Sessions: National Data Standards
Chair Vince Meyer (Wyoming Department of Education) asked the committee to break into small groups to discuss the presentations given during the conference regarding national data standards, legislation, and collections. He asked the small groups to talk about their interests in these areas and to report back to the larger group.

LEA areas of interest included more details on FERPA and other privacy laws (e.g., how they can share information on students' free- and reduced-price lunch/economically disadvantaged status with teachers), the early childhood work reported on by DQC, establishing links between high school and college (both ways), and the possibility of getting LEA folks more involved when SEAs apply for federal grants (e.g., SLDS).

SEA areas of interest included the questions surrounding a PK student identifier, state statues for the P-20 connections, support of national data standards, and examples of P-20 data sharing agreements.

Winter 2010 Planning
Chair Vince Meyer (Wyoming Department of Education) provided an update from the Steering Committee. The two main action items from the Steering Committee were the approval of a new associate member to the Forum (the Bill & Melinda Gates Foundation) and the formation of a Data Use Working Group. PPI asked that the TECH Communications Subcommittee be reinstituted to review all of the completed Forum publications and determine whether they are reaching those who need them.

Meeting Review/Summer 2010 Planning
Chair Vince Meyer (Wyoming Department of Education) opened a discussion on topics of interest for the Summer 2010 Forum Meeting. Topic suggestions included: the National Student Clearinghouse, teacher evaluation, states that support their districts' use of data, the relationship between postsecondary data collections and SLDS (i.e., IPEDS), FERPA guidance, Department of Labor, and a half-day professional development session.

Top

Technology (TECH) Standing Committee


Monday, March 1, 2010

Morning Session

Welcome and Introductions
Chair Patsy Eiland (Alabama Department of Education) and Vice-Chair Lee Rabbitt (Newport Public Schools, RI) called the meeting to order.

Common Data Standards (CDS) Initiative
Nancy Smith (NCES, ED), Elizabeth Laird (DQC), and Hans L'Orange (SHEEO) joined TECH to receive questions about their Forum Opening Session presentation.

TECH members inquired about Nancy's vision for the presentation of the upcoming CDS product. She said that it would probably be in a data dictionary format similar to the NCES Handbooks Online and the National Education Data Model (NEDM) with definitions, specifications, etc., to facilitate "easy" adoption and adaptation in state and local education agencies. She anticipates that the product will be organized into subgroups focused on postsecondary education, technical specifications, data governance, and use cases. Nancy also stated that the CDS wasn't being developed from thin air but, rather, is based on best practices available from the NCES Handbooks Online, NEDM, SIF Association, and PESC. She also said that CDS will support the IES state longitudinal data systems (SLDS) effort. In the first year of the project, CDS will try to tackle the most obvious variables related to "transitions" between K–12 and postsecondary education (in both directions) before branching out to more complex issues in years 2 and 3. Nancy noted that she was especially interested in receiving feedback from state education agencies about whether they expect to adopt CDS recommendations or crosswalk to them.

Elizabeth Laird reported that the DQC was focused on defining the "P" (pre-kindergarten) in P-20 data systems. This task will be especially difficult in PK settings outside of public education (e.g., in Head Start and private schools). Head Start, for example, is administered through Health and Human Services, collects data directly from local sites, and will have SFSF funds administered through governors' offices rather than SEAs.

Hans L'Orange is pleased that SHEEO is at the table during the CDS development efforts because the postsecondary institutions he represents have some different needs than the PK–12 system.

Top



EDFacts Update for 2010–11 through 2012–13

Pat Sherrill (ED, Office of Planning, Evaluation, and Policy Development) reported that EDFacts is currently seeking a new three year clearance from the Office of Management and Budget (OMB). The request will appear in the Federal Register, with Section B1 focusing on the status of legacy collections, Section B2 on how the EDFacts dataset works, and Appendix C specifying proposed EDFacts changes. Appendix D will request public comments on additional topics. Following a 60-day public comment period, ED will have 30 days to submit responses to any concerns that arise, and the OMB will have 30 days to assess whether ED's responses are sufficient. Pat said that ED believes that the EDFacts collection is now considered to be fairly stable, so they are expecting to receive OMB clearance at the end of the four month approval process.

TECH members asked whether and how new collections (e.g., related to various RTTT initiatives) will fit into the EDFacts collection effort. Pat said that ED believes that much of the basic data for RTTT should already be available through EDFacts so any new burden is likely to be minimal. Pat also noted, not insignificantly, that the Office of Civil Rights (OCR) collection will also be receiving OMB approval and will soon be a component of the EDFacts collection.

TECH members inquired about impending technical improvements to be undertaken by EDFacts. Pat said that Ross Santy (ED) would be addressing these issues in his MIS presentation. Finally, Pat reminded TECH members that they are the experts on data collections and reporting—and asked that TECH members share feedback with EDFacts as they continue to evolve to meet ED's collection needs.

Top

Task Force and Working Group Updates
Data Ethics Task Force: Task force Chair Tom Purwin (Jersey City Schools, NJ) reported that the publication has been completed and is now available here.

Longitudinal Data Systems Task Force: Task force Chair Bruce Dacey (Delaware Department of Education) reported that the group has nearly completed its document. TECH members should expect to receive an email later this spring requesting final review and approval.

Section 508 Working Group: Task force Chair Lee Rabbitt (Newport Public Schools, RI) reported that the group is developing a short document to promote awareness about this important topic, with an expected release date in August 2010. She noted that she would be sharing a presentation on the Section 508 draft on Tuesday in the TECH meeting.

Top

Afternoon Session

State of Washington's Shared Services Model Zip File (139KB)
TECH member Peter Tamayo is the Chief Information Officer at the Washington State Office of the Superintendent of Public Instruction. He shared a presentation with TECH about how the state of Washington proposes to transform information technology through consolidation, shared enterprise services, and the virtualization of information systems at the SEA (and other state agencies). Discussion suggested that this type of consolidation is occurring in several other states as well. One key point, however, is that consolidation only works if an SEA receives the same or better service and performance at the same or lower cost. Unfortunately, some states do not believe this to be the case, which leads to concerns about such initiatives. TECH will continue to exchange ideas and discuss experience on this front as this model for consolidating resources matures. See attached PPT.

ED Office of Civil Rights (OCR) Collection
Rebecca Fitch, from the Office of Civil Rights at ED, announced that OCR is planning to accept data from EDFacts for the first time. See discussion in the NESAC notes for more details.

IES Statewide Longitudinal Data Systems Grant Program Update

Tate Gould (NCES) manages the Statewide Longitudinal Data Systems (SLDS) Grant Program at ED. The purpose of this program is to provide grants to state education agencies to design, develop, and implement statewide longitudinal data systems that efficiently and accurately manage, analyze, disaggregate, and use individual student data. The long-term goal of this program is to enable states to generate accurate and timely data to meet reporting requirements, support evidence-based education decisionmaking, and increase the efficiency and organization of transferring educational data among schools, districts, and states to improve student achievement. These grants will support statewide longitudinal systems that link individual student data, promote interoperability across institutions and states, and protect student privacy consistent with applicable privacy protection laws.

Tate shared a presentation about how grant applications from SEAs are scored. TECH appreciated this insight into the grant review process and looks forward to learning about FY 2010 award scheduled to be announced later this spring. See attached PPT.

Top


Tuesday, March 2, 2010

Morning Session

SFSF Metrics and Race to the Top
Cathy Solomon (Office of the Deputy Secretary, ED) joined TECH to receive questions about her general session presentation about "ARRA, Transparency, and Reporting."

TECH members asked Ms. Solomon for more specific guidance about audit expectations from the Inspector General so that SEAs know how to better design and develop their data systems for ARRA reporting. Ms. Solomon acknowledged that changing guidance with respect to "jobs creation" has caused problems for data respondents, and agreed that sharing information about audit expectations would certainly help SEA data reporting efforts. As such, she said she would take the suggestion back to her office and share it with her colleague Jim Butler in the SFSF program office, as well. Ms. Solomon also mentioned that ED hopes to build SFSF reporting requirements into the existing EDFacts system.

SFSF Data Burden Discussion
Chair Patsy Eiland (Alabama Department of Education) and Vice-Chair Lee Rabbitt (Newport Public Schools, RI) facilitated a discussion about data burden.

Some states reported that they were beginning to outsource new reporting demands (burden) because they simply didn't have enough staff to accomplish the additional work. Other techniques for simplifying reporting and reducing burden included pre-populating forms and consolidating services, including some states moving toward (or considering) a single vendor system.

TECH members also raised concerns about governors, legislators, and other powerful people making unplanned, ad-hoc, high-priority (as in, "drop everything else you are doing") data demands. Members also noted that organizations that contract out collections need more time to institute changes because this requires modifying agreements with outside contractors. A final, but consequential, point in this analysis was the inevitability that ongoing budget decreases would have detrimental effects on data quality.

Another important issue that arose dealt with the permanence of the SFSF data collection. To date, it is unclear to SEAs whether these reporting requirements will become permanent, or whether they will fade away as the SFSF funding agenda is completed over time. If it is the former (permanent), SEAs want to focus more attention on how the collections are developed. If the SFSF collections are one-time events, then documentation requirements may not be as important. Clearly, SEA planners need to know which approach to take given the limits they have on resources.

Based on this discussion, TECH members requested that the Forum write a letter to the U.S. Department of Education to voice support of SFSF funding to education, but also to request answers to some of the important questions that arose during this discussion. The issues to be inquired about in the letter include:

  1. Sustainability — Are SEAs building one-time collection tools or ongoing collection systems (if it is the latter, how will these systems be supported financially following the end of the SFSF funds)?
  2. Administrative Support — Can SFSF support the IT infrastructure needed to collect the data it mandates?
  3. Reporting Timelines — Can SFSF reporting deadlines reflect a realistic schedule for designing, introducing, testing, training (staff), implementing, and auditing a data collection system?
  4. Technical Guidance — Can ED provide specific definitions and technical specifications for collection items up front (before money is allocated)?

Top

Afternoon Session

National Education Data Model (NEDM) Update
Hugh Walkup (ED) and Alex Jackl (Choice Solutions) reported on the progress of NEDM, a project that originated in the TECH committee several years ago. They emphasized that the NEDM relies on other data standards (rather than recreating them) and will, hopefully be useful at all levels of education (local, state, federal, and national). Alex demonstrated NEDM on a development website and announced that it will soon be posted to a live site. See the attached PPT.

TECH'S Section 508 Accessibility Working Group Zip File (504 KB)
Vice-Chair Lee Rabbitt (Newport Public Schools, RI) shared a presentation about the progress of the TECH Section 508 Working Group. A first draft has been completed and the document is expected to be released this summer. See the attached PPT.

Student-Teacher Linkages
TECH discussed the technical feasibility and political reality of connecting teachers to student performance data. The overwhelming consensus was that this was coming (quickly) and that data experts needed to help educate policymakers on the reasonable uses and limitations of these data linkages. In other words, just because we can link these entities, doesn't mean we can do it well (as in, there are many factors other than teachers that influence student performance). Issues that arose included:

  • What elements are fair to use in a teacher evaluation?
  • How can student performance be integrated as one of numerous components of fair teacher evaluation?
  • What happens when multiple staff members "team teach" a single student or group of students?
  • How do we acknowledge that student performance is a multi-year process (i.e., advances in one year affect and are affected by prior and subsequent academic years)?
  • How do we address subjects that are not assessed (e.g., music and art)?
  • Are commonly used assessments designed for this type of application? Raymond Yeagley (NWEA) noted that the NWEA assessment and growth model was not designed for teacher evaluation, although some people are trying to use it for this purpose.
  • Are districts using "pay-for-performance" models to save money? If so, are they really going to pay more if a lot of teachers do well?
  • What happens in the case of a former special education teacher who does really well with low-achieving students who are not in special education classes? Can these students be grouped into one class with that gifted teacher? Will the teacher look like an underachiever because his or her students aren't performing particularly well (even though they are doing better than anyone else would expect of them)?

Teacher-student performance linkages are coming, but there are still a lot of unanswered questions related to methodology, data quality, etc. We want people to use data, but do we want them to use data in incorrect ways? How can the Forum help?

Top

Summer 2010 Planning
TECH members suggested the following items for our agenda when we reconvene this summer:

  1. What are the different models being used by states to coordinate data from LEAs to SEAs?
    • Everyone does their own thing as long as the data can eventually be reported comparably
    • A single-vendor mandate
    • A single-vendor option (e.g., financial incentives)
    • A regional model
      • How do these choices trickle through a state? Does one option necessitate or serve as an obstacle to a single transcript? If a state mandates a single vendor, what happens to the 10 percent of the districts in a state that are already happy with their systems? Can we discuss buy-in options versus mandates? Where do CDS and other standards (e.g., data definitions) fit into this discussion?
  2. Virtual education – States are facing a wide range of laws that place differing (and sometimes competing) burden on collecting data about virtual schools, programs, classes, etc. It is complicated further by virtual offerings in regular schools versus charter and virtual schools. How are people dealing with this?
  3. New technologies – Twittering, Facebook, Google apps, etc., are becoming more important data tools at the building level. How are people dealing with them?
  4. Contraband websites – Some organizations shut down access to Youtube, Facebook, etc., but, increasingly, these sites are being used for instructional purposes. How can reasonable management accommodate this without opening up the entire Internet?
  5. NCES Common Data Standards (CDS) – TECH would like to receive another update on the progress of this important project.
  6. National Education Data Model (NEDM) – TECH would like to receive another update on the progress of this important project.
  7. Challenges of collecting "special" program data, including migrant education, early childhood education, Special Education, etc.
  8. How does the Migrant Student Exchange (MSIX) fit into the SLDS landscape?

Top

Closing Session


Tuesday, March 2, 2010

Overview and Agenda Zip File (2MB)

NCES Update
Standing in for the Deputy Commissioner, Lee Hoffman (NCES) updated Forum members on some notable NCES and the Institute of Education Sciences (IES) initiatives:

  • Postsecondary, Adult, and Career Education
    Just over a quarter of NCES's statistics budget is devoted to postsecondary, adult, and career education. The signature program within the Postsecondary, Adult, and Career Education Division (PACE) is the Integrated Postsecondary Education Data System (IPEDS). IPEDS is a single, comprehensive system designed to encompass all Title IV institutions and educational organizations whose primary purpose is to provide postsecondary education. The IPEDS system is built around a series of interrelated surveys to collect institution-level data in such areas as enrollments, program completions, faculty, staff, finances, and academic libraries. Other PACE studies include the National Postsecondary Student Aid Study (NPSAS), the Baccalaureate and Beyond Longitudinal Study (B&B), and the Beginning Postsecondary Students Longitudinal Study (BPS).

    The Office of the Undersecretary of Education coordinates the Department's operational work in postsecondary education. Specifically, it coordinates the work of the Federal Student Aid program, the Office of Vocational and Adult Education, the Office of Postsecondary Education, and a few other offices. The Undersecretary, Martha Kanter, is setting out an aggressive agenda that involves NCES's PACE.

    In line with President Obama's goal of all Americans enrolling in at least one year of higher education or job training, Undersecretary Kanter is embarking on a new initiative to measure more completely and accurately the educational attainment and skills development of adults. This includes the challenging task of measuring industry-recognized credentials (certifications) and other forms of work-skills credentialing and training to add to NCES's measurement of traditional postsecondary qualifications awarded by institutions of higher education. NCES is taking the lead in this effort, working with the Office of Management and Budget and the Departments of Labor and Commerce. Refinements to the Current Population Survey (CPS) or even the American Community Survey (ACS) may be of some help in the short-run. A more thorough solution is likely to require a new survey focusing on adult education and workforce training. There are many conceptual and operational challenges to implementing such a survey successfully, and these will be a major focus of NCES.
  • NAEP Mapping Study
    NCES has completed and released a study entitled, Mapping State Proficiency Standards Onto NAEP Scales: 2005-2007. Continuing work begun with 2003 National Assessment of Educational Progress (NAEP), the study focuses on state proficiency standards, rather than on student assessment. It addresses the challenge of comparing one state's education proficiency standards to another, or one state's AYP-measured educational progress to another, when each state sets its own proficiency standards. The study does not evaluate state assessments or standards, which are established to meet state needs. However, using NAEP as a common measure of educational achievement, the study determines the "NAEP-equivalent score" associated with the percent proficient reported by each state for AYP. Lee highlighted some results of this exercise:

    • There is wide variation in where states set their proficiency standards.
    • Most state standards fall in the NAEP Basic proficiency range; few are in the NAEP Proficient range; and many are below Basic. State 4th grade math standards stand out as being particularly low, with the majority below Basic.
    • The analysis of changes in state standards from 2005 to 2007 shows that, of the 58 grade/subject combinations where there was change in standards, the new standards were more rigorous 21 percent of the time, less rigorous 45 percent of the time, and about the same level of rigor 34 percent of the time.

  • NAEP-TIMSS International Benchmarking Study
    President Obama has made the restoration of international leadership in education a priority. The various international assessments in which the U.S. participates allow us to monitor our progress compared to other nations. To provide states with richer data about the international context without adding cost or significant burden, NCES is taking advantage of the fact that the next measurement of basic subjects coincides with the next administration of the Trends in International Mathematics and Science Study (TIMSS) at grade 8. Braiding the two assessments together will enable NCES to predict reliably from NAEP data how individual states and other jurisdictions would perform on a central international assessment. Lee asserted that successful, robust linkages would pay dividends with future projections based on an approach that does not require data collection for a bridge study. NCES will soon be asking a sample of states to help validate the success of the organization's statistical linkage.
  • Data Privacy Initiatives
    Protecting individually identifiable information while using longitudinal data systems to the greatest benefit for students (e.g., what we can do, and how we can do it most effectively), is an issue for federal, as well as, state and local agencies. NCES, and ED as a whole, are undertaking several data privacy initiatives to 1) develop a comprehensive and robust privacy strategy to ensure that it is complying with federal law, 2) provide clear guidance to states, districts and schools for how to comply with federal law, and 3) provide technical assistance on technical safeguards and other measures to ensure that student information is protected. A few of the projects within that initiative include the following:
  • NCES is working as a member of the Department's Disclosure Risk Working Group to document current ED practices in protecting privacy in published administrative records data. The goal is to promote transparency in published administrative records data by developing a resource that can be used in responding to questions or concerns, from both within and outside the Department, about data protection.
  • NCES is convening a task force of national experts to develop advice to states on ways to connect researchers, analysts, and policymakers to make full use of state longitudinal data systems while protecting individual privacy and confidentiality.
  • NCES is developing a paper and a webinar series that that will provide overviews of statistically- sound methods, based on NCES data confidentiality standards, for ensuring student and teacher privacy and confidentiality when publishing school and district data based on student performance.

Lee wrapped up by telling Forum members that NCES appreciates the work they do and relies heavily on the organization for information on what's going on in SEAs and LEAs, and about what is feasible in terms of education data.

Top



Standing Committee Progress Reports

Task Force/Working Group Progress Reports

Top

Recognition of Completed Projects
Ghedam Bairu distributed plaques to acknowledge the members of the Crisis Data Management and Data Ethics task forces, whose products have recently been published. Linda Rocks also recognized the completion of the Forum's new marketing brochure, Shared Knowledge for Shared Success.

Forum Business
Chair Linda Rocks brought to the Forum a request by the Bill & Melinda Gates Foundation to become an associate member of the Forum. Voting unanimously in favor, members approved the request.

Meeting Evaluations
Forum members shared their opinions on the Winter 2010 Forum Meeting by completing evaluations.

Top

Steering Committee


Monday, March 1, 2010

Welcome and Agenda Review
Chair Linda Rocks (Bossier Parish Schools, LA) opened the Steering Committee meeting and welcomed members to the 2010 Winter Forum Meeting.

Working Group/Task Force Meetings
Sunday's working group and task force meetings were discussed.

  • The Crime, Violence, and Discipline Working Group met on Sunday morning. While members accomplished a great deal, they could have used more time together. This will not, however, delay their progress because they will finish the review of their meeting draft via a WebEx.
  • The Education Privacy Working Group continued its work on Sunday to update the Forum's existing privacy documents, which will be published as a single web document. While the group could have used more meeting time, members don't expect the shortage to affect their timeline. They expect to complete this work in the summer.
  • The Elementary/Middle School Course Classification Working Group has developed a draft coding structure, with the help of a consultant (MPR). The final product, which is expected to be released this summer, will include a detailed course code system.
  • The Longitudinal Data Systems Task Force did not meet on Sunday. Their product is in the final review phase (a partial draft is still posted on the group's webpage), and will soon be submitted for editing and design. Final publication is expected before the Summer Meeting.
  • The Section 508 Working Group had a productive session and advanced the first draft of its document substantially. The Working Group expects to have a public review draft prior to the Summer 2010 Forum Meeting.

Top

Review of Monday's Events

Members thought the presentation from Nancy Smith (NCES) went very well. Task forces were glad to get the opportunity to seek clarification in the follow-up sessions in Standing Committees throughout the day. The presentation on linking early childhood, K–12, and postsecondary education data was also well-received. All committees had plenty of follow-up questions for Elizabeth Laird (DQC) and Hans L'Orange (SHEEO). The link to early childhood education was of particular interest.

Next, Standing Committee Chairs reported on the day's discussions:

  • NESAC Chair David Weinberger (Yonkers Public Schools, NY) reported that the group had a productive discussion on data governance during their afternoon session. Members highlighted the apparent lack of a formal governance process in many state education agencies. They also noted that, in general, SEAs seemed to have a better handle on governance than districts. An important challenge in establishing governance is generating the political will necessary to begin and sustain such an enterprise-wide initiative. The committee considered the formation of a working group to explore the issue.
  • PPI Chair Vince Meyer (Wyoming Department of Education) gave a brief review of the committee's day, which included discussions of the CDS Initiative, P-20 linkages, NEDM, and SHEEO. In light of the resignation of David Fringer (Council Bluffs Community School District, IA), the committee nominated Laurel Vorachek (Anchorage School District, AK) as its new Vice-Chair. She was elected unanimously.
  • TECH Chair Patsy Eiland (Alabama Department of Education) reported that the TECH Committee received a particularly interesting presentation on the State of Washington's Shared Services Model, which is focused on transforming information technology in the state through consolidation, shared enterprise services, and the virtualization of information systems at the SEA (and other state agencies).

Steering Committee members thought that George Noell's (Louisiana Department of Education) presentation on linking student and teacher data was excellent. All agreed that his energy and engaging style provided a great ending to the day.

Top

Other Issues

  • Data Use Task Force: A proposal was presented to establish a working group on data use. There is considerable interest in this topic, specifically, in how teachers use data to inform instruction, how policymakers can use data to make better decisions, data use training methods, key data and questions, etc. Currently, many state data systems may be sophisticated enough to help answer key questions with their data, but the analytical capacity among staff may be lacking. The working group should consider the purposes for which data are being used ("Data use to... what?"). Members also thought that pre-service training on the use of data is an important issue to consider as it is critical if schools are going to "catch up," as one member put it. Members also noted the connection between data governance and data use. The future working group would need to decide on a reasonable scope of work. Consideration must be paid to whether this working group should tackle the issue one sub-topic at a time or produce a more comprehensive product. The Steering Committee voted unanimously to approve the group. Standing Committee Chairs will ask for volunteers on Tuesday to serve on the group.
  • New Forum membership vote: The Steering Committee considered the request from the Bill & Melinda Gates Foundation to become an associate member of the Forum. After some discussion, the group agreed that the Gates Foundation would be a good fit for the Forum and voted to approve the request and bring the vote to the full Forum during Tuesday's Closing Session.

Top

Tuesday, March 2, 2010

Review of Tuesday's Events
Members thought it was unfortunate that the SFSF/RTTT panel presentations ran over time and didn't allow for Q&A time. They also thought more emphasis on the future, rather than what had already been done, would have improved the session, although it was still a good opportunity to learn more about SFSF and RTTT.

Next, Standing Committee Chairs shared highlights from their sessions:

  • TECH proposed that a letter be drafted to ED voice support of the SFSF initiative in general, but also to highlight some of the Forum members' concerns about the SFSF collections, and to make suggestions on how to improve the process going forward. The Steering committee agreed that it would be a good idea to draft this letter to address issues such as the sustainability of the SFSF collection, the increased burden it imposes, privacy issues, the significant administrative and information technology resources the collection requires, and the need for ample lead time prior to a collection to allow education agencies to design and develop a system to handle the task. With all members in favor, the decision was made to draft a letter over the coming weeks to be shared with the Steering Committee.
  • NESAC will look at existing documents (Forum products in particular) to see if they adequately address their concerns about data governance and decide on whether there is a need to form a working group to focus on the topic.

Top

Other Issues

  • Members suggested that, in light of the increased interest in Forum products in recent years, it might be a good time to resurrect the Communications Subcommittee to promote our work.
  • Lists of volunteers for the Data Use Working Group were collected from each of the Standing Committees. Members suggested that this effort may also benefit from including experts from outside the Forum, such as curriculum specialists.
  • A letter will be written to invite the Bill & Melinda Gates Foundation to join the Forum as an associate member.
  • The presence of an increasing number of vendors during Forum meetings has become an issue. The Steering Committee is concerned that SEAs and LEAs may not feel comfortable speaking about their needs, opinions, project ideas, etc., when vendors are sitting in the room taking notes.

Top

Summer 2010 Forum
The Summer 2010 Forum meeting will be held in Bethesda, Maryland on July 26-28, 2010 and followed by the Annual MIS Conference. Looking ahead, members said they enjoyed the professional development session last summer and suggested that a half-day session be planned for the summer meeting. Members also suggested that a 1-1.5 hour time block be set for general topic presentations for standing committee members to attend based on their interests. This would allow members of the three committees to mingle. Participants also suggested that name tents be added to the meeting materials to make it easier to identify members during discussions.

Next Conference Call
The next Steering Committee conference call is scheduled for Friday, April 16, 2010 at 1 pm EST. This call will focus on meeting evaluations and the letter to ED on the SFSF metrics data collections.

 Meeting Notes

Top

Publications of the National Forum on Education Statistics do not undergo the formal review required for products of the National Center for Education Statistics. The information and opinions published here are the product of the National Forum on Education Statistics and do not necessarily represent the policy or views of the U.S. Department of Education or the National Center for Education Statistics.


Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.