Search Results: (1-15 of 877 records)
|REL 2016135||Examining the validity of ratings from a classroom observation instrument for use in a district's teacher evaluation system
The purpose of this study was to examine the validity of teacher evaluation scores that are derived from an observation tool, adapted from Danielson's Framework for Teaching, designed to assess 22 teaching components from four teaching domains. The study analyzed principals' observations of 713 elementary, middle, and high school teachers in Washoe County School District (Reno, NV). The findings support the use of a single, summative score to evaluate teachers, one that is derived by totaling or averaging all 22 ratings. The findings do not support using domain- or component-level scores to evaluate teachers' skills, because there was little evidence that these scores measure distinct aspects of teaching. The information that the total score provides predicts the learning of teachers' students. While the relationship is moderate, it is evidence to support interpreting the observation score as an indicator of teachers' effectiveness in promoting learning.
|WWC IRM449||Saxon Math
Saxon Math is a core curriculum for students in grades K-12 that uses an incremental approach to instruction and assessment. New concepts are introduced gradually and integrated with previously introduced content so that concepts are developed, reviewed, and practiced over time. Based on the available research, the WWC found that Saxon Algebra I has no discernible effects on algebra achievement for secondary students. No studies of Saxon Geometry, Saxon Algebra II, or Saxon Advanced Math meet WWC group design standards and, therefore, more research is needed to determine the effectiveness of these courses.
|WWC IRM052416||University of Chicago School Mathematics Project (UCSMP)
The University of Chicago School Mathematics Project (UCSMP) is a core mathematics curriculum for secondary students that emphasizes a student-centered approach incorporating problem solving, real-world applications, and the use of technology. The WWC reviewed the research on UCSMP’s secondary courses and found that UCSMP Algebra I has potentially positive effects on both general mathematics achievement and algebra for secondary students. In addition, the cumulative effect of multiple UCSMP courses was found to have potentially positive effects on general mathematics achievement for secondary students. No studies of UCSMP Geometry; UCSMP Advanced Algebra; UCSMP Functions, Statistics, and Trigonometry; or UCSMP Precalculus and Discrete Mathematics meet WWC group design standards. More research is needed to determine the effectiveness of these courses.
|REL 2016150||Stated Briefly: College enrollment patterns for rural Indiana high school graduates
This "Stated Briefly" report is a companion piece that summarizes the results of another report of the same name. This study examined 1) average distances traveled to attend college, (2) presumptive college eligibility, (3) differences between two-year and four-year college enrollment, (4) differences in enrollment related to differences in colleges' selectivity, and (5) degree of "undermatching" (i.e., enrolling in a college less selective than one's presumptive eligibility suggested) for rural and nonrural graduates among Indiana's 2010 high school graduates. "Presumptive eligibility" refers to the highest level of college selectivity for which a student is presumed eligible for admission, as determined by academic qualifications. The researchers obtained student-level, school-level, and university-related data from Indiana's state longitudinal data system on the 64,534 students who graduated from high school in 2010. Of the original sample, 30,624 graduates entered a public two-year or four-year college in the fall immediately after high school graduation. Data were analyzed using Chi-square tests, GIS analysis, and hierarchical generalized linear models. Rural and nonrural graduates enrolled in college at similar rates, but rural graduates enrolled more frequently in two-year colleges than nonrural graduates. About one third of rural graduates enrolled in colleges that were less selective than colleges for which they were presumptively eligible. Rural graduates travel farther to attend both two-year and less selective four-year colleges than nonrural graduates.
|REL 2016142||How are teacher evaluation data used in five Arizona districts?
Recent teacher evaluation reforms instituted across the country have sought to yield richer information about educators' strengths and limitations and guide decisions about targeted opportunities for professional growth. This study describes how results from new multiple-measure teacher evaluations were being used in 2014/15 in five school districts in Arizona (according to interviews with district leaders and instructional coaches and surveys of school principals and teachers), with each district administering its own local evaluation system developed to align with the overarching state evaluation regulations passed in 2011. Findings from a majority of the study districts indicated that online data platforms are facilitating observation-based feedback, with evaluation results reportedly influencing subsequent professional development for teachers—in particular shaping the work of instructional coaches and/or the support opportunities that are suggested for teachers within the district's online system. However, responding teachers in the five study districts expressed some skepticism about the relevance of school- and district-level professional development offerings, and viewed themselves as responsible for their own professional growth activities. In addition, respondents indicated that the timing of the release of standardized state test data renders those data less useful for professional development decisions than observation results. Meanwhile, teacher evaluation data are reportedly being less systematically used in talent management decisions, including to identify teacher leaders or to assign teachers to schools or classrooms. Regarding evaluation's impact, principals and teachers in a majority of study districts agreed that their new teacher evaluations have improved teachers' instructional practice, but teachers in all five study districts were less likely than principals to agree that evaluations have benefitted students. Together, these findings are suggestive of positive benefits from organizational structures that support the review of data during the school year, such as standards-based observation frameworks, benchmark assessments, professional learning communities, and instructional coaching and feedback. However, skepticism among teachers (particularly high school teachers) suggests that they may not yet perceive their evaluations as entirely credible and relevant to their work.
|REL 2016107||Reshaping rural schools in the Northwest Region: Lessons from federal School Improvement Grant implementation
This study examines implementation of the School Improvement Grant (SIG) transformation model in rural regions, exploring challenges in implementation and technical assistance to support these efforts. This study is not part of the federal evaluation of the SIG, which provides more comprehensive information about SIG schools. Leaders participating in research alliances with REL Northwest and other regional stakeholders requested this study to learn more about how implementation of the SIG transformation model has played out in rural schools across the nation.
Researchers used data from the first cohort of the U.S. Department of Education's SIG baseline database to administer a survey addressing four research questions: 1) How did principals of rural SIG transformation schools rate their school's implementation of the requirements of the transformation model?; 2) To what extent do principals report challenges to implementation of the transformation model?; 3) To what extent do principals report their school received technical assistance for the implementation of the transformation model?; and 4) To what extent are principals' reports of challenges and technical assistance related to implementation? The survey was sent to all cohort 1 SIG principals of rural schools using the transformation model—a group that represented 42 states and Bureau of Indian Education schools. The final sample size was 135 principals (67 percent of the 201 schools where staff members who worked under SIG were still present). All surveyed principals worked in schools that were similar in size and student characteristics to the total sample.
Principal responses highlight challenges in both implementation and technical assistance. The results confirm previous research, by finding that certain elements of the transformation model are challenging for rural schools to implement—particularly, those related to ensuring high-quality staff and family and community engagement. The study also finds that principals are more likely to implement strategies for which they receive technical assistance; at the same time, they implement fewer strategies that present challenges. This suggests that rural schools working on improvement strategies need help beyond just grant funding.
|REL 2016123||Developmental education and college readiness at the University of Alaska
This study examines the postsecondary readiness of first-time students who enrolled in the University of Alaska system over a four-year period. The study calculates the proportion of students considered academically underprepared for college and how placement rates for developmental education (that is, non–credit-bearing courses) vary for different groups of students. The study also determines the proportion of students placed in developmental education who eventually enrolled in and passed college English and math. Finally, the analysis looks at whether high school grades, rather than exam performance, are a better predictor of success in college-level courses.
Results show that developmental education rates were higher in math than English for students pursuing any degree type and increased as the gap between high school exit and college entry grew. Among students pursuing a bachelor's degree, developmental placement rates were highest for Black students from urban areas of the state (in math) and Alaska Native students from rural areas (in English) compared to all other student groups. Almost half (47 percent) of students placed in developmental courses eventually passed college English and almost a quarter (23 percent) passed college math. For students who enrolled directly in college, high school grade point average was a stronger predictor of college-level English and math performance than were SAT, ACT, and ACCUPLACER scores. Secondary and postsecondary stakeholders can use the findings to help identify students in need of support to be college-ready and to consider further conversation and additional research regarding whether and how to use high school grade point average as part of the placement process.
|REL 2016134||Stated Briefly: Can scores on an interim high school reading assessment accurately predict low performance on college readiness exams?
This "Stated Briefly" report is a companion piece that summarizes the results of another report of the same name. The purpose of this study was to examine the extent to which performance on Florida's interim reading assessment could be used to identify students who may not perform well on the Preliminary SAT/National Merit Scholarship Qualifying Test (PSAT/NMSQT) and ACT Plan. Data included the 2013/14 PSAT/NMSQT or ACT Plan results for students in grade 10 from two districts, as well as their grade 9 results on the Florida Assessments for Instruction in Reading—Florida Standards (FAIR-FS). PSAT/NMSQT Critical Reading performance is best predicted in the study sample by a student's reading comprehension skills, while PSAT/NMSQT Mathematics and Writing performance is best predicted by a student's syntactic knowledge. Syntactic knowledge is the most important predictor of ACT Plan English, Reading, and Science in the study sample, whereas reading comprehension skills were found to best predict ACT Plan Mathematics results. Sensitivity rates ranged from 81 percent to 89 percent correct across all of the models. These results provide preliminary evidence that FAIR-FS scores could be used to create an early warning system for performance on both the PSAT/NMSQT and ACT Plan.
|REL 2016133||Relationship between school professional climate and teachers' satisfaction with the evaluation process
This study, conducted by the Regional Educational Laboratory Northeast & Islands in collaboration with the Northeast Educator Effectiveness Research Alliance, reports on the relationship between teachers' perceptions of school professional climate and their satisfaction with their formal evaluation process using the responses of a nationally representative sample of teachers from the Schools and Staffing Surveys. Specifically, the study used logistic regression analysis to examine whether teachers' satisfaction with their evaluation was associated with two measures of school professional climate (principal leadership and teacher influence), teacher and school characteristics, and the inclusion of student test scores in the evaluation system. The results indicate that teachers' perceptions of their principals' leadership was associated with their satisfaction with the evaluation system—the more positively teachers rated their principal's leadership, the more likely they were to report satisfaction with their evaluation process. The rating teachers received on their evaluation was also associated with their satisfaction, with those rated satisfactory or higher more likely to be satisfied. Teachers whose evaluation process included student test score outcomes were less likely to be satisfied with that process than teachers whose evaluations did not include student test scores. The findings reinforce current literature about the importance of the school principal in establishing positive school professional climate. The report recommends additional research related to the implementation of new educator evaluation systems.
|REL 2016138||Summary of research on the association between state interventions in chronically low-performing schools and student achievement
This report presents a summary of research on the associations between state interventions in chronically low-performing schools and student achievement. The majority of the research focused on one type of state intervention: working with a turnaround partner. In this type of intervention, states assign an individual or team to work with a school to identify strengths and weaknesses, develop a school improvement plan, and provide technical assistance as the school implements the plan. In some cases, additional funding is also provided to support implementation of the school improvement efforts. Most of the studies were descriptive, which limits conclusions about the effectiveness of the interventions. Results of studies of turnaround partner interventions were mixed, and suggested that student achievement was more likely to improve when particular factors were in place in schools such as strong leadership, use of data to guide instruction, and a positive school culture characterized by trust and increased expectations for students. Although researchers sought to include research on a variety of state intervention types, few studies were identified that examined other types of interventions such as school closure, charter conversion, and school redesign.
|REL 2016132||Stated Briefly: The utility of teacher and student surveys in principal evaluations: An empirical investigation
This "Stated Briefly" report is a companion piece that summarizes the results of another report of the same name. This study examined whether measures from student and teacher surveys that reflect principals' practice are related to schoolwide academic performance. The study was conducted using data from 2011–12 on 39 elementary and secondary schools within a midsize urban school district in the REL Midwest Region. The research team used the results of the district's Tripod student and teacher surveys to construct six school-level measures of school conditions that prior research has shown to associate with effective school leadership. The study finds that adding the full set of six survey measures as a group results in statistically significant increases in variation explained in mathematics and composite value-added outcomes, but not in reading. A stepwise regression procedure identified two measures—instructional leadership and classroom instructional environment—as an optimal subset of the six measures. This evidence indicates that student and teacher survey measures can have utility for principal performance evaluation.
|REL 2016130||Decision points and considerations for identifying rural districts that have closed student achievement gaps
Rural districts have long faced challenges in closing the achievement gap between high-poverty students and their more affluent peers. This research brief outlines key decision points and considerations for state and district decisionmakers who wish to identify rural districts that have closed academic achievement gaps. Examining these districts’ experiences with organizational and instructional policies and practices may suggest activities associated with making achievement gains and narrowing achievement gaps that can be systematically investigated. Key issues in the process are highlighted by examples from recent work with rural stakeholder groups in Colorado and Nebraska.
|REL 2016124||Can scores on an interim high school reading assessment accurately predict low performance on college readiness exams?
The purpose of this study was to examine the relationship between measures of reading comprehension, decoding, and language with college-ready performance. This research was motivated by leaders in two Florida school districts interested in the extent to which performance on Florida’s interim reading assessment could be used to identify students who may not perform well on the Preliminary SAT/National Merit Scholarship Qualifying Test (PSAT/NMSQT) and ACT Plan. One of the districts primarily administers the PSAT/NMSQT and the other primarily administers the ACT Plan. Data included the 2013/14 PSAT/NMSQT or ACT Plan results for students in grade 10 from these districts, as well as their grade 9 results on the Florida Assessments for Instruction in Reading – Florida Standards (FAIR-FS). Classification and regression tree (CART) analyses formed the framework for an early warning system of risk for each PSAT/NMSQT and ACT Plan subject-area assessment. PSAT/NMSQT Critical Reading performance is best predicted in the study sample by a student’s reading comprehension skills, while PSAT/NMSQT Mathematics and Writing performance is best predicted by a student’s syntactic knowledge. Syntactic knowledge is the most important predictor of ACT Plan English, Reading, and Science in the study sample, whereas reading comprehension skills were found to best predict ACT Plan Mathematics results. Sensitivity rates (the percentage of students correctly identified as at risk) ranged from 81 percent to 89 percent correct across all of the CART models. These results provide preliminary evidence that FAIR-FS scores could be used to create an early warning system for performance on both the PSAT/NMSQT and ACT Plan. The potential success of using FAIR-FS scores as an early warning system could enable districts to identify at-risk students without adding additional testing burden, time away from instruction, or additional cost. The analyses should be replicated statewide to verify the stability of the models and the generalizability of the results to the larger Florida student population.
|REL 2016120||Stated Briefly: Teacher evaluation and professional learning: Lessons from early implementation in a large urban district
This "Stated Briefly" report is a companion piece that summarizes the results of another report of the same name. REL Northeast and Islands, in collaboration with the Northeast Educator Effectiveness Research Alliance, examined the alignment of teacher evaluation and professional learning in a large urban district in the Northeast. REL researchers examined the types of professional learning activities teachers reported they participated in, the alignment of the reported activities with what evaluators prescribed, and whether evaluation ratings improved from one academic year to the next. The study found that teachers received written feedback across all standards of the evaluation rubric. Each prescription tended to include one or two recommended professional activities, and more of these activities were professional practice activities, such as independent work to improve instruction, than professional development activities, such as courses or workshops. Teachers reported participating in more professional activities for the instruction-based standards than for the non-instruction-based standards. For all standards, less than 40 percent of teachers reported participating in the activities their evaluator recommended. While further work may be needed to strengthen the connection between teacher evaluation and a comprehensive system of teacher support and development, this study takes the first step in illustrating the need for coherence among these related systems.
|REL 2016122||A Review of the Literature to Identify Leading Indicators Related to Hispanic STEM Postsecondary Educational Outcomes
The purpose of this study was to review recent peer-reviewed studies in order to identify malleable factors measured in K–12 settings that are related to students' postsecondary STEM success, particularly for Hispanic students. Postsecondary STEM success was defined as enrollment in, persistence in, and completion of postsecondary STEM majors or degrees. Twenty-three relevant studies were identified, yet only 4 examined K–12 factors predictive of postsecondary STEM success specifically for Hispanic students. The review found that the number of high school mathematics and science courses taken, and the level of those courses is a consistent predictor of postsecondary STEM outcomes for all student subgroups. However, the literature indicates that minority students, including Hispanics, were less likely to take the highest-level mathematics and science courses. Students' interest and confidence in STEM at the K–12 levels was also predictive of postsecondary STEM success. Yet, despite lower levels of postsecondary STEM success, some studies indicate racial/ethnic minority and White students had similar levels of interest and confidence in STEM. The reviewed research suggests that reducing disparities in mathematics and science preparation between Hispanic and White students and increasing the rates at which Hispanic students take high-level mathematics and science classes has promise for informing interventions designed to improve STEM outcomes.