Search Results: (1-15 of 297 records)
|REL 2016111||Measuring school leaders' effectiveness: Findings from a multiyear pilot of Pennsylvania's Framework for Leadership
This "Stated Briefly" report is a companion piece that summarizes the results of another report of the same name. This study examines the accuracy of performance ratings from the Framework for Leadership (FFL), Pennsylvania's tool for evaluating the leadership practices of principals and assistant principals. The study analyzed four key properties of the FFL: score variation, internal consistency, year-to-year stability, and concurrent validity. Score variation was characterized by the percentages of school leaders earning scores in different portions of the rating scale. To measure the internal consistency of the FFL, Cronbach's alpha was calculated for the full FFL and for each of its four categories of leadership practices. Analyses of score stability used data on FFL scores of school years across two years to calculate Pearson’s correlation coefficient. Concurrent validity was assessed through a regression model for the relationship between school leaders' estimated contributions to student achievement growth and their FFL scores. This report is based primarily on the 2013/14 pilot in which 517 principals and 123 assistant principals were rated by their supervisors; an interim report examined data from the 2012/13 pilot year. The study finds that the FFL is a reliable measure, with good internal consistency and a moderate level of year-to-year stability in scores. The study also finds evidence of the FFL’s concurrent validity: principals with higher scores on the FFL, on average, make larger estimated contributions to student achievement growth. Higher total FFL scores and scores in two of the four FFL domains are significantly or marginally significantly associated with both value-added in all subjects combined and value-added in math specifically. This evidence of the validity of the FFL sets it apart from other principal evaluation tools: No other measures of principals' professional practice have been shown to be related to principals' effects on student achievement. However, in both pilot years, variation in scores was limited, with most school leaders scoring in the upper third of the rating scale. As the FFL is implemented statewide, continued examination of evidence on its statistical properties, especially the variation in scores, is important.
|REL 2016106||Measuring school leaders' effectiveness: Final report from a multiyear pilot of Pennsylvania's Framework for Leadership
This study examines the accuracy of performance ratings from the Framework for Leadership (FFL), Pennsylvania's tool for evaluating the leadership practices of principals and assistant principals. The study analyzed four key properties of the FFL: score variation, internal consistency, year-to-year stability, and concurrent validity. Score variation was characterized by the percentages of school leaders earning scores in different portions of the rating scale. To measure the internal consistency of the FFL, Cronbach's alpha was calculated for the full FFL and for each of its four categories of leadership practices. Analyses of score stability used data on FFL scores of school years across two years to calculate Pearson’s correlation coefficient. Concurrent validity was assessed through a regression model for the relationship between school leaders' estimated contributions to student achievement growth and their FFL scores. This report is based primarily on the 2013/14 pilot in which 517 principals and 123 assistant principals were rated by their supervisors; an interim report examined data from the 2012/13 pilot year. The study finds that the FFL is a reliable measure, with good internal consistency and a moderate level of year-to-year stability in scores. The study also finds evidence of the FFL's concurrent validity: principals with higher scores on the FFL, on average, make larger estimated contributions to student achievement growth. Higher total FFL scores and scores in two of the four FFL domains are significantly or marginally significantly associated with both value-added in all subjects combined and value-added in math specifically. This evidence of the validity of the FFL sets it apart from other principal evaluation tools: No other measures of principals' professional practice have been shown to be related to principals' effects on student achievement. However, in both pilot years, variation in scores was limited, with most school leaders scoring in the upper third of the rating scale. As the FFL is implemented statewide, continued examination of evidence on its statistical properties, especially the variation in scores, is important.
|REL 2016104||Analysis of the stability of teacher-level growth scores from the student growth percentile model
This study, undertaken at the request of the Nevada Department of Education, examined the stability over years of teacher-level growth scores from the Student Growth Percentile (SGP) model, which many states and districts have selected as a measure of effectiveness in their teacher evaluation systems. The authors conducted a generalizability study using three years of data in mathematics and reading for nearly 370 elementary and middle school teachers from Washoe County School District in Reno, Nevada’s second-largest district. The study found that in mathematics, half of the variation among teachers’ annual growth score (median SGPs) was attributable to differences among teachers, while half was random or unstable. In reading, .41 of the variance in annual scores was attributable to differences among teachers, while .59 was due to random or unstable sources. More stable measures of effectiveness can be constructed by averaging multiple years of growth scores for a teacher, and the report provides stability estimates for averages of two, three, and four years of annual scores. The results from this study can also be used to examine the accuracy of judgments of teachers’ effectiveness that are based on these scores. Study results suggest that as states examine properties of their estimates of teacher effectiveness and consider their use in teacher accountability, they may want to be cautious in using such scores for teacher evaluation.
|REL 2016108||An educator's guide to questionnaire development
Educators have many decisions to make and it’s important that they have the right data to inform those decisions and access to questionnaires that can gather that data. This guide, developed by REL Central and based on work done through separate projects with the Wyoming Office of Public Instruction and the Nebraska Department of Education, provides educators with a process for developing questionnaires. Principals, superintendents, state-level personnel, or other school or district personnel can use this guide when they need to make a decision about an education policy or practice but (1) lack the information needed to make that decision and (2) lack a preexisting questionnaire that can be used to gather the information. This guide can help these educators collect information about attitudes, perceptions, or factual information to inform their decisions.
|REL 2016113||Where American Indian Students Go to School: Enrollment in Seven Central Region States
This report provides descriptive information about the location and native language use of schools in the REL Central Region with high enrollment of American Indian students, whether Bureau of Indian Education schools or non–Bureau of Indian Education high-density American Indian schools (schools with 25 percent or more American Indian student enrollment). Of the 208 schools with high American Indian student enrollment (33 BIE and 173 HDAI schools), 83% are located in the region’s rural areas. The schools located in counties with the highest concentration of Native North American language speakers are in South Dakota, Wyoming, and Colorado.
|REL 2016116||Teacher retention, mobility, and attrition in Kentucky public schools from 2008 to 2012
The purpose of this descriptive study was to determine the rates of retention, mobility, and attrition for classroom teachers in Kentucky public schools, as well as how those rates might vary by various teacher and school characteristics. The study looks at retention, defined as teachers returning to their same classroom ("stayers"); mobility, when teachers change schools within the school system ("movers"); and attrition, when teachers leave the system ("leavers") from one year to the next. The study used data on teachers collected by the Kentucky Center for Education & Workforce Statistics on every teacher employed in PK-12 public schools in academic years 2008/09, 2009/10, 2010/11, and 2011/12. Data on schools were obtained from the National Center for Education Statistics Common Core of Data. The study found that the Kentucky teacher workforce was largely stable across the study period (2008-2012). Most teachers (85.6 percent, on average) stayed in the same school from one year to the next, 6.0 percent moved to a different school, and 8.4 percent left the public school system. The study revealed some variation in rates based on select teacher and school characteristics. In particular, teachers with the fewest years of experience, teachers in urban schools, and teachers in schools where more students are eligible for free or reduced price lunch were retained at the lowest rates.
|REL 2016091||A Systematic Review of the Relationships Between Principal Characteristics and Student Achievement
This systematic review of the relationships between principal characteristics and student achievement was created for educators, administrators, policy-makers, and other individuals interested in a comprehensive catalogue of research on relations between principal characteristics and student achievement. It synthesizes what is known about associations between principal characteristics and student achievement; specifically it summarizes the studies, highlights the effects found by the studies, and describes the steps of the systematic review process used. Of the 52 studies included in the comprehensive review, only one used an experimental design, and a positive effect was found. An additional 38 quantitative and two mixed method studies provided evidence that some principal characteristics are positively correlated with student achievement. However, causal relationships could not be established. The remaining eleven qualitative studies mirrored the quantitative findings.
|REL 2016110||Professional Experiences of Online Teachers in Wisconsin: Results from a Survey About Training and Challenges
REL Midwest, in partnership with the Midwest Virtual Education Research Alliance, analyzed the results of a survey administered to Wisconsin Virtual School teachers about the training in which they participated related to online instruction, the challenges they encounter while teaching online, and the type of training they thought would help them address those challenges. REL Midwest researchers and Virtual Education Research Alliance members collaborated to develop the survey based on items from the Going Virtual! survey (Dawley et al., 2010; Rice & Dawley, 2007; Rice et al., 2008). Wisconsin Virtual School administered the survey to its 54 teachers, and 49 (91 percent) responded to the survey. The responses of the 48 teachers who indicated that they taught an online course during the 2013/14 or 2014/15 school year were analyzed for the report. Results indicate that all Wisconsin Virtual School teachers reported participating in training or professional development related to online instruction and that more teachers reported participating in training that occurred while teaching online than prior to teaching online or during preservice education. The teachers most frequently reported challenges related to students’ perseverance and engagement and indicated that they preferred unstructured professional development to structured professional development to help them address those challenges. Further research is needed to determine what types of professional development and training are most effective in improving teaching practice, especially related to student engagement and perseverance.
|REL 2016101||Stated Briefly: Redesigning Teacher Evaluations: Lessons from a Pilot Implementation
This “Stated Briefly” report is a companion piece that summarizes the results of another report of the same name. REL Northeast and Islands, in collaboration with the Northeast Educator Effectiveness Research Alliance and the New Hampshire Department of Education conducted a study of the implementation of new teacher evaluation systems in New Hampshire’s School Improvement Grant schools. While the basic system features are similar across district plans, the specifics of these features vary considerably by district. Further, district fidelity to the plans, as measured by the exposure of teachers to different features of the evaluation system, ranged from moderate to high. Finally, researchers identified several factors related to implementation: (1) capacity of administrators to conduct evaluations; (2) initial and on-going evaluator training; (3) the introduction and design of student learning objectives; (4) the professional climate of schools, including the support of the new system by teachers and evaluators.
|REL 2016102||A Descriptive Study of the Pilot Implementation of Student Learning Objectives in Arizona and Utah
Approximately 30 states are now adopting teacher evaluation policies that include student learning objectives (SLOs), which are classroom-specific student test growth targets set by teachers and approved (and scored) by principals. Today state and district leaders are trying to determine the appropriate level of guidance and oversight to provide in support of this work. This study describes results of the pilot implementation of SLOs in two states—Arizona (with 363 teachers) and Utah (with 82 teachers)—that were implementing SLOs with the same aims: to positively affect student achievement and to fulfill the state's required student-accountability component for teacher evaluations. Findings indicated that, in their SLOs, Arizona teachers tended to target student proficiency growth on vendor-developed tests, without including any specifics about instructional strategies, while Utah's pilot teachers (over half of them special education teachers) tended to define their own SLO-focused instructional strategies and/or use their own classroom-level tests or rubrics, with goals geared toward students demonstrating knowledge (through project completion) or a physical skill. Arizona teachers' end-of-year SLO scores from their principals varied, distinguishing high- and low-performing teachers, and teachers with higher SLO scores were also rated higher on classroom observations and student surveys. Conversely, SLO scores varied little in Utah's pilot, with 89 percent of teachers meeting expectations." (Utah's pilot teachers were not rated on other measures.) On end-of-year surveys, Utah pilot teachers generally perceived the SLO process as worthwhile and beneficial to their students and to their own professional growth; however, they did not perceive the SLO pilot as positively affecting their instruction or their knowledge of effective ways to assess students. (A low response rate precluded parallel survey analysis in Arizona.)
|REL 2016112||Stated Briefly: Suspension, Expulsion, and Achievement of English Learner Students in Six Oregon Districts
The "Stated Briefly" report is a companion piece that summarizes the results of another report of the same name. This study examines the rates of exclusionary discipline (i.e., suspensions and expulsions) among English learners and non-English learners in six diverse Oregon districts that serve a third of the state's English learner students. Using 2011/12 databases from the Oregon Department of Education, the study found that differences in suspension and expulsion rates between English learners and non-English learners were much larger in middle school and high school than in elementary school. Approximately 3 percent of English learners and non-English learners were suspended or expelled in elementary school. In middle school, 18 percent of English learners and 11 percent of non-English learners were suspended or expelled, while in high school 14 percent of English learners and 8 percent of non-English learners were suspended or expelled. In addition, English learners in high school were suspended for almost a full day more than non-English learners. Overall, the findings suggest that educators, parents, and community members should examine discipline policies and practices to see if they are being applied inequitably and consider extra supports for any student who is expelled or suspended.
|REL 2016100||The Examining Evaluator Feedback Survey
This report presents a survey tool, developed by REL Central at Marzano Research, designed to gather information from teachers about their perceptions of and responses to evaluator feedback. District or state administrators can use this survey to systematically collect teacher perceptions on five key aspects of evaluation feedback: (1) feedback usefulness, (2) feedback accuracy, (3) evaluator credibility, (4) access to resources related to feedback, and (5) teacher response to feedback. The survey tool was developed using an iterative process that included expert review, cognitive interviews, and a pilot study. Evidence regarding the reliability and validity of the survey tool is also reported.
|REL 2016099||Advanced Course Completion in Magnet and Comprehensive High Schools: A Study in Nevada's Clark County School District
The purpose of the study reported here was to explore the relationship between the type of high school attended (magnet versus comprehensive) and the likelihood of graduates having completed an advanced course, after accounting for students' prior achievement. In addition, the study examined the relationship between students' prior achievement and the likelihood of students completing an advanced course, and whether the nature of this relationship differs between different types of high schools. The REL West study team conducted a series of logistic regressions using records for 26,529 Clark County School District (CCSD) graduates from 43 high schools in 2011 and 2012. Student achievement prior to entering high school was measured using each student’s grade 8 ELA and mathematics scores from Nevada's Criterion-Referenced Test (CRT).
The results indicate that among students with similar levels of prior achievement, students have a greater likelihood of completing an honors English language arts course if they attend a magnet high school than if they attend a comprehensive high school, but there is no statistical difference between school types in the likelihood of students completing an honors mathematics course. Also, there is a stronger relationship between past achievement and the likelihood of completing an Advanced Placement course for students in the comprehensive high schools compared to those in magnet high schools. However, this was not the case for the relationship between past achievement and the likelihood of completing an honors course.
|REL 2015098||The Achievement Progress of English Learner Students in Arizona
To understand the learning trajectories of the growing numbers of English learner students in the West, especially those who struggle to pass state English language arts and math content tests, this study followed three cohorts of English learner students in Arizona over six school years, 2006/07 through 2011/12, to assess their progress in English proficiency and their academic progress in English language arts and math content knowledge. Key findings from this study include: More than 90 percent of Arizona’s English learner students scored at or above the required level for reclassification as fluent English proficient students over a period of six school years. Their cumulative passing rate was highest for the English language proficiency test, followed by academic tests in English language arts and math. English learner students who were eligible for special education services had the lowest passing rates on all three tests. In general, English learner students in higher grades had lower cumulative passing rates on all three tests than students in lower grades. Educators might consider devoting additional attention to improving teaching practices and support services to help the English learner student subgroups with the poorest performance: students in higher grades, students eligible for special education services, students eligible for school lunch programs, and male students.
|REL 2015095||Comparing Success Rates for General and Credit Recovery Courses Online and Face to Face: Results for Florida High School Courses
This report describes the results of a REL Southeast study comparing student success in online credit recovery and general courses taken online compared to traditional face-to-face courses. Credit recovery occurs when a student fails a course and then retakes the same course to earn high school credit. This research question was motivated by the high use of online learning in the Southeast, particularly as a method to help students engage in credit recovery. The data for this study covered all high school courses taken between 2007/08 and 2010/11 in Florida (excluding Driver’s and Physical Education). The study compares the likelihood of a student earning a C or better in an online course as compared to a face-to-face course. Comparisons for both general and online courses include those courses taken for the first time and credit recovery courses. The results show that the likelihood of a student earning a grade of C or better was higher when a course was taken online than when taken face-to-face, both for general courses and credit recovery courses. Most subgroups of students also had higher likelihood of success in online courses compared to face-to-face courses, except that English language learners showed no difference in outcomes when taking credit recovery courses online. However, it is not possible to determine whether these consistent differences in course outcomes are attributable to greater student learning, other factors such as differences in student characteristics, or differences in grading standards.