Normally, when somebody hears about an evaluation of an education program, they reasonably assume the evaluation will tell them whether the program is working or not. When reading an evaluation report, policymakers, parents, and educators hope the evaluation will tell them if the program is helping the participating students. These seem like obvious, uncontroversial points.

On Monday, June 4, researchers from NC State released a working paper, An Impact Analysis of North Carolina’s Opportunity Scholarship Program on Student Achievement, which purports to be an evaluation of North Carolina’s largest private school voucher program. The authors enthusiastically publicized and distributed the report, making sure to provide advance copies to media organizations and a pro-voucher advocacy group. The report has been highlighted by all of the state’s major media outlets.

But there’s a problem: the report fails to tell us whether the Opportunity Scholarship program is working. The researchers’ efforts tell us nothing about whether accepting an Opportunity Scholarship will help or harm a student’s education.

The Heritage Foundation’s report, Focusing on School Safety After Parkland, was published following the mass shooting at Marjory Stoneman Douglas High School. The report highlights a handful of cherry-picked, pro-gun research but depends mainly on anecdotal evidence to support its findings that teachers should be armed, schools hardened, family and social services expanded, and discriminatory discipline practices de-emphasized. Yet the report’s credibility is harmed by the limited and biased presentation of research evidence and the omission of educator and student perspectives. The problems with supporting research and the questionable methods prevent generalizability and call the paper’s usefulness into question. The end result is a poorly refereed, one-sided defense of positions taken by gun advocates. That said, the report does provide conservative perspectives that can serve as a springboard for broader discussions about the impact of value-laden policies on promoting a broader sense of safety in schools.

A report from the Cato Institute opens with Horace Mann’s well-known conviction that public schools are the bedrock of a democratic society – a public good that should be made available to all. Yet the report, Is Public Schooling a Public Good? An Analysis of Schooling Externalities, improperly conflates the civic and economic definitions of a public good. Although the report begins with Mann’s vision of the role of public schools as building a better society, it then misleadingly shifts the analysis to the economic value of public schools as a market-based “good” like steel or corn. The report relies on a false equivalence of the civic and economic definitions of a public good to advance a proposal for de-funding public schools and introducing a nationwide education savings account (voucher) program. While there is extensive research on the educational purposes of schooling, the Cato report’s limited review of this literature consistently misrepresents the meaning, scope and implications of this literature. The result is a portrayal of public schools as “agents of harm” for what appears to be an ideologically driven thought experiment. Even for those who might be in favor of vouchers, the report’s imbalance, flawed logic and limited research base render the report of no use to policymakers.

A study published by the American Enterprise Institute seeks to illuminate how students in career and technical education (CTE) programs demonstrate higher high school graduation rates, overall educational attainment, and earnings than students in academic programs whose previous test scores are similar. Although those students who take many CTE courses by 12th grade tend to have significantly lower test scores, this study finds that these students have higher noncognitive skills (e.g., attendance and homework time). The most remarkable feature of this study is the broad array of indicators it compiles, including how much effort students exhibit on a routine task (e.g., a long and boring survey in school), and teacher reports of student effort. Using such data, the study suggests that CTE may improve attainments by improving noncognitive skills. The key implication is that, instead of the narrow policy focus on academic skills, educators need to consider how to improve students’ other skills to improve education and job outcomes. While this implication is reasonably drawn from the study, educators need a clearer interpretation of these “noncognitive skills” and whether they are persistent attributes or highly changeable behaviors. Overall, however, the study presents a strong empirical analysis of a strong dataset and should prove useful for policymakers.

NCTQ released its 2018 review of U.S. teacher preparation programs. Employing open-records requests and online searches, the report ranks 567 graduate teacher preparation programs, 129 alternative route programs, and 18 residencies on practice, knowledge and admissions. The report seeks to determine if the teacher preparation programs are aligned with NCTQ’s standards. Such alignment, the report insists, will produce teachers “not only ready to achieve individual successes, but also to start a broader movement toward increased student learning and proficiency.” However, the report determines that most programs are not aligned with its standards. Accordingly, it finds “severe structural problems with both graduate and alternative route programs that should make anyone considering them cautious.” However, the report has multiple logical, conceptual, and methodological flaws. Its rationale includes widely critiqued assumptions about the nature of teaching, learning, and teacher credentials. Its methodology, which employs a highly questionable documents-only evaluation system, is a maze of inconsistencies, ambiguities, and contradictions. Further, the report ignores accumulating evidence that there is little relationship between graduates’ classroom performance and NCTQ’s ratings. Finally, the report fails to substantively account for broad shifts in the field of teacher education that are nuanced, hybridized, and dynamic. Regrettably, the report exacerbates the dysfunctional dichotomy between university programs and alternative routes and offers little guidance for policymakers, practitioners, or the general public.

A report from NCTQ begins with nine goals purportedly based on the “best available research evidence” about teacher quality. Yet neither this report nor its companion, which describes the original development of the goals, cites any research evidence. The report also uses the terms “teacher quality” and “teacher effectiveness” (on raising test scores) interchangeably. The report assumes reader buy-in to its goals, to its focus on test scores, and to its assumption that “great teachers” have an “outsize impact” on students’ learning and lives. Grounded in these assumptions, the report highlights examples of “leading state work” in 37 policy areas related to teacher quality, aiming to hold up these state policies as exemplars for other state policymakers to replicate. Despite its intentions, the report has multiple flaws that undermine its validity and usefulness. It offers no explanation about how the 37 best practices were selected in the first place and no justification for its selection of “leading” policy work, some of which has occurred in states that have consistently been low performers on national assessments. In addition, the report offers no evidence to support its approach and makes no references to the nuanced and complex research literature in this area. The report focuses primarily on human capital policies that explicitly target the qualifications and evaluation of the teacher workforce. This ignores the growing consensus that many other factors matter in the production of students’ learning, including supports that help teachers succeed, school contexts and cultures, state and regional labor markets, teachers’ relationship-building capacities, and the social organization of teachers’ work. In the end, the report is of limited use.

A report released by the University of Arkansas Department of Education Reform contends that charter schools produce more achievement per dollar invested, as compared to public schools. This newest report is focused on city-level analyses in eight US cities (Atlanta, Boston, Denver, Houston, Indianapolis, New York City, San Antonio, and Washington D.C.) and uses cost effectiveness and Return on Investment (ROI) ratios. It concludes that charter schools deliver a weighted average of an additional 4.34 NAEP reading points and 4.73 NAEP math points per $1000 invested. The report also argues that that charter schools offer an advantage of $1.77 in lifetime earnings for each dollar invested, representing a ROI benefit of 38%. However, there are a variety of methodological choices made by the authors that threaten the validity of the results. For example, the report uses revenues rather than actual expenditures – despite well-established critiques of this approach. The report also fails to account for the non-comparability of the student populations in charter and comparison public schools. Three other problems also undercut the report’s claims. First, even though the think tank’s earlier productivity report included a caveat saying that causal claims would not be appropriate, the new report omits that caution. Second, the report’s lack of specificity plagues the accuracy and validity of its calculations; e.g., using state-level data in city-level analyses and completely excluding race and gender. Finally, the authors again fail to reconcile their report with the extensive literature of contrary findings.

Reports from Bellwether and New America each review the current state of early childhood education (ECE) teacher preparation programs at the community college and university levels. Both raise awareness about the challenges facing the ECE workforce and educational institutions in dealing with the recent policy push to increase the required educational credentials for pre-K teachers. Specifically, these policies elevate the requirement from the Child Development Associate certificate to an associate’s degree or to a bachelor’s degree with early childhood specialization. The reports also illustrate innovative practices that some states and colleges are implementing to improve the quality, flexibility, mobility, and diversity of ECE teachers. They end with policy recommendations for states, colleges, and ECE leaders. The methods consist of simple literature reviews (which generally rely on think tank white papers, as opposed to peer-reviewed journal publications). The reports also incorporate informal interviews with ECE stakeholders and leaders, but the methodology and selection procedures used are unclear. Notwithstanding these limitations, the analyses appear sound and the conclusions and recommendations made are generally reasonable, supported by the relatively limited data available. The longer report from Bellwether on the role of community colleges appears useful for policymakers and the public, due to its depth and effectiveness in presentation. The report from New America is too short and rather awkward in presentation, which limits its effectiveness.

A report from the American Enterprise Institute (AEI), Do Impacts on Test Scores Even Matter? Lessons from Long-Run Outcomes in School Choice Research, examines whether student achievement scores on math and English language arts tests align with “long-run” attainment outcomes such as high school graduation rates, college enrollment, and college graduation rates. Drawing on a systematic review of the literature, it concludes that the impacts of school choice programs on test scores are not well connected to such attainment outcomes, which are presented as more positive. This review considers two issues with the report: consistency and evidence. Regarding consistency, the report’s suggestion that achievement scores should play a smaller role in determining the efficacy of school choice models represents a stunning effort to move the goalposts in search of new justifications for supporting their preferred policies. After decades of pro-school-choice research and advocacy promoting test score comparisons with public schools as the primary measurement for evaluating school choice models (e.g., charters and school vouchers), the AEI report now suggests that less attention be given to these learning outcomes. Regarding evidence, the AEI report is riddled with numerous internal inconsistencies in its discussion and treatment of a set of studies that were selected by questionable methods. In view of the 180-degree turn based on questionable evidence, the report — despite the authors’ assertions — is of little use to policymakers.

A report, Charter Schools and the Achievement Gap, finds that, though charter schools on average perform no better than traditional public schools, urban “no-excuses” charter schools—which often use intensive discipline to enforce order—demonstrate promising results. It recommends that these schools and their practices be widely replicated within and outside of the charter school sector. We find three major flaws with this conclusion. First, the report’s recommendations are based solely on the academic success of these schools and fail to address the controversy over their use of harsh disciplinary methods. No-excuses disciplinary practices can contribute to high rates of exclusionary discipline (e.g., suspensions that push students out of school) and may not support a broad definition of student success. Second, the recommendation that schools replicate no-excuses practices begs the question of what exactly should be replicated. It does not confront the lack of research identifying which school practices are effective for improving student achievement. Third, the report does not address many of the underlying factors that would allow no-excuses schools and their practices to successfully replicate, such as additional resources, committed teachers, and students and families willing and able to abide by these schools’ stringent practices. Thus, while the report is nuanced in its review of charter school impacts, it lacks this same care in drawing its conclusions—greatly decreasing the usefulness of the report.

This report offers political advice to school superintendents. The authors draw solely on a collection of anecdotes from superintendents of large urban districts to offer a narrative replete with maxims, aphorisms, and pithy advice. Superintending is, in their view, a continuous string of almost unavoidable conflicts, so the reader is led through internal groups (central office, school board, unions, teachers and principals) and external groups (foundations, businesses and state politicians) and advised on how to cultivate support and deal with opponents. There is no research design, literature review, or systematic data collection. The report relies heavily on the authors’ experiences, yet neither author appears to have worked in a K-12 district or served as a superintendent. Moreover, the advice that follows is often contrary to contemporary professional practices. The report improperly generalizes from more politicized large urban districts and does not consider how the favored approaches may limit vision, flexibility and effectiveness. In the end, the report perpetuates antiquated methods and does not advance our knowledge.

This review examines a report evaluating four years of implementation of the 2010 Teacher Incentive Fund (TIF) to determine whether pay-for-performance (PFP) bonuses to teachers and principals improve student achievement. The report notes that the goal of PFP is to motivate improved educator performance and to attract and retain more effective teachers, thereby increasing student achievement. The report found that PFP marginally increased student test scores in reading and math by year two and that it was associated with slightly higher classroom observation ratings; it also found that classroom observation ratings were not associated with student test scores and that PFP had no effect on filling teacher vacancies. Problematic issues in the report involve combining non-comparable sites, employing different outcome measures, and generating a misleading cost-effectiveness analysis. Although several thorough analyses are presented, the data do not support the report’s suggestion that PFP holds promise to address achievement disparities. Overall, because of the report’s incorporation of a polyglot of outcome measures of questionable validity, this review finds that the report lacks utility for policymakers.

A report from the Fordham Institute investigated the impact of a reform in the School District of Philadelphia that eliminated suspensions for certain low-level misbehaviors. The report considered whether the policy change was associated with any of the following: (a) district-wide out-of-school suspension rates, (b) academic and behavioral outcomes for students (looking separately at students who had a record of prior suspensions and those with no prior suspensions), and (c) racial disparities in suspensions. While the report concluded that the reform was a failure, the actual results were mixed, with the positive trends for students who were earlier suspended being much stronger in magnitude than evidence of negative outcomes for students who were not. A strength of the report is the use of advanced statistical methods and a longitudinal dataset to answer the questions of interest. However, the report is plagued by logical fallacies, overly simplified interpretations of findings, and inflammatory language. Moreover, the report uses misleading causal (“consequences”) language in the title and to describe study results, even though the study design is limited by unmeasured confounding factors and inappropriate comparison groups. Thus, while the analyses upon which the report is based have some technical merits, the narrative seems more of an attempt to advance a political agenda opposed to the reform studied than to improve understanding of complex policy issues.

Balancing Act explores the racial imbalance of schools and neighborhoods across the nation, finding that the racial composition of schools often reflects the surrounding neighborhoods. It suggests that racially concentrated schools are the result of residential segregation and how school district boundaries are drawn to separate populations. The authors also posit that charter schools, which are freed from the constraints of traditional boundaries, can interrupt the school-housing relationship. Yet the report finds that charter schools are, on average, more racially imbalanced than other public schools. The report deserves credit for tackling the complicated relationship between school and housing segregation, and for bringing renewed attention to issues that are often considered separately. Still, a shaky grounding in the research and law, and in the methods that inform school desegregation, limits the report’s usefulness when it comes to policy and practice. Moreover, crucial methodological decisions, like how neighborhoods are defined or why metro areas are treated the same as rural ones, either are not fully explained or lack a research-based rationale. This weakness undermines the utility of the newly created database—which the authors view as a vital contribution of the report. In short, Balancing Act represents a missed opportunity to accurately explore the nexus between school and housing segregation.

A report from New America argues that online programs, appropriately structured, have the potential to professionalize many early childhood (EC) teachers. The report contends that online bachelor’s programs offer more flexible and financially accessible pathways for pre-K teachers to obtain a degree, thereby propelling these teachers towards higher quality practice and higher salaries. The report presents a perspective on what a professionalized pre-K workforce should entail, recognizing the significant knowledge and skills required of these teachers, and equating raising pre-K teacher credential standards with professionalization, and therefore, quality. While the report draws attention to some of the unique considerations necessary for EC workforce policy, it underplays a number of critical issues in professionalizing the field. It lacks adequate treatment of the EC workforce’s economic realities that place them at the poverty line, regardless of credential or degree. It presents testimonials rather than evidence of outcomes for online programs. It makes pre-K teachers responsible for closing the pay gap and for upgrading their low professional esteem. Finally, though recognizing the importance of high-quality EC programs, it leaves the term under-analyzed at best. Unfortunately, without a nuanced discussion of these issues, the report’s usefulness is limited.

The policy brief Student Centered State Funding: A How-to Guide for State Policymakers attempts to illustrate and explain to state policymakers how to design a state school funding system whereby all state and local dollars may flow freely, by parental choice, following children to local district, charter or private schools. In particular, the brief seeks to provide state policy guidance on setting specific funding levels for each child. These “money follows the child” approaches provide a system that eases the process of funding an assortment of school choice programs. Unfortunately, this colorful multi-page infographic suffers from three major shortcomings as a guide for state policymakers: First, the brief advances the false dichotomy that advocates for state and district school finance systems to focus on funding the child, not funding the essential institutions that serve those children. This dichotomy wrongly promotes the idea that there is no benefit to children of equitably and adequately financing educational institutions. Second, the brief is based on overly simplistic, frequently misrepresented, and often outright incorrect versions of the status quo, including overbroad mischaracterizations of how schools are currently financed. Third, the details of the brief’s proposals and espoused benefits are entirely speculative and unsubstantiated, in some cases simply made up and in other cases supported only by insular and circular self-citation to previous work that itself cites no strong empirical support. For these reasons and others, the report is of absolutely no value to policymakers for informing the design of state school finance systems or school district resource allocation formulas.

In 2010, Facebook CEO Mark Zuckerberg announced, with great fanfare, a $100 million challenge grant for the support of a series of reforms in Newark, NJ schools. The two reports reviewed here are the first attempt at a comprehensive assessment of the impact of the Zuckerberg donation on student achievement. As such, the reports, which were written by a team of economists, have garnered significant attention. The reviewers evaluate the reports and their potential contribution to education policy research.

The Every Students Succeeds Act (ESSA) directs states and districts to identify equity gaps in students’ access to excellent educators and transformative school leaders. States are encouraged to use Title II funds strategically in order to identify and remedy these gaps. A new report from The Education Trust draws on ESSA documents and state teacher equity plans to provide guidance to state leaders, including some sound advice—but with significant omissions. The report does not engage with thorny issues around alternative pathways into teaching, and it largely skirts issues around incentives for supporting teacher recruitment and retention in hard-to-staff schools. The report also does not consider what attracts teachers into the profession and into particular school environments. Likewise, the report fails to draw on the explicit remedies sought by ESSA to link high-quality leadership with strong teacher recruitment and retention. Instead, the report casts the teacher equity problem primarily in terms of labor supply shortages and treats teachers like interchangeable widgets. Relying heavily on advocacy sources, it misses an opportunity to unpack the root causes of the teacher retention problem, particularly the corrosive impact of past federal and state policies on the teaching profession. The report does not help state leaders understand how they might build incentives and cultures that draw strong teachers into high-need schools, and they will thus be left with an incomplete and insufficient set of tools for ensuring that all students have equitable access to excellent educators.

In this report, a school choice advocacy group presents results from its survey of K-12 parents within and across the public and private sectors. They report that parents are highly satisfied with voucher and tax credit scholarship programs and suggest that the findings support the expansion of school choice programs. However, these and other findings are consistent with research showing general parental satisfaction with their children’s schools. One underemphasized finding is that substantial proportions of public school families—the largest constituency of K-12 parents in Indiana—did not participate in private school choice programs because they are happy with their current schools and want to support public schools. The survey and analysis fall short in four ways. First, three incompatible data collection methods were used to collect small samples of non-representative groups of Indiana parents. Second, the statistical analyses are too weak to draw clear conclusions. Third, while organized like a conventional research study, the report appears to be designed to advance an agenda rather than provide substantive answers to important policy questions. Finally, the report provides little new information about parents’ experiences with their children’s schools. Thus, the report does not add to our knowledge about school choice in Indiana or provide much useful information about public support for school choice programs.

This report compares average rates of frequent teacher absence (more than 10 days) for teachers with and without union or union-like contracts in traditional public schools and charter schools. The study’s rationale is that such absences substantively harm students and cost taxpayers billions of dollars. It finds that teachers contractually allowed more absences are frequently absent more often than teachers allowed fewer absences. Based on these averages, the report concludes that the contracts result in non-beneficial, or uncalled-for absences, rather than absences for legitimate reasons, and it recommends that contracts be made less generous. However, the report lacks support for its major claims, ignores known discrepancies in data, uses cited resources in highly selective ways, ignores large bodies of contradictory research, and draws unwarranted conclusions. In addition, the report’s idiosyncratic use of the term “chronic absenteeism” misrepresents the data and, along with its use of graphics, appears intended to create a national alarmist picture unwarranted either by the data or by other research. Accordingly, the report appears to be an effort to generate numbers and charts useful in discrediting teachers as irresponsible shirkers.

This report provides an extensive analysis based on the most comprehensive dataset ever assembled for school closure research, including 1,522 low-performing schools that were closed across 26 states between 2006 and 2013. The report finds that even when holding constant academic performance, schools were more likely to be closed if they enrolled higher proportions of minority and low-income students. It also finds test score declines, relative to the comparison group, for two groups of students displaced by closures: those who transferred to schools with a prior record of relatively lower test-score performance and those who transferred to schools with equivalent past test-score performance. The slightly less than half of students who transferred to higher performing schools showed academic improvement relative to their matched peers. In general, although the reviewers found this to be a careful and rigorous study, they see a few missed opportunities. First, the report’s focus on some tenuous analyses (involving pre-closure transfers) obscures its most important findings – disproportionality in school closures and inadequate numbers of higher quality receiving schools, leading to performance declines for most. Second, the reviewers are concerned about statistical modeling choices and matching challenges that may threaten the validity of subgroup analyses (charter school students). Finally, the reviewers would have liked to see the report acknowledge the inescapable moral dimensions of school closure: The communities most likely to be negatively affected are unlikely to have participated in closure decisions.

A report by researchers at the University of Arkansas concludes that charter schools in New York City are not fairly funded, in comparison to district schools. The report asserts that this inequity is especially big for charter schools that are not co-located in public schools; and describes expenditure patterns across schools that show no clear differences between charter schools and district schools. This review of the report raises several concerns. Perhaps most importantly, the report simply does not attempt a rigorous comparison across schools. It assumes that any differences in student characteristics across charter and district schools are trivial, therefore concluding that raw, unadjusted funding amounts are sufficient for assessing fairness. It does not undertake any sensitivity testing to identify the precision of its estimates, nor does it investigate in detail what the optimal amount of funding should be for charter schools that are not co-located in public school buildings. Finally, New York City has significantly reformed its funding regulations for charter schools since the data upon which the report is based were collected, rendering the report’s estimates policy-irrelevant.

The Center for Research on Education Outcomes (CREDO) assessed the impact of different types of charter school-operating organizations on student outcomes in 24 states, plus New York City and Washington, D.C. The study finds that students in charter schools display slightly greater gains in performance than their peers in traditional public schools, especially students in charter schools operated by certain types of organizations. CREDO’s distinctions between organization types are, however, arbitrary and unsupported by other research in the field. This raises concerns about the practical utility of the CREDO findings. In addition, CREDO researchers made several dubious methodological decisions that threaten the validity of the study. A number of the problems flagged by the reviewers of this study have been raised in reviews of prior CREDO studies. Specifically, CREDO studies tend to over-interpret small effect sizes; fail to justify the statistical assumptions underlying the group comparisons made; and not take into account or acknowledge either the large body of charter school research beyond their own work or the limitations inherent in the research approach they have taken. Because these problems have gone unaddressed in the present study, and because the CREDO researchers have compounded them by creating a confusing and illogical charter organization classification system, the report is of limited if any value in policymaking. Readers should review the report with care.

This report asserts that tax credit scholarship programs, that distribute scholarships to students via Scholarship Tuition Organizations (STOs), have saved state treasuries between $1.7 and $3.4 billion dollars since 1998. The report argues that these programs are able to realize fiscal savings as a result of students leaving public schools and entering private schools (defined as “switchers”). The report claims that the percentage of students leaving public schools, coupled with the offset of variable per-student costs that districts no longer need to expend, have resulted in the sizable financial savings for state governments. This review questions the method used to estimate the percentage of switcher students across these various programs, and examines how the report determines variable cost fluctuations for each student that leaves public schooling. Since no STO programs require officials to track data on which students transfer out of public schooling into private, these lax accountability standards have led the report author to estimate fiscal savings using conjecture. Instead of following students, they interpreted broad population changes to STOs. Consequently, the results of this report do not provide an acceptable causal conclusion for policymakers. Suggestions for more extensive accounting procedures along with more nuanced methodologies for calculating true variable student costs are discussed.

A report from EdChoice documents two staffing trends in public schools. After a temporary pause during the Great Recession, school staffing in the U.S. resumed an upward trajectory, and hiring tilted toward non-teaching personnel as compared to teachers. The report concludes that staffing growth outpaces enrollment growth and that there has been no corresponding increase in student performance. The report then recommends increasing teacher pay, at the expense of non-teaching staff, and school choice. While the data on staffing trends are obtained from NCES publications, the report’s discussion of inputs, outcomes, and policy approaches is poorly grounded, leading to unsupported conclusions and policy prescriptions. It does not examine why there has been a staffing surge or whether it reflects a valid use of personnel. It evaluates the effectiveness of school staffing changes using concurrent achievement and finance measures, and it fails to acknowledge that educational outcomes have steadily improved, even though any benefits of staffing increases will be lagged and only show up gradually over time. The report presents no logical relationship between staffing trends and the solutions proposed. As a result, the report is irrelevant and is devoid of any important policy implications.

Two reports contend that the introduction of school choice can promote economic development in economically distressed urban areas. The first report, published by EdChoice, presents a case study of a charter school that has, according to the report, contributed to the economic development of the city of Santa Ana, California. The second report, published by the American Enterprise Institute, presents a proposal for a hypothetical voucher-like program that, if implemented, would purportedly spur economic development in high-poverty neighborhoods by luring higher income families into those neighborhoods. This review explains that both reports overlook significant bodies of relevant research literature and make unsupported claims that rely on flawed logic and data. The EdChoice report fails to collect and analyze data related to the report’s causal assertion that economic development in Santa Ana resulted from the establishment of the charter school. The American Enterprise Institute report’s claims about the benefits of the proposed program to publicly fund private schooling are unsupported by existing research. We conclude that these reports offer little useful guidance for policy or practice.

While inequalities in school funding resulting from state and local policies have long been a source of concern to education researchers and policymakers, a report from the Center for American Progress examines a source of educational inequality that receives less attention: private fundraising by parents. It focuses on the 50 Parent-Teacher Associations (PTAs) that raised the most money in 2013-2014, with two main findings. First, the PTAs raising large amounts were located in schools and districts with low rates of student poverty. Second, while a PTA in a high-poverty community may raise only a few hundred dollars, PTAs in this sample raised hundreds of thousands of dollars each year. Using case studies, the report considers district regulation of private fundraising. This review concludes that the report’s findings about the scope and beneficiaries of private fundraising are credible and important—showing the impact successful PTAs can have. However, the focus on a small number of schools and districts, a lack of attention to school and community context, and problems with the case study design limit the report’s overall relevance. In addition, it is important to note that most funding inequalities arise at the state level; funds raised by parents represent only a minute portion of overall school spending. Nevertheless, the report’s recommendations, especially in support of equity grants, will be useful to district-level policymakers.

A report offers a how-to guide for reform advocates interested in removing communities’ democratic control over their schools. The report explains how these reformers can influence states to use the Every Student Succeeds Act (ESSA) Title I school improvement funds to support a specific set of reforms: charter schools, state-initiated turnarounds, and appointment of an individual with plenipotentiary authority over districts or schools. While the report acknowledges that the research evidence on the effectiveness of these reforms as school improvement strategies is limited, it uses a few exceptional cases to explain how advocates seeking to influence the development of state ESSA plans can advance them anyway. As this review explains, support for the effectiveness of these approaches is simply too limited to present them as promising school improvement strategies. The report omits research that evaluates the models relative to other school reform initiatives, and it fails to take into account the opportunity costs of pursuing one set of policies over another. It also relies on test-score outcomes as the sole measure of success, thus ignoring other impacts these strategies may have on students and their local communities or the local school systems where they occur. For these reasons, policymakers, educators and state education administrators should be wary of relying on this report to guide them as they develop their state improvement plans and consider potential strategies for assisting low-performing schools and districts.

A report from the Department of Education Reform at the University of Arkansas examines the association between out-of-school suspensions and student test scores. Using dynamic and multilevel regression modeling of six years of school records from all K-12 public schools in Arkansas, the paper purports to estimate a causal relationship between exclusionary discipline and academic performance. It concludes, in contrast to prior work, that the number of days of suspension a student receives has a positive relationship to math and language arts test scores. However, the outcomes (end-of-year test scores) are measured at least a full year after the hypothesized causal factor (days suspended), and there is no control for days suspended in the year the test was taken. Consequently, the results do not reflect missed instructional time for the tested material or other associated mechanisms through which suspension might adversely affect test scores. The analyses also control for a large number of infractions that are strongly related to days of out-of-school suspension, which may produce biased results. The findings also have weak face validity, in light of the weight of evidence suggesting that exclusionary discipline and school absences have adverse effects on test scores, GPA, grade retention, and dropping out. For these reasons, the reviewers caution that this paper is not useful for policymakers.

A response from the authors is below the review.

The reviewers of “Understanding A Vicious Cycle: Do Out-Of-School Suspensions Impact Student Test Scores?” respond to criticisms of their review in two parts, Part I by Daniel Losen and Part II by Brea Louise Perry.

A common argument leveled against charter schools is that they attract the most motivated and intelligent students from already struggling public schools. Marcus Winters seeks to examine this claim, known as “cream-skimming,” by comparing the performance of New York City’s (NYC) charter middle schools with a set of traditional selective public middle schools, which admit students on the basis of prior performance. Findings indicate that, controlling for student characteristics, charter schools perform no differently in English Language Arts (ELA) and significantly better in math than selective schools. Based on this, the report concludes that the success of NYC charter schools cannot be explained by cream-skimming. While on its face this conclusion may seem logical, the report suffers from two primary flaws. First, it assumes that selective school applicants are higher performing and more motivated than charter school applicants. This is unlikely to be the case because all students are required to apply to traditional middle schools in NYC, while applying to a charter school requires navigating an additional application process. Second, the report relies on a single year of data to make comparisons of ill-defined and inappropriate outcomes—an approach that does not address either the question of cream-skimming or charter school success. As an evaluation of cream-skimming in charter schools, this report misses the mark.

This recent Heritage Foundation report argues that Title IX does not require schools to protect students from discrimination on the basis of gender identity. It criticizes the Obama Administration for issuing guidance and enforcing Title IX to protect transgender students, and it urges the Trump Administration and courts to keep gender identity protections out of federal laws. Oddly, transgender youth are at the center of, and yet somehow go wholly unexamined in, this report on gender identity policy. The report never acknowledges or addresses (a) legal opinions that gender identity discrimination is a form of sex discrimination; (b) the near-consensus within the medical, scientific, and educational communities concerning how transgender students should be treated; and (c) other research or literature shedding light on the appropriate care for and education of transgender youth. Additionally, the report erroneously asserts that transgender-inclusive policies will embolden men to enter women’s facilities to assault or abuse them. What is entirely missing from this report — and what policymakers and educators urgently need — is guidance in an area that may be new or unfamiliar to them. Fortunately, many states and districts have adopted positive gender-identity-related laws, policies, and practices that answer questions and serve as useful guidance for other jurisdictions about how to successfully integrate transgender students in schools.

The report reviewed here compares Wisconsin student test score performance for the 2015-16 school year across public schools, charter schools and private schools participating in one of the state’s voucher programs. Comparing a single year’s test scores across school sectors that serve different student populations is inherently problematic. The report uses linear regression models to attempt to adjust for these differences and make what the authors claim are “apples to apples” comparisons. Based on these comparisons, the report concludes that charter schools and private schools participating in the voucher programs are more effective than traditional public schools. Unfortunately, the limited nature of available data undermines any such causal conclusions. The inadequate and small number of school-level variables included in the regression models are not able to control for important confounding variables, most notably prior student achievement. Further, the use of aggregate percent proficient metrics masks variation in performance across grade levels and makes the results sensitive to the (arbitrary) location of the proficiency cut scores. The report’s description of methods and results also includes some troubling inconsistencies. Thus, while the report does present important descriptive statistics about test score performance in Wisconsin, it cannot provide answers for those interested in determining which schools or school choice policies in Wisconsin are most effective.

A report by the American Enterprise Institute (AEI), Differences by Design?, compares differences in approaches and demographics between and among charter school models and local “traditional public schools.” Using three national data sets, the report effectively captures the national universe of charter schools. It empirically demonstrates that cream-skimming occurs and that charters segregate by income, special education, race and ethnicity, in that different demographic groups attend different types of charter schools. Charter schools, the authors contend, provide differentiated and “innovative schooling options” through varied academic models that cater to, and ultimately reflect, parental choices for their children. The resulting de facto segregation is presented as a benign byproduct of beneficial choices differentially associated with different racial and ethnic groups. They contend this is “in line with a properly functioning charter sector.” Unfortunately, the report does not demonstrate familiarity with the research on parent decision-making or with the extensive research suggesting that charter schools are not particularly innovative in the curricular or instructional options. Despite what the report claims, traditional public schools do, in fact, offer various academic model specializations like the ones offered by the charter schools. Ultimately, the report’s dismissive characterization of de facto segregation in charters, as a benign byproduct of parental choice, is at odds with the purpose and aims of equitable public education.  

Bellwether Education Partners’ report, A New Agenda, calls for a “rational” and “rigorous” research agenda for teacher education. Although the report’s rationale is not fully explicated, it asserts that programs are “blindly swinging from one popular reform to the next” and that decades of input- and outcome-based research has failed to improve teacher education. Instead, the report calls for “rapid cycle evaluations.” Regrettably, this depiction of past research includes mischaracterizations and also omits a wide swath of relevant literature about teacher education. Further, the report does not adequately explain what “rapid cycle evaluations” would entail or how they would work to improve teacher preparation program design, nor does the report offer a research foundation for this approach. The report also fails to recognize the socio-political context of teacher education, wherein programs are often left scrambling to meet competing accountability expectations. A New Agenda leaves practical questions unanswered, muddies the waters about promising research avenues, and ignores important bodies of literature in teacher education. Ultimately it represents a missed opportunity to offer guidance to either policymakers or institutions. 

Based on a review of GPA and SAT/ACT requirements at 221 institutions in 25 states, a new report from the National Council on Teacher Quality (NCTQ) recommends that states, institutions of higher education, and the Council for the Accreditation of Educator Preparation (CAEP) maintain or establish a higher bar for entry into teacher preparation programs. The NCTQ report suggests that boosting teacher candidate entry requirements in ways they advocate would significantly improve teacher quality in the U.S.. Yet the report does not provide the needed supports for its assertions or recommendations. In addition, the report makes multiple unsupported and unfounded claims about the impact on teacher diversity of raising admissions requirements for teacher candidates, about public perceptions of teaching and teacher education, and about attracting more academically able teacher candidates. Each claim is based on one or two cherry-picked citations while ignoring the substantial body of research that either provides conflicting evidence or shows that the issues are much more complex and nuanced than the report suggests. Ultimately the report offers little guidance for policymakers or institutions. 

Philanthropic involvement in K-12 education is growing, and it increasingly shapes the direction of reforms pursued throughout the country. A recent report from the NewSchools Venture Fund offers a thought experiment on how philanthropists can make a “big bet” over the next decade on innovative schools—a broad category that generally includes schools with a high degree of education technology use and so-called personalized approaches to learning that likely utilize digital platforms. Unfortunately, the report fails to provide a meaningful examination of research or a thorough basis for its recommendations. This critique focuses on six key concerns regarding the report: it fails to consider human capital constraints or to sufficiently consider obstacles confronting classroom technology usage, it overlooks equity concerns and past problems with dependence on external professional services, and it ignores both the potential for disruptive reform churn and the danger of philanthropic efforts altering public education systems in undemocratic ways. For these reasons, the report’s usefulness to policy and practice is limited.

Although teacher evaluation reform has rapidly expanded across the U.S. in recent years, the newly enacted Every Student Succeeds Act (ESSA) seems to mark a federal deprioritization of teacher evaluation reform. This may in turn prompt states to consider changes to their evaluation systems. A new report from Bellwether Education Partners, For Good Measure? Teacher Evaluation Policy in the ESSA Era, argues for the maintenance of key elements of high-stakes teacher evaluation, including the use of student outcomes to evaluate teachers and a heavy focus on accountability. It urges policymakers to move slowly in revising their systems and to invest in management, capacity, and strategies to capture lessons learned. While the report raises several good questions with regard to the future of teacher evaluation, it overstates the likelihood that ESSA will result in widespread changes to evaluation systems, it ignores the literature regarding substantial technical challenges and unintended consequences of growth measures, and it dismisses the ideological and political debates surrounding teacher accountability. The unsubstantiated claims and dogged defense of student growth metrics provides little fresh or worthwhile new directions to policymakers seeking a nuanced and research-based discussion of teacher evaluation reform in the ESSA era. 

The stated goal of this report is to strengthen the evidence base on state-initiated turnarounds and to provide guidance to help states use turnaround strategies more effectively. The report draws on multiple sources of information to develop a conceptual framework and profile of state-initiated turnaround strategies, to array the evidence on the effectiveness of turnaround initiatives, and to identify key elements of a successful turnaround strategy. However, given multiple methodological limitations, the report fails to elevate either the research base or the policy discourse. Specifically, the methods used to carry out the original research (e.g., analysis of state policies, interviews with stakeholders, and illustrative cases) are neither explained nor justified. Likewise, the methods employed in the eight evaluations selected to assess the effectiveness of turnaround approaches are not described, and the evidence base produced by these evaluations is not sufficient to support the sweeping claims made in the report. Equally important, the report neglects to consider relevant research on the specific mechanisms (e.g., school reconstitution, intensive professional development, private management systems) that states use when they employ the broad turnaround strategies discussed in the report. As a result of these problems, the report does not enhance the evidence base or provide the substantive guidance state policymakers require to make informed decisions about the use of various school turnaround strategies. 

Two recent reports from the Fordham Institute address the question of the impact of state accountability systems on “high achievers,” referred to in the reports as “students who have already crossed the proficiency threshold.” Both reports argue that this group is being neglected educationally, and they advocate for accountability systems to be redesigned to attend to the needs of high-achieving students. Both reports also recommend that states use a “performance index,” as opposed to proficiency rates, to measure school achievement. This review, however, concludes that: 1) the reports’ central assumptions about high-achieving students are problematic; 2) growth measures are not an effective means for directing attention to high-achieving students; 3) narrow, high-stakes forms of assessment may negatively impact the education provided to these students; and, 4) further stratifying educational settings and reallocating resources toward “high-achieving” students has troublesome implications for the democratic goals of education. Implementation of the reports’ recommendations may in fact result in a furthering of the inequitable educational opportunities that ESSA was designed to reduce.

Better Evidence, Better Choices, Better Schools, a recent report from the Center for American Progress and the Knowledge Alliance, focuses on the evidence-based research provisions in the Every Student Succeeds Act (ESSA). ESSA provisions ask districts and schools to consider various sources of evidence, make judgments about the strength and reliability of that evidence, and use that evidence to justify their choices of interventions. Contrasting these new standards with the scientifically based research requirements featured in previous federal legislation, the report aims to provide guidance for the state and local education agencies now tasked with implementing evidence-based school improvement practices. The guidance consists of eight recommendations for implementing evidence-based reform strategies: defining roles, using evidence clearinghouses, supporting robust decision-making, ensuring high-quality service providers, promoting communication, partnering with intermediate organizations, facilitating effective implementation and promoting continuous learning. While helpful general pieces of advice, these recommendations remain relatively vague and are grounded in neither the existing research literature nor the empirical study featured in the report. The nebulous idea of evidence-based policy is easy to agree about, but skims over more challenging questions. For instance, what counts as evidence, and who is able (and authorized) to determine what kinds of evidence are most relevant to local contexts? While this report raises a number of important issues, it leaves these more difficult questions unexplored.

Noting the nation’s renewed attention to remedying school segregation, Segregation, Race, and Charter Schools presents evidence about the extent of school segregation and its relationship with improving student achievement for students of color. The report argues that school segregation has remained flat for decades and also argues that the nation would be wise to instead attend to improving the quality of schools that students of color and low-income students attend. It points to some urban charter schools as exemplars of this latter approach. However, this review finds that the report omits significant research directly related to the topic and includes other studies that are less relevant. Moreover, the report draws questionable conclusions from studies that are included—conclusions that are not reflective of the research consensus. The report’s selective interpretation of existing research leads to two erroneous conclusions about improving educational outcomes for students of color: (1) that focusing on school integration is relatively unimportant; and (2) that attending to school quality via school choice, rather than addressing the complex array of policies to combat racial segregation, should instead be pursued. In fact, because most forms of school choice further segregation, the report’s recommendation will likely only further segregation and inequality for students.