Edison Crows, But Report Makes Them Eat Crow

Edison Schools, Inc., recently commissioned an audit by the RAND Corporation to review the inner-workings of Whittle's company.

Not surprisingly, Edison spins the findings of the report very favorably, saying, "The 290-page study . . . offers the most comprehensive independent analysis to date of the achievement gains generated by school districts and charter schools that partner with Edison Schools." The press release continues, "RAND is one of the premier independent research organizations in the world," said (Edison Schools Chief Academic Officer John) Chubb. "We commissioned RAND's study of our schools for several reasons. (W)e were confident it would affirm the general proposition that most Edison schools bring significant achievement gains to our public-education partners. The study shows that, over time, this is exactly the case, and reconfirms results previously reported in Edison's annual reports on achievement."

But in reading the report, I discovered a number of findings that contradicted this glowing praise.

Data-driven assessments
In regard to Edison's benchmark assessment system and data-driven decision making, the authors write, "Occasionally, even the quality of information provided by the benchmarks is in doubt. A few teachers have viewed the benchmarks as high-stakes tests in themselves rather than diagnostic tools (although interviews in our case study schools suggested that this is not common). Although Edison’s explicit accountability system attaches no consequences to benchmark results (unlike state test results, which constitute a major part of each school’s Edison star rating), it is not surprising that some teachers would wonder whether benchmark tests might have implicit consequences for their own evaluation. Teachers who view the benchmarks as high-stakes tests may prepare their students in ways that are effective in promoting performance on the benchmarks but ineffective in promoting general academic skills. If this happens, the diagnostic value of the benchmarks is degraded. Recognizing the potential problem, Edison assessment staff are well aware that misuse of the benchmarks will undermine their utility for diagnosis (for teachers, for schools, and for the central office), and they consistently send the message that the benchmarks should not be viewed as high-stakes tests." (p. 69)

Narrowing the curriculum
In regard to narrowing the curriculum in order to concentrate on test preparation, the authors write, "In fact, we observed an intense focus on achievement in many of the Edison case study schools. This clearly had a payoff, as the case study schools had generally positive achievement trajectories in both reading and math. In some instances, however, a focus on test scores created a tension with Edison’s broader goal of promoting “worldclass” education. In a few of the schools we visited, some teachers admitted that the broader Edison curriculum is sometimes pushed aside by narrowly focused test preparation activities. Non-tested subjects such as art, music, foreign language, and (in many states) science and social studies are in some instances downplayed in favor of additional practice in basic skills in reading and math. And the ambitious, problem- and concept-focused mathematics curriculum used by Edison is sometimes displaced by worksheets used for test preparation in basic math skills. The most egregious instance we encountered (and the only such instance of which we are aware in the case study schools) involved one Edison principal who was dismissed after the district discovered evidence of cheating on a high-stakes assessment. . . (I)t must be acknowledged that to the extent that Edison’s accountability systems reward test results, they will reinforce both the productive and unproductive incentives associated with NCLB." (pp. 71-72)

Teaching to "the bubble kids"
Finally, in regard to teaching to "the bubble kids," i.e., adjusting instruction to raise the test scores of children near the proficiency level as indicated by the benchmark tests, the authors write, "The standards movement in K–12 schooling has encouraged schools to move away from achievement measures defined by reference to a larger population (e.g., percentile rankings) and toward achievement measures defined by external standards of proficiency in a particular content or skill area. NCLB cemented this trend by requiring all states to establish school accountability systems based on the proportion of students achieving proficiency. Many public schools around the country have rationally responded to the policy by seeking to identify and direct interventions toward those students who are closest to the cut-point for proficiency—the bubble kids (Pedulla et al., 2003). Edison has responded similarly: Its monthly benchmark assessments give its schools unusually good information for identifying bubble kids, and Edison actively encourages schools to identify such students and develop interventions to prepare them for state exams. The Edison schools we visited had some variation in attention to bubble kids. Some Edison principals and teachers embrace the concept as a logical and appropriate way to have data drive instructional decisionmaking. Others, however, are disturbed by the possible implication that students on both ends of the achievement spectrum— high achievers and low achievers—might be neglected in favor of those in the middle." (p. 72)