Jay Greene on Talk of the Nation

On Talk of the Nation today, Jay Greene argued that students who take and pass exit exams show they have the skills that employers are looking for. But more and more classrooms are focused on prepping students for the test, not giving them the skills that employers want and need. After all, a multiple choice test can't measure critical thinking skills, communication skills, and team-building skills -- all things that employers say they want. But in prepping students for these kinds of exams, students are not taught these things. So are we really preparing them for the future, or are we preparing them to be good test-takers?

In the article in TCR Record that Greene mentioned (http://www.manhattan-institute.org/html/cr_33.htm), the issue posed is as follows:

"Whether the high stakes of high stakes testing are in fact motivating schools to manipulate results without actually improving real student achievement is a question that can be investigated empirically. By comparing results from high stakes tests with results from other standardized tests administered around the same time, we can determine whether the high stakes associated with high stakes tests are distorting test results. If high stakes tests produce results that are similar to the results of other tests where there are no incentives to manipulate scores, which we might call “low stakes” tests, then we can have confidence that the high stakes do not themselves distort the outcomes. If, on the other hand, high stakes tests produce results that are not similar to the results of low stakes tests, then we should be concerned that schools have managed to produce results on high stakes tests that are inaccurate reflections of actual student achievement."

The conclusion reached is as follows: "The report finds that score levels on high stakes tests closely track score levels on other tests, suggesting that high stakes tests provide reliable information on student performance."

The idiocy of this conclusion is staggering. What Greene fails to take into consideration is the logical conclusion that ALL test prep -- whether "high stakes" or "low stakes" -- results in both manipulation of results and a narrowing of the curriculum. Because the results of "high stakes" and "low stakes" tests are the same, he naively (or willfully and blindly) concludes that they must be produced under the same conditions, i.e., no manipulation of data and no narrowing of curriculum. But this could be argued from the opposite perspective, i.e. the results are the same precisely because they were produced under the same conditions (data manipulation and narrowing of curriculum).

Greene naively asserts that there are no incentives to manipulate scores or instruction in so-called "low stakes" tests. Yet the evidence speaks to the contrary.

The Center on Education Policy reported last year that 27 percent of school systems say they are spending less time on social studies, and nearly 25 percent say they are spending less time on science, art and music. In another study from 2004, the Council of Basic Education surveyed 954 principals in 4 states in different parts of the country: Indiana, Maryland, New Mexico, and New York. They all reported that their schools were spending less time on social studies, art, and foreign languages. According to the Council of Basic Education’s report, 47% of high-minority schools reported decreases in social studies instruction.

In more and more classrooms, teachers are told to identify the low achievers, the high achievers, and the kids just on the verge of passing the state test, the so-called "bubble kids." Jennifer Booher-Jennings (American Educational Research Journal, Summer 2005, Vol. 42, NVo. 2, pp. 231-268) notes the practice of teaching to the bubble kids -- what she calls “educational triage” -- has become increasingly widespread in response to accountability systems, and has been documented in Texas, Chicago, California, Philadelphia, New York, and England.

Booher-Jennings interviewed a teacher at an urban elementary school in Texas; the teacher had this to say about "educational triage" and teaching to the bubble kids: "I guess there’s supposed to be remediation for anything below 55%, but you have to figure out who to focus on in class, and I definitely focus more attention on the bubble kids. If you look at her score [pointing to a student’s score on her class test score summary sheet], she’s got a 25%. What’s the point in trying to get her to grade level? It would take two years to get her to pass to the test, so there’s really no hope for her...I feel like we might as well focus on the ones that there’s hope for."

"Data-driven assessments" such as those administered at Edison schools and an ever-increasing number of other public schools lead quite logically to the phenomenon of "educational triage," i.e., teaching to the bubble kids. In defense of the schools that engage in “data-driven assessment,” their approach is not indicative of the school avoiding accountability, shirking its role, and looking for wiggle room. In fact, such an approach is both efficient and logical under NCLB's terms and conditions that determine what "efficient" and "logical" are. It actually shows how well schools follow orders and how well they do their (new) job. Public schools are doing exactly what they are told, are towing the party line, and are doing what is logical and efficient under NCLB. Cutting social studies and other non-tested subjects is also logical and efficient under the logic that defines NCLB.

Finally, as David Berliner and Sharon Nichols point out in "The Inevitable Corruption of Indicators and Educators Through High-Stakes Testing," (executive summary at http://www.asu.edu/educ/epsl/EPRU/documents/EPSL-0503-101-EPRU-exec.pdf) the logic of test prep leads to the fulfillment of Campbell's law (no relation), i.e., "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” Berliner and Nichols conclude, "Applying this principle, this study finds that the over-reliance on high-stakes testing has serious negative repercussions that are present at every level of the public school system."