Beating the odds, or beating the test?
High performing Charter Schools
“Odds-beating charter school.” Those words are like an impenetrable shield for those who operate such places. They are also the holy grail of the education reform movement, which is constantly seeking shortcuts to radically increase measures of educational achievement, which these days is pretty much defined by increased math and language test scores.
One problem with radical test score gains, as many researchers have noted, is that miraculous improvements in test scores over short periods of time are more often the result of cheating, student skimming, or other test manipulation. We’ve seen this pattern repeated all over the nation, starting with the so-called Texas Miracle under former US education secretary Rod Paige’s oversight.
Something very similar happened in New York state, where unrevealed relaxations in state testing standards led to a multi-year belief in the efficacy of reforms there. When results from the National Assessment of Educational Progress (NAEP) tests revealed the gains were illusory, New York admitted the test changes, thereby wiping out half a decade of supposed achievement gains on math and language tests.
Real improvements in test scores are more likely the result of longer-term changes that cause slow but steady rises in achievement. In fact, test results from decades of the NAEP show just such changes for the country as a whole. Scores for all US students have gone up by a full academic year over the past few decades, including breakout scores for various demographic groups including blacks and whites, and achievement gaps have actually been declining. Ironically (or, not) these gains have actually been slowing in the age of No Child Left Behind.
In Minnesota, birthplace of the charter school movement, one charter school operator labeled an “odds beater” has put a new twist on the concept. Eric Mahmoud, a former engineer and convicted mortgage fraudster, operates a group of segregated charter schools targeted at black, inner-city poor children that employs longer school days and years, strict discipline, and an unabashed strategy of beating state achievement tests.
For his success in raising math and reading scores, Mahmoud has been given rock-star status among charter proponents and recently was inducted into the National Alliance for Public Charter Schools hall of fame. In return, the Minneapolis school board recently awarded him the right to open four more charter schools sponsored by the city’s school district, all presumably to be run in what a local conservative columnist laudingly called his “drill and kill” style. News stories and opinion pieces from traditional to the alternative press have nonetheless sung his praises, so desperate is the reform community to raise test scores. Meanwhile, almost no one has questioned the efficacy or morality of deliberately segregated schools, nor ones so focused on improving specific test scores. It seemed that, until very recently, Mahmoud could do no wrong.
Three weeks ago that might all have changed with a series of investigative stories in the Minneapolis Star Tribune by veteran reporter Steve Brandt highlighting fiscal and management questions that, given the lax controls on charter schools and Mahmoud’s previous mortgage crime are unsurprising. Among Brandt’s cumulative reporting is the following:
But more important than the financial and management problems at Seed schools is the nature and structure of the educational pedagogy. The schools teach to the tests. Not just teach to them, but engineer high test scores with frequent testing, 100 minute blocks of reading and math each day and other methods. Few local writers or pundits pause to understand the implications of a strategy so strictly focused on state tests. Alleen Brown quoted Mahmoud’s thoughts on teaching to the test for an article in the TC Daily Planet (emphasis added):
“If the standards were different, content absolutely would look different,” said director Mahmoud. He dismissed a question about whether designing instruction after standards amounts to teaching to the test, saying that if an assessment is aligned to the standards, then there shouldn’t be a problem. “The worst thing in the world is to teach to something other than what the students are going to be assessed on.”
That is the discussion we should be having. Do we want to turn over even more of our black students to authoritarian schools that “drill and kill”? The tests themselves, which actually test a very narrow range of skills, are still sometimes useful for what they reveal. Below is a common chart about Mahmoud’s schools marvelous achievement rates:
From the chart you can tell that Harvest and Best both had superior proficiency rates and that Harvest had truly outstanding results. Education reformers who tout the primacy of these test score measures say the focus on math and reading is necessary to begin the learning pyramid. If that were true you’d expect Mahmoud’s students to do better in science, too, since math is an important component. In fact, as the charts from the Minnesota Department of Education below show, Harvest and Best students are doing poorly and getting worse on science tests. The first chart compares Harvest fifth grade science scores with those of all Minnesota students:
An average of all Minnesota students for all grades is a science proficiency rate of approximately 45%. Harvest Prep has a proficiency approaching zero. Next are the science scores for the Best Academy:
Here the story is much the same. Science scores for Best Academy fifth graders are bad and trending worse. In fairness, Best has improved science scores for its eighth grade students, who have improved to about a 30 percent proficiency rate. Best and Harvest science scores for fifth graders are actually much worse than statewide scores for students who are both black and receive free or reduced lunch, where scores are approaching 20 percent proficiency:
The perils of our obsession with narrow testing and the incentives for charter schools to teach to the test were spelled out last year by Michael Diedrich, a then Policy Associate for the think tank Minnesota 2020:
“..the data … support the hypothesis that what isn’t tested isn’t taught (or at least not with the same intensity).
We need to remind ourselves what the goal of our education system is. It’s not just about scoring well on the tests we have. It’s about comprehensive student readiness for post-secondary success. To punish and reward schools using just one and a half subjects (reading comprehension being only one part of English/Language Arts) is to encourage a narrow-minded focus on low-level testable skills. The MCAs disproportionately assess students at the lowest levels of learning; the skills necessary for critical thinking, effective creation, and original problem-solving don’t make the cut.
Even the reading tests being used in Minnesota for students before the ninth grade have severe deficiencies that further constricts what’s taught to early learners, as Diedrich pointed out in a comment to his piece:
“…there are plenty of math and reading skills that aren’t tested by the MCA. The vast majority of MCA questions come from the bottom two levels of Bloom’s Taxonomy (“remembering” and “understanding”), a few come from the middle (“applying,” “analyzing,” and “evaluating”) and none at all come from the top (“creating”). Because higher level skills aren’t tested, they don’t get taught, yet these are the skills necessary for students’ future success.”
Education is not the same as beating the test
The problems Minnesota has seen with “test beating” schools like those run by Eric Mahmoud – financial and management irregularities, forms of self-dealing that enrich operators, and engineering test results that bear little resemblance to forms of deep and wide learning – are foreseeable given the relaxed rules that apply to charters and our obsession with test results from a narrow range of subjects.
Mahmoud does not come from a teaching background – he was an engineer, a profession that looks at a problem domain, assesses the metrics on which he or she will be judged and finds a way to meet those benchmarks. From this perspective it is easy to see why his schools operate in a fashion to beat state tests – those are virtually the only metrics that matter, as Mahmoud himself has admitted. But this only shows how an unhealthy emphasis on testing has perverted the educational process.
In 1976 social science researcher Donald Campbell coined his “Campbell’s Law” hypothesis, which succinctly explains why our unhealthy obsession with standardized testing is leading to educational failure:
“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
Campbell explained his law in a way that directly anticipated Mahmoud’s teaching strategy:
“…achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.”
No one can fault the parents of black children in Minneapolis for wanting their kids to excel in math and reading. But methods matter and children aren’t widgets to be produced in an industrial process. We shouldn’t be surprised that Mahmoud manages to increase math and reading scores given the longer school year, longer school days, a singular focus on beating test scores, and the self-selected nature of his charter school student population. It would be more surprising if he wasn’t able to raise those test scores. But those marks come with a high price to a fully rounded education. Beating tests must not be confused with beating the odds. On a national level President Obama’s educational policies encompassed in his Race To The Top program only make these problems worse with their focus on expanding charter schools and more standardized testing. We can do better.
MORE ORIGINAL RESEARCH
Getting Minnesota charter school history wrong, again.
At one time Hiawatha had passable test scores, but this story, like so many education reform stories, was not what it seemed. In recent years Hiawatha's test scores have dropped steadily back down to earth, so that now they're about half of the state averages. For some reason national, and especially local media aren't interested in that now.
Ember Reichgott Junge's book provides a clear view into the history of charter schools in Minnesota, just not the one she intended
This is a big year for charter school aficionados in Minnesota, as 2017 marks the 25th anniversary of the opening of the the nation’s first charter school in St. Paul. The legislation that authorized charter schools, enacted a year earlier, though limited in scope, promised a thorough consideration of the experimental education model, which has since sprouted up in 43 states and the District of Columbia.
Poverty Academies, Segregation Academies and a foundation plan to erase the Minneapolis public school district
“School choice” is all the rage in Minnesota these days. The kind of school choice most in vogue are charter schools, where, according to promoters, less affluent parents can experience the same kind of education “marketplace” that rich people enjoy with their private schools.
The implicit deal was to trade charter school integration for higher test scores - but it hasn't turned out that way
Questions about segregation, integration, and academic performance have been intrinsically linked in American education policy since at least 1954, when Brown v. Board held that segregated educational facilities are inherently unequal. The research leading to that decision,and the overwhelming social science consensus ever since, has suggested that segregated schools produce a host of harms for their students, and integrated schools generate a host of benefits.