fun, fun, fun with statistics

by marjorieingall on September 16, 2009

my daughter the alevin

Whoo hoo, so my kids’ school was given a B grade by the New York City Department of Education. Last year, we got a D. I would love to say we kicked some ass and beat the underperforming children with a mallet until they got 4s on all their standardized tests, but not so much.

The DoE’s letter grading system is deeply, massively flawed. Last year (aka The Year We Got a D), the city’s Quality Review ranked our school “well-developed” on 34 of 35 criteria. Independent evaluators hired by the city praised its individualized instruction and “rich curriculum.” According to the DoE’s own Progress Report, the school was off the charts (above the 100th percentile, which is gratifying despite being statistically impossible) in “Academic Expectations,” “Communication” and “Engagement.” And our New York State School Report Card said the school exceeded every target for performance in every tested subject, in every grade, for every subgroup of students. It even won a special state award as a “High Performing/Gap Closing School.”

And then there’s the trout. My first grader’s class raised them. The kids measured and recorded the pH and ammonia levels in the aquarium, worked together to clean the water, sketched the physiology of fry. When the tragic Mass Die-Off of 2007 occurred, the kids pondered, according to established processes of scientific inquiry, what might have gone wrong. (Conclusion, confirmed by visiting trout expert: Not enough gravel in the tank to aerate it.) The kids read fish-centric fiction and non-fiction. They went to see a musical about New York’s intricate waterways. Josie dressed as an alevin, a baby trout still attached to its yolk sac, for Halloween. (She wore a silver dress and silver swim cap and taped a big orange balloon to her stomach. In the East Village, she fit right in.)

As you can tell, I’m blissfully in love with this school. I love the multidisciplinary approach, small size, mixed-age classrooms, and racial and economic diversity (it’s about 33% white, 27% Latino, 22% African-American, 16% Asian-American and 2% Native American, with 30% of the kids qualifying for free lunch). Josie’s thriving; a kid who used to bang her head with her fist and sob “I’m stupid!” now devours chapter books (Matilda! The Penderwicks! All-of-a-Kind Family!), explains echolocation in bats over the dinner table while the rest of us struggle to appear interested, and showing much more tolerance for her own human frailty. Instead of wailing, “I’m stupid!” she’s much more likely to say, “I don’t know. Can we look it up?”

I did not teach her to read, or to be more self-forgiving; her teachers did.

AND YET. In 2008, the DoE put this school in the 4th percentile of schools citywide. Why?

Well, in a small school like ours — and a vast body of research indicates that small classes and small schools are highly correlated with student achievement and well-being — tiny changes can have huge raw score and percentile impacts. The data only looked at 168 kids in four grades. So when the school’s attendance rate dropped from 93.5% to 92.7%, hello, dip of 20 percentile points. And when a busing glitch kept three siblings from getting to school on time for weeks (yup), those kids, through no fault of their own, had a massive impact on attendance scores.

But the main reason our scores plummeted so jaw-droppingly that year: Our peer index (the group of schools the DoE compares us to) was changed. The school was moved into a much whiter, wealthier, more advantaged cohort. And thereby hangs a tale.

The DoE weights school scores according to the numbers of students viewed as disadvantaged, factoring this data into a formula so complex that pondering it too closely causes one’s head explode like a character in a David Cronenberg movie. The city said that 50% of the Neighborhood School population is made up of African-American, Latino, special needs and/or ELL (English Language Learner) students. But in the grades that were actually tested, 58% of the students fit those criteria. (As our neighborhood gentrifies, it makes sense that the more advantaged students are in the youngest grades.) The number of “special needs” students the DoE  said the school had  (30%) is also significantly lower than the number actually enrolled in the tested grades (35%).

As for the scores themselves? Median English Language Arts (ELA) and math proficiency actually went up a tiny bit. But compared to our upscale new peer group, these gains were unimpressive, and there was an equally tiny dip in ELA progress scores. Still, we’re talking about the scores of fewer than 10 kids. The upshot: the equivalent of TWO KIDS PER CLASS got lower scores than they did the previous year. As 9-year-olds everywhere are wont to say, big whoop.

Look, very few parents on this planet are opposed to assessment. We want to know how our kids are doing. But many of us do object to excessive, misused standardized testing that distract us from rich curriculum and provides worthless data. Can any reasonable person argue that three-quarters of the schools that got an F in 2007 improved so much in one year that they deserved an A or B in 2008? Because that’s what happened.

This year, my kids’ school missed scoring a C by .5 point. Had it kept up this dread bad-stats-driven downward spiral, it would have been faced with the possible removal of the principal, financial penalties, reorganization, removal from the Empowerment Zone in which parents and teachers work together to make decisions they feel best for children as learners. All this despite ample evidence that the school is, you know, awesome.

So yeah, we got a B — because DUH, a few kids did better and the raw scores translated to super-impressive-looking gains. But guess what? In a school this small, there’s no way we’re going to show “sufficient progress” next year. I promise you, our scores will dip. And I promise you, they’ll still be meaningless. Please bear this in mind, my fellow neurotic parents.

I suspect I’m one of few Jewish mothers in America to turn down a Gifted and Talented placement for her child. But I honestly felt that in a cutthroat environment, a kid like mine would be hunched over a toilet, puking from anxiety, by third grade. I chose this school over the gifted program because I knew my competitive, tightly wound kid would thrive in a warm, nurturing place. Diversity and the teaching of good citizenship were more important to me than letter grades. (And anyway, my kid gets a 10-page report from her teachers twice a year; it tells me way more than a letter grade ever could.) I also knew about psychologist Carol Dweck’s work on how calling kids “gifted” can do more harm than good. I want Josie and her little sister Maxine to care about learning, not grades. I wish I felt confident that the Department of Education did too.

(Explanation of photo: My daughter’s underperforming school taught her to act like a trout.)

{ 2 trackbacks }

testing, testing…is this thing on?
October 6, 2009 at 8:52 am
yay, my kid’s school gets a D
October 2, 2012 at 8:06 am

{ 2 comments… read them below or add one }

polly kanevsky October 13, 2012 at 7:07 pm

sorry, i’m an idiot… how do i subscribe to this blog? This is a great piece.

marjorieingall October 16, 2012 at 1:11 pm

I am an idiot too. I don’t know how to do a “subscribe” thing on an individual blog! FWIW, I use Google Reader to gather automatically all the blogs I follow in one convenient tab. (And thank you!)

Leave a Comment

Previous post:

Next post: