All Our Elementary Schools Are Equally Great!

(Note to readers: this is the second version of this post, so if it looks different from what you remembered reading, you’re not losing your memory :). I decided to edit it based on some questions raised by visitors, and, in the course of making changes, I accidentally deleted the original and had to rewrite from scratch).

Ever since I’ve been on the Board there’s been an opinion circulating among some parents in San Carlos that Heather Elementary just isn’t quite as good a school as the other three District-run elementary schools, Arundel, Brittan Acres and White Oaks. “The API scores are consistently lower,” people will say. “That proves the school just isn’t quite as good.”

In fact, that conclusion is based on the false assumption that all of our elementary schools serve the same student population. That’s an understandable mistake to make, since San Carlos is a small community and people therefore assume all four schools have the same mix of students.

But take a look at this chart (click to enlarge):

Notice anything? Last year Heather had almost 4 times as many students from out of district, on a percentage basis, as the other three elementary schools.

Here’s another chart you may find interesting:

Last year Heather had almost 2.5 times as many students with identified learning disabilities, again on a percentage basis, as the other three elementary schools.

Why do these differences exist? Because the area of San Carlos served by Heather, as it existed up until a couple of weeks ago, has a much lower population of school age children than the other parts of San Carlos. Consequently, Heather has, for many years, been underenrolled. So, when the District was looking to recruit or host children from outside San Carlos, or locate special programs (like the special day classes that serve children with identified learning disabilities), the obvious answer was to utilize the unused capacity at Heather. That was both easier and more cost effective than renting portables or building classrooms on other campuses.

But the net result is that Heather’s student population ended up being significantly different from that of its sister schools.

So if people are going to compare any metric about our elementary schools they need to ensure they’re comparing apples to apples. Which isn’t possible to do using the State-reported school API scores, because they don’t provide enough detail to adjust for the differences in student populations.

But the District has access to all the API records for its students and the State publishes the formula for calculating school APIs, and most of the rules for applying it. So the District can determine pro forma APIs based on any filtering rules it thinks would result in apples being compared to apples (or oranges to oranges; what is it that people have with apples, anyway?).

I recently did this kind of analysis at the District’s request (and please note that the data I used doesn’t contain information which could be used to identify students). There are different ways of “equilibrating” school populations, but the easiest method to use is simply to exclude the elements that create the differences. Which I could do, by restricting the API calculation to students living in the 94070 zipcode and who were not identified as having one or more learning disabilities.

Here are the results of making those adjustments:

I’ve included the As Calculated by District – All Students column so you can see that, in fact, the methodology I used to calculate the pro forma APIs reproduces the State-reported numbers quite accurately. That tells me I’ve successfully recreated the way the State applies the official formula.

Notice anything interesting? When you look at similar populations, all four elementary schools score within four points of each other. District staff and the Board have known this intuitively for years. But now you can see it in black and white.

This is one of the reasons why you hear the District, and the Board, say it doesn’t matter which elementary school your child goes to. They’re all great.

Including Heather.

Of course, I realize this doesn’t answer all the questions everyone might have about relative school performance. For example, I haven’t discussed why differences in student populations cause differences in API scores (which is related to the broader topic of why using simple API comparisons often leads to erroneous conclusions). That’s an interesting question, and perhaps at some point I’ll address it.

But that’s neither relevant nor necessary to understand my point here: when you compare apples to apples — which is the intuitively obvious, and fair, thing to do —  all the District-run elementary schools achieve the same outcome on the State’s standardized tests.