DPI survey still looks partisan

Last week, I posted a quick and dirty reading of the DPI/WASD survey on the impact of the biennial budget on school districts. I thought that the survey needs a more thorough vetting but that it seemed to be a polemical document and did not support the claims of disaster that some are making in response to it.

Jay Bullock tries to defend the survey but I am afraid that he totally misses the mark. I have no reason to doubt that a number of districts had some kind of staff reduction. Most did not but it appears that somewhere in the neighborhood of 42% of the surveyed districts did.

But the doesn't tell us much. How deep were the reductions? How do they relate to changes in enrollment? What impact, if any, do they have on the delivery of services. Jay thinks that any reduction in staffing is a catastrophe, writing "[s]o, yes, a lot of districts were able to stave off disaster in this area but, you know, a full third didn't. " (emphasis in original)

Let's take a closer look at these "disasters." As I wrote in my original post, the mean "student/teacher ratio went from 13.27 in 2010-11 to 13.51 in 2011-2012. " Is that a disaster? Looking at median reductions, doesn't change the picture. For example, in the survey's comparison of districts that were "in" and "out" of contract, we see that the median loss for both groups were 1.2 and 1.0 FTE or, as human beings usually say "person." Is that a disaster?

I criticized DPI for excluding Milwaukee from its analysis of "in" and "out" of contracts districts. The defense for doing so was that Milwaukee is an "outlier" and doesn't affect the median teacher loss or "dramatically" impact the weighted average mean loss.

Well, of course (as I wrote), it wouldn't affect the median loss. You report medians to counteract the impact of outlying results. I would, however, like to see how it affects the mean average loss particularly since the mean average losses are so small.

The failure to show the impact is particularily curious since DPI begins its report by giving the aggregate losses across reponding districts which include reductions in Milwaukee. As the Journal Sentinel reports, almost half or the total staff reductions and 38% of teacher reductions were in Milwaukee. Thus, DPI uses the "outlying" district if it serves their anti-reform agenda and doesn't when it might not.

Beyond that, DPI tries to argue that the "reforms" did not help districts because those districts who were "in contract" were not all that different from those that were "out of" contract. But the survey seems to have counted as "in contract" districts who extended their contract by incorporating the compensation adjustments mandated by the reform bill. It is unlikely that those concessions would have been made in the absence of Act 10. A more meaningful concession would be between those districts who adopted the concessions and those (like Milwaukee) who did not. Jay himself concedes - albeit in a backhanded way - that this would have saved a lot of jobs in Milwaukee.

One of my specialites as a litigator is the cross examination of social science experts and results. A cursory examination suggests that a more complete vetting is in order.