Hanlon's Razor.

So, the Index-Tribune took the time to review the post I made here yesterday. They acknowledged mistakes.  However, they missed the substantive error with their reporter's analysis.  Further, others have now weighed in, pointing out that the STAR test is obsolete -- a point recognized by their sister publication, The Press Democrat, and echoed by the SVUSD Superintendent in another comment today on this blog.  Finally, while I generally always apply Hanlon's Razor in situations like this, I have concerns regarding the adequacy of the reporter's byline that test the applicability of that adage.

To summarize what happened yesterday, I pointed out that the crushing effects on families of the continued Greater Recession/Lesser Depression is increasingly being revealed by standardized test scores in California.  The I-T's article never mentioned the effect of a depressed economy on educational outcomes for students.  However, I also pointed out the technical shortcomings of the reporter's analysis, including specific factual errors included in the article.

While the paper acknowledged the specific errors I pointed out, to understand the magnitude of the mistakes made means that we have to look at the explanatory PDF on appropriate comparisons of the API prepared by the California Department of Education.

"Invalid Comparisons of the API,"
2012-13 API Reports Information Guide, p. 13
California Department of Education
available at http://tinyurl.com/q7a8ty7  

California helpfully explains invalid comparisons using the API on page 13 (!) of the PDF.  The article in question made a series of these invalid comparisons -- in the same sentence.

The sentence at issue was "[t]he [high] school’s base API was 712 in 2013, down from 723 last year and down from 735 in 2008."

• First, this sentence mistakes the 2013 Growth API for the 2013 Base API. The California Department of Education Guide never even considers the possibility of a mistake like that -- it's such a basic mistake, I think it would kind of blow their minds. This is the mistake the I-T has admitted. 
• Then, the sentence sought to compare what it thought was the 2013 Base API to the 2012 Base API, which is invalid comparison Number One from the list.  The I-T hasn't admitted this mistake yet. 
• Then, the sentence sought to compare the 2013 Growth API to the 2008 Base API, which, coincidentally, is invalid comparison Number Two from the list.  The I-T hasn't admitted this mistake yet, either. 

These errors using the API aren't limited to the discussion of Sonoma Valley High -- they run throughout the discussion of all the other schools as well.  That's why I noted that "most of the multiyear comparisons in the article thus don't really hold up as a consequence." The fact that the article reports nonexistent numbers is one thing, but the basic error here is that the author really doesn't understand how the testing system works. 

Of course, there's yet another issue that's compounding the problems with the reporting in this article, which is that California's schools are in the middle of implementing Common Core.  My opinion (and I'd really like to have completed that blog post by now, but I do actually have to run my law practice and coach soccer, too) is that the individuals behind the creation of Common Core have taken the legal doctrine undergirding the Free Software (Open Source) movement and have implemented it brilliantly to revolutionize American education.

As a consequence, the STAR testing regime has been akin to legacy software for several years now -- and the advent of Common Core renders it obsolete. It's probably time for application retirement.  The Press Democrat's editorial board weighed in this week, calling for exactly that. The Superintendent of SVUSD commented this morning here, and based on that post, I think she concurs. Frankly, I have to agree -- STAR testing results get overwhelmed by demographic noise that obscures the signal concerning educational effectiveness, which has come up on this blog before.

California Schools Guide, Los Angeles Times

Screenshot Taken June 7, 2013.
Screenshot available at http://tinyurl.com/p8ztwrd

Regarding the balance of the comment from the I-T, I'd note that neither of the URLs posted in the comment work, and stating "our print article included a huge chart" is a "defense" that explains many of the problems faced by the newspaper industry.  Further, telling parents "you can go search here" rather than doing the analysis seems to miss the point of reporting.

Finally, I respect the paper when it notes that "[a]s for the reporter being remiss in not speculating why the scores went down, that isn't our role." I agree that the role, as described, is indeed the proper province of the paper. However, I have a hard time reconciling that statement with comments like the one the author of the article left on the LA Times' web site, a screenshot of which is at the right, where the reporter explicitly speculates about what causes test scores to move -- a comment that also suggests a level of partisanship one would think is inappropriate in a reporter.

If there is some relationship, business or otherwise, between the reporter and a private school here in Sonoma Valley, I think that relationship should, at a minimum, be disclosed in the reporter's byline.  Whether the existence of such a relationship should render that individual ineligible to serve as a reporter on the subject of public school test scores is a question that is, however, above my pay grade.

"It's the Economy, ..."

(November 5, 2013 -- For the benefit of readers, please note that the comments on this post include statements from the Sonoma Index-Tribune and the Sonoma Valley Unified School District, that were the subject of a followup post here.)

The Sonoma Index-Tribune wrote an article last week about "declining" test scores in Sonoma Valley. While I'm blogging about another topic at the moment, I did not want to let this one fly by without comment, as there was an interesting data trend buried in the (somewhat confused) article.

The students entering kindergarten this year were born at the end of 2007 and the first part of 2008.  These are children whose "First 5" years of child development took place during an economic collapse.  All of the students in elementary school today have spent at least half of their lives in a depression.  It is my suspicion that the consequences of that experience will be visible in the data for decades.

The data trend stumbled upon by the I-T is not unique to Sonoma Valley. Austin Creek Elementary is a nice example -- it's ranked as a "10" on the statewide rankings, and it's the Santa Rosa school geographically closest to Sonoma Valley.  Austin Creek's 2013 Growth API Report shows the same trends as Sonoma Valley.  Its scores are plummeting -- Hispanic/Latino student performance fell by 51 points, and Socioeconomically Disadvantaged student performance fell even more dramatically, by 113 points.

Austin Creek Elementary

 available at http://tinyurl.com/pjq7vok

Yet it's unlikely that Austin Creek's (or anyone else's) ranking will change as a consequence of these declines, because school rankings are relative things. If everyone starts doing worse, the rankings stay the same.  And that's exactly what's been going on, as nearly every student, across California, has faced an extraordinary tumult and disruption in their daily life.  This is not just due to school budget cuts, but more importantly, because their families are being crushed by an ongoing, drawn out economic depression.

I do, though, want to draw attention to the fact that there were problems with the substance of the reporting, which probably has more to do with the author's unfamiliarity with the testing regime than anything else.

First, the article consistently confuses the difference between Growth API and Base API.  For instance, the article states that "[t]he [high] school’s base API was 712 in 2013, down from 723 last year and down from 735 in 2008." This is easily, demonstrably wrong -- the 2013 Base API won't be released until Spring of 2014; the only thing that's been released for 2013 is the Growth API.  It's not uncommon for the less experienced observer to make that mistake, and most of the multiyear comparisons in the article thus don't really hold up as a consequence.

Further, while the title of the article concerns STAR results, there's next to no discussion of the specific results at all.   The STAR test results are granular -- results come out for each school, each class, and each quintile within each class. An article that gave some guidance for parents on how to understand those reports would have been useful. However, that explanation would not have been a short one, and would have shown some very positive results for the District.   But the article wanted to talk about Base API for 2013 (even though that's not available yet), and unfortunately, a detailed discussion and analysis of the STAR results was a casualty of that decision.

Finally, the tone of the article will probably cause some unnecessary headaches.  Every time one of these pieces comes out, there is a certain amount of rending-of-clothes by the teachers and administrators at SVUSD. But beyond the illustration of the economic impact of the Great Recession/Lesser Depression, I don't think there's much to take away from the article for those interested in improving the condition of Sonoma's schools.

Glass Ceiling, Iron Lady.

This is the third in a series of posts, about where the State of California is headed.  The first is here, and the second is here.

In the first post, I took a look at the Nordic countries of Europe, and how their high quality governments arose out of a history of difficult geography and benign history. I noted that California has a similarly difficult geography and benign history, and that California has dealt with those challenges through a series of "big projects," the ultimate example of which is California's Master Plan for Education. Yet while both the Nordics and California have enjoyed subsequent economic success, they have developed very different levels of trust in government. To try to understand why California's culture has developed differently, I looked for a foil to California, a place with technical sophistication and high levels of education that is nonetheless suffering from economic stagnation -- a description that fits Japan.

In the second post, I took a look at Japan, and how its stagnant economy has been hobbled culturally. I drew attention to the Japanese disdain for entrepreneurs. Even more importantly, I pointed out its yawning gender gap. I found this surprising given Japan's long term cultural flexibility. However, I argued that the Japanese repression of women seems to reflect a longing for an arcadian rural society. I pointed out how England shared the Japanese preoccupation with so-called romantic country life. I argued that the English, unlike the Japanese, have self-critically examined the consequences of this ideal, and that the English had tempered their views by the late 1970's, when Margaret Thatcher, chemist by training, Education and Science minister in government, became Prime Minister.

---

Surrey docks, September 7, 1940
Image available at http://tinyurl.com/l9mtdsu

In some respects, it is hard to pick "winners" from World War 2.  But it is clear who the losers were, and the British Empire was one of them.  Much as the collapse of the Western Roman Empire was foretold by the sack of Rome by Alaric in 410 AD, the Blitz of London in 1940 shocked the British and signaled the acceleration of Imperial decline. Despite eventual American entry into the War, the UK found itself on the brink of bankruptcy in 1945, and a harsh postwar austerity followed.

However, the UK did not stop bearing the costs of maintaining international order. Unlike the Nordics (and, effectively, since 1945, Japan), the UK continued to expend substantial sums on defense, while the Japanese and Swedes spend far below the average to this day. The UK, like the US, remained committed (most of the time) to collective defense through NATO, and to collective security through other international organizations.

 Image available at http://tinyurl.com/kn87o24

By the 1970s, though, the UK economic situation had become dire. Wages were amongst the lowest in Western Europe -- half those of the West Germans, and only two-thirds of the Italians -- and educational achievement lagged behind a series of other industrialized nations. Massive strikes became common. The UK's "Winter of Discontent" in 1978-1979 is hard to imagine today, but it was a place where Chris Mullin's A Very British Coup was a plausible, if frightening potential future for the "Sick Man of Europe." 

Into the maelstrom stepped Margaret Thatcher as Prime Minister. Thatcher herself said, as late as 1970, that "[t]here will not be a woman prime minister in my lifetime - the male population is too prejudiced." This was a keen observation from a skilled observer; it was also wrong. The culture of the UK changed, rapidly, under pressure. 

It is difficult to understate how significant a change it was; in 1832, the UK had taken the vote away from women. While the US record on women's rights is not exactly stellar, Jeannette Rankin of Montana was serving in Congress before women in the UK even had the right to vote.  Julian Fellowes' Downton Abbey (a short clip is below) illustrates the UK upper class ... backwardness, for lack of a better phrase, right up to the start of World War 2.

Thatcher's policies in office were conservative -- essentially, to promote low inflation, the small state, and free markets through tight control of the money supply, with privatization of state-controlled businesses and constraints on labor unions.  The success of her efforts is attested to by the fact that most of the major UK political parties today accept the approach that Thatcher's government installed.  A cultural consensus formed regarding Thatcherism, which spread across the political spectrum. Tony Blair, in his autobiography A Journey, argued that the UK "needed the industrial and economic reforms of the Thatcher period ... much of what she wanted to do in the 1980s was inevitable, a consequence not of ideology but of social and economic change."

However, while Thatcher's policies were anything but socialist, her government did not challenge the existence of the National Health Service -- the UK has had what Californians would characterize as "Medicare for All" since 1948.  Just as a consensus in the UK formed around the economic policies of Thatcherism, so too had a consensus formed concerning health care. It is easy to underestimate how important portable health care is for women (or, indeed, any entrepreneur) in the workforce; the US system tying it to employment has been argued (I believe correctly) to have created a degree of "entrepreneurship lock." When Thatcher implemented her policies, no laid-off worker needed to fear losing their health care, a critical point often missed by US observers. Thatcherism, while a radical series of reforms, was therefore a great deal less harsh than similar policies are when implemented in the US.

And in that forming of a cultural consensus -- of trust in government, alternately coalescing around policies championed by the left and the right, is the thread that ties the stories of the Nordics, Japan, and the UK in the last century to the future of California in this one.  And arriving at that point is the signal that it's time to turn the focus to California itself.

---

Janet Yellen

Image available at http://tinyurl.com/khgsorw

I take the trip back across the Atlantic in 1980 with a pair of somewhat nerdy economists, heading to new teaching jobs on the West Coast of the United States. They had been teaching for a few years at the London School of Economics, where the wife had encountered the same resistance Margaret Thatcher described in 1970 in the UK's culture. The husband and wife team was done with the LSE, because, as the faculty there would later admit, "we only thought of her as someone’s wife ... ”

The couple's names were George Akerlof and Janet Yellen. And that's where this story will pick up again, another day, over here.