Private School Performance?

Infographic available at https://infogr.am/selected-schools-gpa/

In speaking with friends recently, we started puzzling over a question we all share: how do we know what the benefits are of a private school over public school? Clearly, many families see the benefit of substantial expenditures on private education, and surely there must be some return on that investment?  In essence, the consensus amongst the group was that the marketing of such schools leads parents to believe that the students that attend those schools would be expected, test scores and grades in high school being equal, to outperform public school students when the public and private school students arrive together at college.

In general, I have always thought that there is no way to determine the truth of that proposition -- that there is no useful data concerning performance in the first year of college.  Grades issued by different institutions are generally not comparable, even if they were available.  We tend to instead retask predictive measures of college success, such as the SAT, as a proxy for that data -- but such measures are always second best to the grades themselves.

But it looks like there's a way to answer the question after all.  The California State University system accepts students from virtually every high school in California (1,590 different ones in the last 18 years). While CSU is not the most elite part of California's Education Master Plan (that would be the UC, which accepts only the top 12.5% of high school students), CSU generally accepts the top third of California's students. While these students are not typically those who have accumulated many advanced placement units, and while these students tend to have test scores within one standard deviation of the norm, the students that attend CSU from private and public schools, at least as a matter of general consensus, are believed to have similar if not the same test scores and grades.  Thus, the performance of such students is perhaps the best example of head-to-head, apples-to-apples performance of public and private school students, after arriving at college, in California (if not in America).

I was surprised to discover last week that CSU makes those grades available on the web, sorted by high school and by year.  It was a relatively straightforward task to script cURL to pull CSU's data for 31 high schools in Marin, Napa, and Sonoma County from 1996-2012 (527 requests, which took the bash script less than 60 seconds), and I have zipped those files and made them available here, for anyone interested in examining the raw data. 

At the top of this post is a graph of the average freshman year college GPAs of students who have graduated from Sonoma Valley High School, Justin Siena, Sonoma Academy, Cardinal Newman, St. Vincent's, and Marin Catholic in that time period. While in 1996 every private school's college freshmen performed better than those from Sonoma Valley, the reverse has been true since 2005.  Sonoma Valley's students outperform every one of the private schools, and by a significant amount.

I've excluded Marin Academy, because they don't have enough data to graph.  Marin Academy sent six students to CSU in 2012, but before that Marin Academy fell below the threshold for reporting (5 students) every year -- so there's only one year's worth of data.  It's a good result for them, though -- in 2012, Marin Academy posted a 3.22, but their difference from Sonoma Valley (.17 of a grade point, 3.05 versus 3.22) is significantly smaller than the difference between Sonoma Valley and the next private school on the list (a .27 difference, between Sonoma Valley at 3.05 and Cardinal Newman, at 2.78).

Elizabeth Warren
Image available at http://tinyurl.com/ml4sw9g

This strongly suggests that those pursuing private education, with the exception of the very expensive Marin Academy (tuition $37,430 yearly), are perhaps subject to the phenomenon uncovered by Elizabeth Warren in her early research.  In "The Two Income Trap," Warren and her co-author (and daughter!) Amelia Tyagi pointed out that middle class families drive themselves into bankruptcy to buy homes they cannot afford in order to live in a neighborhood with better schools. As Warren and Tyagi argued, the actual "benefits" such parents obtained for their children were slim at best, and were more likely than not illusory in truth.

Yet the problem in Sonoma may be even worse.  Many parents are sending their children to private schools in the belief that they are obtaining an academic advantage.  This is not to discount other reasons for sending children, for instance, to religious schools -- educating one's children regarding deep religious convictions, shared by an organized group, and intimately related to daily living is a right protected by the First Amendment. However, to the extent that parents are financially straining themselves to obtain a perceived academic advantage, they should know that the evidence shows no increase in the children's later academic success, and instead shows that the opposite may in fact be true ...

"Bringing Up Baby Bilingual"

image available at http://tinyurl.com/m693ucz

The Prospero blog on Economist.com today examines bilingual education, and the data it presents is dramatic:

"The benefits ... are both strong and long-lasting. Bilingual children as young as seven months outperform monolinguals at tasks requiring “executive function”: prioritising and planning complex tasks and switching mental gears ... [s]uch studies control for socio-economic status, and in fact the same beneficial effects have been shown in bilingual children of poor families. Finally, the effects appear to be lifelong: bilinguals have later onset of Alzheimer’s disease, on average, than do monolinguals ..."

"Many parents once believed that a second language was a bad idea, as it would interfere with developing the first and more important one. But such beliefs are woefully out of date today. Some studies (such as this one) seem to show that bilinguals have smaller vocabularies in each language (at early stages) than monolinguals do. But other studies (such as this one) find no vocabulary shortfall in either language. In any case, the influence of mono- or bilingualism on vocabulary size is later overtaken by the importance of education, socio-economic status, reading and writing habits. In short, there is little evidence that raising a child bilingual will hurt their primary language."

The columnist speculates that the benefit of the second language comes from monitoring the use of two languages (which is itself an exercise of the executive function), and that seems to make a certain amount of sense.  Interestingly, the author also notes that the benefits of bilingualism wither unless the student must use the language in certain circumstances -- either at home, work, or school -- which explains a lot about the power of dual immersion ...

Income Per Capita and Teacher Pay.

Motoko Rich

image available at http://tinyurl.com/mvynl6s

The New York Times has an article up today by Motoko Rich, which is second in their "most emailed" list.  It concerns a newer study by Anne Fernald, a psychologist at Stanford University.  Earlier research on language development found that by age 3, the children of wealthier professionals have heard words millions more times than those of less educated parents, giving them a distinct advantage in school and suggesting the need for increased investment in prekindergarten programs. Fernald's follow-up study has found a language gap as early as 18 months.

I wanted to look at the study itself; Wiley published the journal in question, and after paying a fee (grr...) I took a look at the PDF.  The sample sizes Fernald uses are small, and somewhat oddly, she was vague about the two locations the study's based upon.  From the article, it's clear that the first site she tested was at or near Stanford University; the second location is "a few hours north."  The study itself gives a few more clues in the table below.  

'SES differences in language processing skill
and vocabulary are evident at 18 months,'
Developmental Science (doi: 10.1111/desc.12019) (on file)
available at http://tinyurl.com/b42epvg

The total population numbers, percent non-Hispanic white, and cost-of-living index are a demographic fingerprint of where this study was conducted.  Those numbers match up very well indeed with Redding, California, in Shasta County -- the white population there, according to city-data.com, is 82.2% (Fernald reports it as 83%), the population is 90,755 (she reports it as 90,500) and per capita income is $21,585. Dr. Fernald's evasiveness is probably explainable by her relationship with the Shasta County Office of Education (see this and this).  I mean, I get it, it's not like anyone in Shasta County is going to be particularly happy being described in the New York Times as California's version of the poor whites of rural Appalachia.

And of course, Shasta County really isn't like that at all; the article in the New York Times slipped on the difference between median income and median per capita income.  The median income in both locations is roughly double what the New York Times reported.  I suspect the mistake comes from the fact that Motoko Rich from the Times just couldn't believe that a family is "low income" in California when their income is over $40,000 (the idea that median income in Palo Alto is $69,000 should have been a big warning sign).  However, this is a fact that many here know all too well -- the cutoff for affordable housing in Sonoma Valley, for example, has often been a family income of approximately ~$90,000 a year, which simply astounds the rest of the United States.

"Language-Gap Study Bolsters a Push for Pre-K"
The New York Times, October 22, 2013, p. 2.
available at http://tinyurl.com/qzcufq5

However, I don't want the slip in the article to swamp what I think is the key point, which is the following, and which is spot on:

"Literacy experts have previously documented a connection between a child’s early vocabulary and later success in reading comprehension. In a study tracking children from age 3 through middle school, David Dickinson, now a professor of education at Vanderbilt University, and Catherine Snow, an education professor at Harvard University, found that a child’s score on a vocabulary test in kindergarten could predict reading comprehension scores in later grades."

"Mr. Dickinson said he feared that some preschool teachers or parents might extract the message about the importance of vocabulary and pervert it. 'The worst thing that could come out of all this interest in vocabulary,' he said, 'is flash cards with pictures making kids memorize a thousand words.'"

"Instead, literacy experts emphasize the importance of natural conversations with children, asking questions while reading books, and helping children identify words during playtime."

"Even these simple principles may be hard to implement, some educators say, because preschool instructors are often paid far less than public schoolteachers and receive scant training. In one study, Robert Pianta, dean of the Curry School of Education at the University of Virginia, found that in observations of 700 preschool classrooms across 11 states, teachers in less than 15 percent of the classes demonstrated 'effective teacher-student interactions.'"

That's right.  We're treating the people who interact with our kids like crap, and we're getting what we've paid for.

And the consequences are severe over time.

Further Reading.

Regarding Monday's post, three articles caught my eye in the past few days touching on some of the particular points made in that post.  The first is from the Economist, the second from Time Magazine, and the third from the New York Times.

Angela Ahrendts

image available at http://tinyurl.com/katsqtm

Starting with the Economist, this will be among the few times this blog ever mentions Burberry.  Angela Ahrendts, its CEO, has quit to run Apple's retail operations. In six paragraphs, the Economist's editors managed to miss (or chose to ignore) the significance of the fact that Ms. Ahrendts is a woman. Apple's executive suite is composed of CEO Tim Cook and eight male senior vice presidents. Ahrendts will be Apple's ninth SVP, and first female SVP since 1992.

Why should Ahrendts quit as a CEO to play second fiddle to Tim Cook?  There's a lot of things that Tim Cook's been responsible for, but amongst his biggest achievements was the move from PowerPC to Intel chips for the Mac line; that's an engineering, not a design function.  The Economist correctly notes that Ahrendts is very effective at fusing design and technology, but I have a strong suspicion that Ahrendts is willing to make the jump because she has confidence she has a shot to run Apple if she's successful, and that she's risen as far as she practically can in the UK, although perhaps not in California ...

The second article, from Time Magazine, notes that the end of the government shutdown was, in many ways, attributable to a group of female senators.  The U.S. Senate has been called the ultimate men's club, with, "unspoken rules, hidden alliances, off-hours socializing and an ethic based at least as much on personal relationships as merit to get things done."  But the article instead drew attention to the success that the group of female senators, regardless of party, have managed to achieve:

image available at  http://tinyurl.com/mkcsu8c

"Most of the Senators say they feel they speak not just for the voters in their states but for women across America. Over the years they have pushed through legislation that has vastly expanded funding of women’s- and children’s-health research, testing and treatment. They’ve passed the Lilly ­Ledbetter Fair Pay Act and other anti­discrimination laws. And they’ve won federally mandated maternity and family medical leave. While most of these efforts were driven by Democrats, the women are strongest when they unite on legislation like the Homemakers IRA, which allows tax-deductible contributions to retirement plans by stay-at-home parents."

image available at http://tinyurl.com/khgcs7w

The final piece, from the New York Times, concerns California's government. Noting that the State has long been "the national symbol of partisan paralysis and government dysfunction," the article points out that a sunny assessment of changes in the State have been voiced by people inside and outside the government:

"... [T]he new atmosphere in Sacramento also offers the first evidence that three major changes in California’s governance system intended to leach some of the partisanship out of politics — championed by reform advocates — may also be having their desired effect in a state that has long offered itself as the legislative laboratory for the nation." 

... 

"Lawmakers came into office this year representing districts whose lines were drawn by a nonpartisan commission, rather than under the more calculating eye of political leaders. This is the first Legislature chosen under an election system where the top two finishers in a nonpartisan primary run against each other, regardless of party affiliations, an effort to prod candidates to appeal to a wider ideological swath of the electorate."

... 

"The turnaround from just 10 years ago — striking in tone, productivity and, at least on fiscal issues, moderation — is certainly a lesson in the power of one-party rule. Democrats hold an overwhelming majority in the Assembly and Senate and the governor, Jerry Brown, is a Democrat. The Republican Party, which just three years ago held the governor’s seat and a feisty minority in both houses, has diminished to the point of near irrelevance."

California, where Malala Yousafzai becomes Janet Yellen.

This is the fourth (and final) part of a series on the future of California. The first post is here, the second is here, and the third is here.  A three paragraph summary of those posts is here.

This post is not brief, but a summary of it can be.  Education makes women powerful, and California is the place where women can get it, use it, and be recognized for it better than anywhere else.  It comes from the character of the State's institutions --  there is evidence that they are earning the public's trust, while nationally faith in government has collapsed.  California, like the Nordics, is building a society that can support women who are working. It will increasingly be immigrants, particularly those that are the mothers of young children, that will hold the keys to California's future. Government must re-earn their trust, generation after generation.  Through education and law, California can continue to win that confidence, making it one of the world's citadels of hope in the face of primitive barbarism.

---

Malala Yousafzai

Image available at http://tinyurl.com/kbt55hd

Malala Yousafzai's on my mind this week. She's come up here before, back in May, when I noted the most popular movie trailer on Apple's site was for Girl Rising, a film partially about her experiences. Malala is a Pakistani girl from the northwestern Khyber Pakhtunkhwa province. In early 2009, when Malala was 11, she began writing a blog detailing her life under Taliban rule, and their ban on girls attending school. In October of 2012, she was shot in the head and neck in an assassination attempt by Taliban gunmen while returning home on a school bus. The assassination attempt sparked a national and international outpouring of support for Malala.

Because she was nominated this year for the Nobel Peace Prize, there were a series of interviews with her in the run-up to the announcement of the award last week (she did not win this time, but don't count her out). Malala made one comment a few days ago on "The Daily Show" that specifically caught my attention, in response to the question "[w]here did your love of education come from?" Her response was:

"We are human beings, and this is the part of our human nature, that we don’t learn the importance of anything until it is snatched from our hands … and when, in Pakistan, we were stopped from going to school, and at that time, I realized, that education is very important, and education is the power for women, and that is why the terrorists are afraid of education, they do not want women to get education, because then women would become more powerful.” 

---

Janet Yellen

Image available at http://tinyurl.com/kt5qv3z

It was also last week that Barack Obama nominated Janet Yellen to be the Chairman of the Federal Reserve. As the Daily Telegraph's Ambrose Evans-Pritchard put it:

"Janet Yellen is to take over the US Federal Reserve, the world's monetary hegemon, the master of all our lives ... [n]o Fed chief in history has been better qualified. She is a glaring contrast to Alan Greenspan, a political speech writer for Richard Nixon, who never earned a real PhD (it was honorary) or penned an economic paper of depth ... [s]he has pedigree. Her husband is Nobel laureate George Akerlof, the scourge of efficient markets theory. She co-authored "Market for Lemons", the paper that won the prize ... [c]urrently vice-chairman of the Fed, she was a junior governor from 1994 to 1997 under Greenspan, and then president of the San Francisco Fed from 2004 to 2010. She was head of Bill Clinton's Council of Economic Advisers from 1997 to 1999, when she handled the Asian crisis. You could hardly find a safer pair of hands."

Yet despite her obvious ability, Janet Yellen hasn't exactly had an easy time of it; in 1978, when she and her husband were both teaching at the London School of Economics, Yellen encountered the same resistance Margaret Thatcher described in the UK's culture. ” Meghnad Desai, Baron Desai, who worked alongside Yellen at the LSE, said that:

“She was very undervalued at that time, because we only thought of her as someone’s wife ... I could see that she felt she was not getting her due.”

While neither Akerlof nor Yellen were from the San Francisco Bay Area, they had jumped ship from the LSE to UC Berkeley by 1980.  Yellen became a full professor within five years, the position from which she vaulted to her most impressive achievements.

---

So much for Yousafzai and Yellen, how does California come into this story?

David Brooks, in his "Sidney" Awards in December 2012, called attention to an article by Ron Unz, entitled "The Myth of The American Meritocracy." Unz's article demonstrates the sharp differences in admissions practices between, on the one hand, the Ivy League, and on the other, CalTech, MIT, and the flagship UC campuses, Berkeley and UCLA.

Between 1980 and 1993, Asian students went from ~5% of the entering Ivy League freshman classes to over 20% -- but then the percentage fell to 16.5%, where it has held constant more or less since. Meanwhile, CalTech, MIT, and the UC campuses saw the same increase starting in 1980 -- except that the rise never stopped. Unz points out that CalTech, MIT, Berkeley and UCLA have about the number of Asian students that would be expected based on the number of Asian National Merit Scholars overall (about 40% as of 2013).

I cite to Unz (while disagreeing on several points) because he singles out the University of California (along with the science-focused CalTech and MIT) as the schools continuing to fulfill the meritocratic ideal. I think he's right on that particular issue. Berkeley and UCLA will not discriminate against a student that looks like Malala Yousafzai based on the color of her skin.  It is the UC system that will, regardless of sex or religion or race or whatever, catapult one to the apogee of their field. It is where someone like Janet Yellen is not "someone's wife."

---

The meritocratic ideal, of which UC and CalTech are examples, is a value that has spread far beyond California's universities -- the State's institutions draw in women with ability from all over the world, and advance them to positions of power in government, science, business, and law. Dianne Feinstein. Barbara Boxer. Condoleezza Rice. Laura Tyson and Christina Romer. Meg Whitman, Carly Fiorina, Carol Bartz, and Marissa Meyer. Rose Bird, Tani Cantil-Sakauye, and Kamala Harris. Elizabeth Cabraser. Mary Meeker. Susan Wojcicki and Sheryl Sandberg. 

"Asia Has Surpassed Latin America As The

 Leading Source of Immigrants to California"

Infographic, "Immigrants In California," 

Public Policy Institute of California, 

available at http://tinyurl.com/kgnorac

When we speak of the technical achievements of California researchers, or of the skill of the State's businesses in the creative arts and in cutting-edge technology, it is in no small part due to the fact that California is the place in the US that has done the best at unlocking the intellectual abilities of all of its citizens. This is a feat unimaginable in Japan, and a continuing challenge even for the UK. As a consequence, California has become a magnet for talent worldwide.

This means there's been a lot of very powerful, very intelligent women in California demanding practical solutions to the issues such a situation creates. The Nordics were (probably) the first to encounter the problem of how to organize society when almost all women work -- and California has learned from their example. Systematically, the kinds of programs supporting working mothers taken for granted in the Nordics have been, or are being put in place in California. And it is the challenges faced by those mothers -- particularly by immigrant mothers -- that will determine the future of California.

---

"How Much of the Time Do You 

Trust The Government in Washington?"

Public Trust in Government Survey, 1958-2013, 

Pew Research Center

available at http://tinyurl.com/knmrhmy

The Pew Research Center has conducted a survey of Americans for over fifty years, asking them "How much of the time do you trust the government in Washington?" The chart on the right graphs the "just about always/most of the time" response. It has gone from 73% in 1960 to 26% today.

From the early 1960's through the present, the US embraced an elite educated at Ivy League (and similar) schools. On a national level, those leaders have systematically lost the confidence of the nation's citizens. The result of this dysfunction is now on display, as our federal government remains closed for business. Returning to Ron Unz for a moment, his criticism should sting when he writes that:

"Over the last few decades America’s ruling elites have been produced largely as a consequence of the particular selection methods adopted by our top national universities in the late 1960s ... the elites they have produced have clearly done a very poor job of leading our country[.]"

California, though, is different from the nation as a whole. For a generation, the basic operating levers of the California Republic were virtually locked via onerous voting requirements -- a statement by the citizens that the government could not be trusted to make decisions. But Californians have now created a supermajority in their legislature -- and they handed it to the party of government, the Democrats.

Despite an 8.9% unemployment rate, and a tax policy that horribly distorts economic behavior, California's future is brightening. Its voters have engaged in fundamental political reform, creating a nonpartisan citizen's commission to conduct redistricting, instituting a top-two system for general elections that defangs partisan primaries, while at the same time increasing funding for education. Its elected officials have simultaneously ended ineffective, pork-barrel economic development programs and have returned control over education to local school districts.

Meanwhile, perhaps the most significant problem for California has long been, both culturally and from a budgetary perspective, the costs associated with health care. It is no mystery why California has enthusiastically embraced the Affordable Care Act. The former US system, where health insurance was typically provided by employers, has long since been abandoned by the UK, Japan, and the Nordics. Not only does the ACA address the massive long term growth of the cost of health care for the State, but delinking health care from employment (and breaking entrepreneurial lock) means starting a business doesn't require changing (or losing) your doctor -- an annoyance for a thirty-something man, but an entirely different matter for an entrepreneurial female (or, one better, a business-minded mother).

This is not to suggest that California's government is pursuing magical thinking; California has a balanced budget, passed on time.  It is to point out that California and the Nordics are converging on a model that bears great promise for the future.

"The Share of California's Residents Who 

Are Foreign Born Has Plateaued at High Levels"

Infographic, "Immigrants In California," 

Public Policy Institute of California, 

available at http://tinyurl.com/lspkztn

There are countries in the world that deserve low ratings on trust -- in many countries fear of the "government" (kleptocracies) is well warranted. Half of the children in California have at least one immigrant parent, and many of the immigrant families who come to California have lived with governments of that type. So when those parents hear that the government cannot be trusted, they believe it, with damaging consequences.

The mother in such a family, oftentimes without any traditional networks to draw on for support, and perhaps not speaking the language, is making decisions about how to raise her children that will chart the future of the State. Education is the escalator to the future for her kids, and especially for her daughters, and that mother must be able to trust her local school -- to trust her government -- for that to happen. And it is the enduring challenge for the State to make sure that she can.

---

"The Economist Staff's Favorite
Covers From The Past 20 Years"
Business Insider, July 22, 2011
 available at http://tinyurl.com/ojvaqf7

I have been asked, what is it about California, that makes it a place of such remarkable achievement for women? Surely government cannot be given credit for this?

I have always been struck by one of the covers of the Economist from March of 2010, which is on the right. The article is here. I caution the reader, the stories related by Xuē Xīnrán are amongst the most terrible, poignant, and emotional you will ever read, and the following passage, describing what she witnessed in the Yimeng area of the Chinese province of Shandong, is heart rending:

"We had scarcely sat down in the kitchen when we heard a moan of pain from the bedroom next door...The cries from the inner room grew louder—and abruptly stopped. There was a low sob, and then a man's gruff voice said accusingly: ‘Useless thing!' ... Suddenly, I thought I heard a slight movement in the slops pail behind me ... to my absolute horror, I saw a tiny foot poking out of the pail. The midwife must have dropped that tiny baby alive into the slops pail! I nearly threw myself at it, but the two policemen [who had accompanied me] held my shoulders in a firm grip. ‘Don't move, you can't save it, it's too late.' ‘But that's...murder...and you're the police!' The little foot was still now. The policemen held on to me for a few more minutes. ‘Doing a baby girl is not a big thing around here,' [an] older woman said comfortingly. ‘That's a living child,' I said in a shaking voice, pointing at the slops pail. ‘It's not a child,' she corrected me. ‘It's a girl baby, and we can't keep it. Around these parts, you can't get by without a son. Girl babies don't count.'”

The horror of a police officer that refuses to protect a female baby is the most dramatic example of why this issue is all about government.

It may surprise the reader, but the message I took from the Economist's series of articles on this subject was one of hope, rather than despair. I remind myself of the following passage when I consider this issue today:

"Baby girls are thus victims of a malign combination of ancient prejudice and modern preferences for small families. Only one country has managed to change this pattern. In the 1990s South Korea had a sex ratio almost as skewed as China's. Now, it is heading towards normality. It has achieved this not deliberately, but because the culture changed. Female education, anti-discrimination suits and equal-rights rulings made son preference seem old-fashioned and unnecessary. The forces of modernity first exacerbated prejudice—then overwhelmed it."

Like South Korea, female education, anti-discrimination suits, and equal-rights rulings are ultimately what can put daughters on an equal footing with sons -- all things that California's government has accomplished. But a country cannot summon the will to take such action, to continue such action, unless it first believes in the effectiveness of the efforts of its own government -- if it possesses the trust of the public.

Where families can trust that their daughters will enjoy the same rights as sons, where the law will protect their daughters, the horror of gendercide can be brought to an end. And the success of California (fittingly, named after a fictional queen) is thus a beacon of hope for families across the world.  It is the place where the immigrant mother can obtain an education, for herself and her daughters, of the highest quality at nearly no cost -- thanks to the California Master Plan for Higher Education. California's commitment to equality reflects the emerging public trust that promises the same rewards for the Golden State as it has for Sweden, Denmark, Finland, Norway and Switzerland.

And so when I consider the future of California, and how the State is not infrequently mocked for its liberal politics and concern with social welfare, I think of the little pink shoes from the Economist's cover.  And as a father, I think of my daughters, and why government matters.

And I have hope for the future of us all.

Hanlon's Razor.

So, the Index-Tribune took the time to review the post I made here yesterday. They acknowledged mistakes.  However, they missed the substantive error with their reporter's analysis.  Further, others have now weighed in, pointing out that the STAR test is obsolete -- a point recognized by their sister publication, The Press Democrat, and echoed by the SVUSD Superintendent in another comment today on this blog.  Finally, while I generally always apply Hanlon's Razor in situations like this, I have concerns regarding the adequacy of the reporter's byline that test the applicability of that adage.

To summarize what happened yesterday, I pointed out that the crushing effects on families of the continued Greater Recession/Lesser Depression is increasingly being revealed by standardized test scores in California.  The I-T's article never mentioned the effect of a depressed economy on educational outcomes for students.  However, I also pointed out the technical shortcomings of the reporter's analysis, including specific factual errors included in the article.

While the paper acknowledged the specific errors I pointed out, to understand the magnitude of the mistakes made means that we have to look at the explanatory PDF on appropriate comparisons of the API prepared by the California Department of Education.

"Invalid Comparisons of the API,"
2012-13 API Reports Information Guide, p. 13
California Department of Education
available at http://tinyurl.com/q7a8ty7  

California helpfully explains invalid comparisons using the API on page 13 (!) of the PDF.  The article in question made a series of these invalid comparisons -- in the same sentence.

The sentence at issue was "[t]he [high] school’s base API was 712 in 2013, down from 723 last year and down from 735 in 2008."

• First, this sentence mistakes the 2013 Growth API for the 2013 Base API. The California Department of Education Guide never even considers the possibility of a mistake like that -- it's such a basic mistake, I think it would kind of blow their minds. This is the mistake the I-T has admitted. 
• Then, the sentence sought to compare what it thought was the 2013 Base API to the 2012 Base API, which is invalid comparison Number One from the list.  The I-T hasn't admitted this mistake yet. 
• Then, the sentence sought to compare the 2013 Growth API to the 2008 Base API, which, coincidentally, is invalid comparison Number Two from the list.  The I-T hasn't admitted this mistake yet, either. 

These errors using the API aren't limited to the discussion of Sonoma Valley High -- they run throughout the discussion of all the other schools as well.  That's why I noted that "most of the multiyear comparisons in the article thus don't really hold up as a consequence." The fact that the article reports nonexistent numbers is one thing, but the basic error here is that the author really doesn't understand how the testing system works. 

Of course, there's yet another issue that's compounding the problems with the reporting in this article, which is that California's schools are in the middle of implementing Common Core.  My opinion (and I'd really like to have completed that blog post by now, but I do actually have to run my law practice and coach soccer, too) is that the individuals behind the creation of Common Core have taken the legal doctrine undergirding the Free Software (Open Source) movement and have implemented it brilliantly to revolutionize American education.

As a consequence, the STAR testing regime has been akin to legacy software for several years now -- and the advent of Common Core renders it obsolete. It's probably time for application retirement.  The Press Democrat's editorial board weighed in this week, calling for exactly that. The Superintendent of SVUSD commented this morning here, and based on that post, I think she concurs. Frankly, I have to agree -- STAR testing results get overwhelmed by demographic noise that obscures the signal concerning educational effectiveness, which has come up on this blog before.

California Schools Guide, Los Angeles Times

Screenshot Taken June 7, 2013.
Screenshot available at http://tinyurl.com/p8ztwrd

Regarding the balance of the comment from the I-T, I'd note that neither of the URLs posted in the comment work, and stating "our print article included a huge chart" is a "defense" that explains many of the problems faced by the newspaper industry.  Further, telling parents "you can go search here" rather than doing the analysis seems to miss the point of reporting.

Finally, I respect the paper when it notes that "[a]s for the reporter being remiss in not speculating why the scores went down, that isn't our role." I agree that the role, as described, is indeed the proper province of the paper. However, I have a hard time reconciling that statement with comments like the one the author of the article left on the LA Times' web site, a screenshot of which is at the right, where the reporter explicitly speculates about what causes test scores to move -- a comment that also suggests a level of partisanship one would think is inappropriate in a reporter.

If there is some relationship, business or otherwise, between the reporter and a private school here in Sonoma Valley, I think that relationship should, at a minimum, be disclosed in the reporter's byline.  Whether the existence of such a relationship should render that individual ineligible to serve as a reporter on the subject of public school test scores is a question that is, however, above my pay grade.

"It's the Economy, ..."

(November 5, 2013 -- For the benefit of readers, please note that the comments on this post include statements from the Sonoma Index-Tribune and the Sonoma Valley Unified School District, that were the subject of a followup post here.)

The Sonoma Index-Tribune wrote an article last week about "declining" test scores in Sonoma Valley. While I'm blogging about another topic at the moment, I did not want to let this one fly by without comment, as there was an interesting data trend buried in the (somewhat confused) article.

The students entering kindergarten this year were born at the end of 2007 and the first part of 2008.  These are children whose "First 5" years of child development took place during an economic collapse.  All of the students in elementary school today have spent at least half of their lives in a depression.  It is my suspicion that the consequences of that experience will be visible in the data for decades.

The data trend stumbled upon by the I-T is not unique to Sonoma Valley. Austin Creek Elementary is a nice example -- it's ranked as a "10" on the statewide rankings, and it's the Santa Rosa school geographically closest to Sonoma Valley.  Austin Creek's 2013 Growth API Report shows the same trends as Sonoma Valley.  Its scores are plummeting -- Hispanic/Latino student performance fell by 51 points, and Socioeconomically Disadvantaged student performance fell even more dramatically, by 113 points.

Austin Creek Elementary

 available at http://tinyurl.com/pjq7vok

Yet it's unlikely that Austin Creek's (or anyone else's) ranking will change as a consequence of these declines, because school rankings are relative things. If everyone starts doing worse, the rankings stay the same.  And that's exactly what's been going on, as nearly every student, across California, has faced an extraordinary tumult and disruption in their daily life.  This is not just due to school budget cuts, but more importantly, because their families are being crushed by an ongoing, drawn out economic depression.

I do, though, want to draw attention to the fact that there were problems with the substance of the reporting, which probably has more to do with the author's unfamiliarity with the testing regime than anything else.

First, the article consistently confuses the difference between Growth API and Base API.  For instance, the article states that "[t]he [high] school’s base API was 712 in 2013, down from 723 last year and down from 735 in 2008." This is easily, demonstrably wrong -- the 2013 Base API won't be released until Spring of 2014; the only thing that's been released for 2013 is the Growth API.  It's not uncommon for the less experienced observer to make that mistake, and most of the multiyear comparisons in the article thus don't really hold up as a consequence.

Further, while the title of the article concerns STAR results, there's next to no discussion of the specific results at all.   The STAR test results are granular -- results come out for each school, each class, and each quintile within each class. An article that gave some guidance for parents on how to understand those reports would have been useful. However, that explanation would not have been a short one, and would have shown some very positive results for the District.   But the article wanted to talk about Base API for 2013 (even though that's not available yet), and unfortunately, a detailed discussion and analysis of the STAR results was a casualty of that decision.

Finally, the tone of the article will probably cause some unnecessary headaches.  Every time one of these pieces comes out, there is a certain amount of rending-of-clothes by the teachers and administrators at SVUSD. But beyond the illustration of the economic impact of the Great Recession/Lesser Depression, I don't think there's much to take away from the article for those interested in improving the condition of Sonoma's schools.