Showing posts with label Sonoma. Show all posts
Showing posts with label Sonoma. Show all posts

Tuesday, September 10, 2013

Hanlon's Razor.

So, the Index-Tribune took the time to review the post I made here yesterday. They acknowledged mistakes.  However, they missed the substantive error with their reporter's analysis.  Further, others have now weighed in, pointing out that the STAR test is obsolete -- a point recognized by their sister publication, The Press Democrat, and echoed by the SVUSD Superintendent in another comment today on this blog.  Finally, while I generally always apply Hanlon's Razor in situations like this, I have concerns regarding the adequacy of the reporter's byline that test the applicability of that adage.

To summarize what happened yesterday, I pointed out that the crushing effects on families of the continued Greater Recession/Lesser Depression is increasingly being revealed by standardized test scores in California.  The I-T's article never mentioned the effect of a depressed economy on educational outcomes for students.  However, I also pointed out the technical shortcomings of the reporter's analysis, including specific factual errors included in the article.

While the paper acknowledged the specific errors I pointed out, to understand the magnitude of the mistakes made means that we have to look at the explanatory PDF on appropriate comparisons of the API prepared by the California Department of Education.

"Invalid Comparisons of the API,"
2012-13 API Reports Information Guide, p. 13
California Department of Education
available at 
http://tinyurl.com/q7a8ty7  
California helpfully explains invalid comparisons using the API on page 13 (!) of the PDF.  The article in question made a series of these invalid comparisons -- in the same sentence.

The sentence at issue was "[t]he [high] school’s base API was 712 in 2013, down from 723 last year and down from 735 in 2008."
  • First, this sentence mistakes the 2013 Growth API for the 2013 Base API. The California Department of Education Guide never even considers the possibility of a mistake like that -- it's such a basic mistake, I think it would kind of blow their minds. This is the mistake the I-T has admitted. 
  • Then, the sentence sought to compare what it thought was the 2013 Base API to the 2012 Base API, which is invalid comparison Number One from the list.  The I-T hasn't admitted this mistake yet. 
  • Then, the sentence sought to compare the 2013 Growth API to the 2008 Base API, which, coincidentally, is invalid comparison Number Two from the list.  The I-T hasn't admitted this mistake yet, either. 
These errors using the API aren't limited to the discussion of Sonoma Valley High -- they run throughout the discussion of all the other schools as well.  That's why I noted that "most of the multiyear comparisons in the article thus don't really hold up as a consequence." The fact that the article reports nonexistent numbers is one thing, but the basic error here is that the author really doesn't understand how the testing system works. 

Of course, there's yet another issue that's compounding the problems with the reporting in this article, which is that California's schools are in the middle of implementing Common Core.  My opinion (and I'd really like to have completed that blog post by now, but I do actually have to run my law practice and coach soccer, too) is that the individuals behind the creation of Common Core have taken the legal doctrine undergirding the Free Software (Open Source) movement and have implemented it brilliantly to revolutionize American education.

As a consequence, the STAR testing regime has been akin to legacy software for several years now -- and the advent of Common Core renders it obsolete. It's probably time for application retirement.  The Press Democrat's editorial board weighed in this week, calling for exactly that. The Superintendent of SVUSD commented this morning here, and based on that post, I think she concurs. Frankly, I have to agree -- STAR testing results get overwhelmed by demographic noise that obscures the signal concerning educational effectiveness, which has come up on this blog before.

California Schools GuideLos Angeles Times
Screenshot Taken June 7, 2013.
Screenshot available at http://tinyurl.com/p8ztwrd
Regarding the balance of the comment from the I-T, I'd note that neither of the URLs posted in the comment work, and stating "our print article included a huge chart" is a "defense" that explains many of the problems faced by the newspaper industry.  Further, telling parents "you can go search here" rather than doing the analysis seems to miss the point of reporting.

Finally, I respect the paper when it notes that "[a]s for the reporter being remiss in not speculating why the scores went down, that isn't our role." I agree that the role, as described, is indeed the proper province of the paper. However, I have a hard time reconciling that statement with comments like the one the author of the article left on the LA Times' web site, a screenshot of which is at the right, where the reporter explicitly speculates about what causes test scores to move -- a comment that also suggests a level of partisanship one would think is inappropriate in a reporter.

If there is some relationship, business or otherwise, between the reporter and a private school here in Sonoma Valley, I think that relationship should, at a minimum, be disclosed in the reporter's byline.  Whether the existence of such a relationship should render that individual ineligible to serve as a reporter on the subject of public school test scores is a question that is, however, above my pay grade.

Monday, September 9, 2013

"It's the Economy, ..."

(November 5, 2013 -- For the benefit of readers, please note that the comments on this post include statements from the Sonoma Index-Tribune and the Sonoma Valley Unified School District, that were the subject of a followup post here.)

The Sonoma Index-Tribune wrote an article last week about "declining" test scores in Sonoma Valley. While I'm blogging about another topic at the moment, I did not want to let this one fly by without comment, as there was an interesting data trend buried in the (somewhat confused) article.

The students entering kindergarten this year were born at the end of 2007 and the first part of 2008.  These are children whose "First 5" years of child development took place during an economic collapse.  All of the students in elementary school today have spent at least half of their lives in a depression.  It is my suspicion that the consequences of that experience will be visible in the data for decades.

The data trend stumbled upon by the I-T is not unique to Sonoma Valley. Austin Creek Elementary is a nice example -- it's ranked as a "10" on the statewide rankings, and it's the Santa Rosa school geographically closest to Sonoma Valley.  Austin Creek's 2013 Growth API Report shows the same trends as Sonoma Valley.  Its scores are plummeting -- Hispanic/Latino student performance fell by 51 points, and Socioeconomically Disadvantaged student performance fell even more dramatically, by 113 points.

Austin Creek Elementary
Yet it's unlikely that Austin Creek's (or anyone else's) ranking will change as a consequence of these declines, because school rankings are relative things. If everyone starts doing worse, the rankings stay the same.  And that's exactly what's been going on, as nearly every student, across California, has faced an extraordinary tumult and disruption in their daily life.  This is not just due to school budget cuts, but more importantly, because their families are being crushed by an ongoing, drawn out economic depression.

I do, though, want to draw attention to the fact that there were problems with the substance of the reporting, which probably has more to do with the author's unfamiliarity with the testing regime than anything else.

First, the article consistently confuses the difference between Growth API and Base API.  For instance, the article states that "[t]he [high] school’s base API was 712 in 2013, down from 723 last year and down from 735 in 2008." This is easily, demonstrably wrong -- the 2013 Base API won't be released until Spring of 2014; the only thing that's been released for 2013 is the Growth API.  It's not uncommon for the less experienced observer to make that mistake, and most of the multiyear comparisons in the article thus don't really hold up as a consequence.

Further, while the title of the article concerns STAR results, there's next to no discussion of the specific results at all.   The STAR test results are granular -- results come out for each school, each class, and each quintile within each class. An article that gave some guidance for parents on how to understand those reports would have been useful. However, that explanation would not have been a short one, and would have shown some very positive results for the District.   But the article wanted to talk about Base API for 2013 (even though that's not available yet), and unfortunately, a detailed discussion and analysis of the STAR results was a casualty of that decision.

Finally, the tone of the article will probably cause some unnecessary headaches.  Every time one of these pieces comes out, there is a certain amount of rending-of-clothes by the teachers and administrators at SVUSD. But beyond the illustration of the economic impact of the Great Recession/Lesser Depression, I don't think there's much to take away from the article for those interested in improving the condition of Sonoma's schools.

Tuesday, July 2, 2013

34 Cents of Your Property Tax Dollar Goes To Our Schools.

So, this post is about increasing the resources available to Sonoma's public schools. There's background here and here. Due to Sonoma Valley's basic aid status, local property tax revenue controls our school funding, and 34 cents of every new property tax dollar goes to our schools.

The Lodge at Sonoma
image available at http://tinyurl.com/mg2dku9 
This post is a long one, and it's in four parts.  The first part explains a bit of the history of school finance since 1971.  The second explains the impact of redevelopment.  The third part describes how property tax revenues can increase.  The fourth gives an example of a specific project, The Lodge At Sonoma -- for were the Lodge to be built today, the school district would get nearly $100,000 each year in additional revenue.

Because of the dramatic impact of property taxes on local schools, this issue should come up in every planning decision made by the City of Sonoma. 

---
"Education is a unique influence on a child's development as a citizen and his [or her] participation in political and community life ... '[t]he pivotal position of education to success in American society and its essential role in opening up to the individual the central experiences of our culture lend it an importance that is undeniable' ... [e]ducation is the lifeline of both the individual and society." Serrano v. Priest (1971), 5 Cal.3d 584 (Serrano I)

California State Supreme Court Chambers.

image available at http://tinyurl.com/k3ewj4k
In 1971, by far, the major source of school revenue was the local real property tax. The amount of revenue a district could raise depended largely on its tax base -- on the assessed valuation of real property within its borders. Then the California State Supreme Court ruled in Serrano v. Priest (1971), 5 Cal.3d 584 (Serrano I), finding that the funding scheme invidiously discriminated against the poor because it made the quality of a child's education a function of the wealth of his parents and neighbors.

The California Legislature responded by establishing a formula that calculates a ceiling on how much local property tax revenue each district should receive. If a K-12 district's local property tax revenue is not sufficient to meet this "revenue limit," the state provides additional funds up to that level. Today, we call most of the ~1,000 school districts in California "revenue limit" districts, because this formula applies to them, and their funding is determined by a (heavily modified) variant of the system created in response to Serrano I. 

The courts ultimately approved the State's plan, which continued to allow a relatively small number of districts to retain a higher level of funding, based on well-above-average local property taxes.  The rub was that if there was other State aid those districts were to receive, that aid would be reduced, dollar for dollar, by the amount that local property taxes exceeded the revenue limit.  These districts became known as "basic aid" districts, a term that comes from the State Constitutional requirement that all students receive a minimum level of state aid, defined as $120 per pupil, regardless of how much local property tax revenue their district receives.

Thus, we ended up with a naming system whereby the "revenue limit" districts are poorer than the "basic aid" districts.  Which is a wonderfully delightful piece of counterintuitive nomenclature.

The initial number of basic aid districts was small -- places like Pasadena and Beverly Hills. In the intervening decades, the number of basic aid districts has continued to increase, to the point now where nearly 15% of all districts are basic aid, and some of those districts are quite large indeed.  After forty years of trying to equalize school funding, places like Sonoma find themselves right back where they were before Serrano I -- that they are entirely dependent on local property tax revenue -- and for such districts, the exception has swallowed the rule.

---

For nearly a generation, local property taxes were essentially irrelevant to school funding in Sonoma Valley.  This was not merely due to the consequences of Serrano I.  Redevelopment also played its part.

As California's Legislative Analyst points out, prior to the dissolution of redevelopment agencies in 2011, most of the growth in property taxes from redevelopment project areas went to the redevelopment agency, rather than other local governments like school districts. In Sonoma, this meant that any significant commercial development, most of which took place inside city limits, almost always saw the increased property tax revenue redirected exclusively to the City of Sonoma, rather than SVUSD.

I don't think that any local leaders were intending that the consequence of this policy would be that the School District would thereby qualify for additional State assistance as a revenue limit district, but that was the actual consequence.

Along came ABX1 26 in 2011, which dissolved all redevelopment agencies. Under the dissolution process, the property tax revenue that formerly went to redevelopment agencies is first used to pay off redevelopment debts and obligations, and the remainder is distributed to local governments, and the school districts receive their share.

When redevelopment went "boom," this was thus a stark change.  It's difficult to point out just how significant the change was.  Indeed, many individuals who are quite knowledgeable regarding California's school system had no idea at all how much property tax was being diverted from the schools through the use of redevelopment, and City officials themselves hotly disputed that money was being redirected away from schools at all -- a point that is now nearly universally recognized to have been incorrect.

---

I've touched on the importance of increased funding for schools in previous posts here and here.  Because local property tax revenue controls school funding for Sonoma Valley due to the district's basic aid status, more revenue depends on increasing local property tax revenue.

How can that happen?  California's Legislative Analyst, as usual, has a nice explanation.  There are three mechanisms -- recently sold properties, newly improved (or newly built) properties, and then Proposition 8 "decline in value" properties.
  • When a property sells, its assessed value resets to the purchase price. This represents additional value that is added to the tax base because the sale price of the property is often much higher than its previous assessed value. 
  • Newly built property and property improvements add new value to the county's tax base when new construction takes place or improvements are made — mainly additions, remodels, and facility expansions — because structures are assessed at market value the year that they are built. 
  • Finally, Proposition 8 "decline in value" properties contribute significantly to growth or decline in a county's tax base because their assessed values may increase or decrease dramatically in any year. A particularly large impact on assessed valuation tends to occur in years when a large number of these properties transfer from Proposition 13 assessment to reduced assessment (due to falling real estate prices) and vice versa in a rising market.
What do these changes in revenue look like, and how can someone determine how much property tax revenue will change?  The San Francisco Chronicle has a nice overview of calculating the numbers for a home purchase, and the rules generally hold up for commercial properties, too.  If you pick a typical Sonoma Valley tax revenue area (TRA), like, say, 006-032 (which covers part of the City of Sonoma), the rate's easily determinable by checking Sonoma County's table here

There's one piece of data that I've not been able to find, though, and that's the portion allocated to each entity of the property tax collected.  If a reader happens to know where Sonoma County's put that data, please forgive this bleg and let me know. 

However, the number (ratio) is not too difficult to determine for a particular entity for a particular year.  For Sonoma Valley Unified, the numerator is the property tax received for a particular year (in 2012-13, that appears to have been $25,176,110).  The denominator is the total assessed value of all real property in the school district ($7,176,806,784) multiplied by the rate (1.108800%). 31.44% of each property tax dollar thus ends up going to SVUSD.

"Allocation of Ad Valorem Property Tax Revenue"
"Understanding California's Property Taxes"
California Legislative Analyst
available at http://tinyurl.com/ljkr59b
That's not the end of the story.  As the Legislative Analyst's graph on the right shows, about 40% of property tax revenue is allocated to K-14 (not K-12) education; about 8% of every property tax dollar heads to the Santa Rosa Junior College.

Further, there's another 8% marked as "redevelopment," which is ending.  The school district will end up with about 3/9ths of that money.

When redevelopment is wrapped up, the school district will therefore receive ~34% of each property tax dollar.

---

So how much money is at issue with a particular project?  Well, consider, for a moment, The Lodge At Sonoma. It's a good example, because the development is unitary -- it's on one piece of land for tax purposes -- and because its development was relatively recent (opening in 2000, IIRC).

The land that the Lodge sits on is valued at $7.2 million for property tax purposes, and the improvements (the structures) are valued at ~$19.2 million.  The total assessment is thus $26,387,300.   Knowing that the effective tax rate for the parcel is 1.108800%, we just multiply those two numbers and get $292,582.38 per year.  The official "bill" is $381,108.52, but that includes non-property tax water charges, among other things.
City of Sonoma GIS, APN 128-261-009
"The Lodge at Sonoma"
1325 Broadway, Sonoma CA
available at http://tinyurl.com/nxfz9rk  

So how much does the school district get? You'd think $91,987.90 if you just applied the rules I posted above. You'd be completely incorrect, but at least you'd have followed instructions properly.

The reason you'd be wrong is due to redevelopment.  As Bob Klose reported in the Press Democrat on June 14, 1998, the Lodge was built in part with money from the City of Sonoma's Redevelopment Agency. Thus, the entire increase in property tax went to the City of Sonoma.  The school district's share was limited to the assessed value of the parcel prior to the project.  I've looked at the records, and it looks like the assessed value of the parcel before the construction of the Lodge was $425,000.  The assessed value may have been higher than that due to some increase over time under Prop. 13, so call it $600,000 as an estimate.  The school district's share of the property tax collected is ~$2,091.64.

It is probably therefore unsurprising that people have rarely brought up the school district budget at planning commission meetings in the City of Sonoma.  But they probably should. Because were the Lodge to be built today, the school district would get approximately $100,000 each year.  That's equivalent to an Impact 100 donation every twelve months.

---

A final point: because Preserving Sonoma is under discussion in town, I can certainly see how someone like Darius Anderson would try to use a post like this to his advantage. This post, though, is about more than the debate about any single project.  It's about a factor that's just not being weighed at all in the planning process in Sonoma. 

Because commercial development (and, increasingly, tourist-oriented development) is concentrated in the City of Sonoma, the impact on our schools of planning decisions in the long run is profound indeed. To paraphrase Serrano I, education is a unique influence on a child's development as a citizen and her or his participation in political and community life. The pivotal position of education to success in American society and its essential role in opening up to the individual the central experiences of our culture lend it an importance that is undeniable -- education is the lifeline of both the individual and society. Failing to consider education in the planning process is a disservice not least to our students, but to our entire Valley.  

Wednesday, February 13, 2013

A Billion Reasons to Like Peg Melnik.

Peg Melnik
Peg Melnik writes a nice blog for the Press Democrat which weighs in on interesting wine and tourism related issues in Napa and Sonoma from time to time. She's a great resource for anyone visiting the area, and her book, the Explorer's Guide to Napa & Sonoma, is now in its 9th Edition (you can pick up a copy on Amazon). She writes with her husband, Tim Fish, who is an associate editor for Wine Spectator.  

She put up a piece a few weeks ago about Sonoma Valley rebranding itself, a screenshot of which is at the right. It's an interesting article, but there was just one thing that really jumped out at me ... $7 million spent in Sonoma County on tourism per year? 
Peg Melnik, "Identity Crisis?"
Tasting Room Blog, The Press Democrat, Jan. 29, 2013
available at http://tinyurl.com/cgxg555

Someone gave Peg some very bad statistics.  The actual number in 2011, was, of course, much higher - about $1.32 billion.

By way of contrast, the value of the County's entire grape crop in 2011 was $347 million.

The figures are courtesy of Sonoma County's Economic Development Board, whose graph on the issue I excerpt at the right, and the Sonoma County Agricultural Commission.  

So -- Peg -- don't underestimate the effects of books like yours on the local economy ...

Moodys.com, "Annual Tourism Report, 2012"
Sonoma County Economic Development Board
available at http://tinyurl.com/ckwm5t3
These statistics are part of the very comprehensive economic reporting done on Sonoma County on a yearly basis.  The statistics on the EDB's web site go back more than a decade, and reading through them (and watching the changing predictions) is often illuminating.  These materials, like the UCLA Anderson Forecast and the Moody's annual reports to the County, are a great for answering questions about the economy for anyone involved in local government.

Dan Walters on Education Funding, Part 2.

I've blogged previously about Dan Walters and his views on California's budget. Dan has access to most of official Sacramento, and I generally believe that if he's thinking and writing about a certain problem, it is something that most of Sacramento already is (or soon will be) thinking about, too.

California State Assembly
The column that has my attention is about ELL.  Dan points out that Jerry Brown's latest plan for education reform "provides a 'base grant' of about $6,800 per student and then, over several years, adds as much as $5,000 to districts that have above-average concentrations of English learners and students who qualify for free or reduced-price lunches[.]" Dan Walters points to Los Angeles Unified as the potential biggest winner from this change in policy, noting that 76% of LA Unified is Latino or Hispanic.

However, Dan did miss a bit of the story; just pointing out the racial demographics of a school district isn't necessarily a good proxy for how many ELL students there are.  Those numbers are available.  27.3% of LA Unified, for example, are ELL students -- 180,495 out of 662,140.  Sonoma Valley's numbers are available, too.  31.7% of Sonoma Valley Unified students are ELL students -- 1,483 out of 4,673.

It's probable that Sonoma Valley Unified wouldn't receive the maximum grant under the program, because the calculation includes free and reduced price lunch enrollment, where SVUSD is just about at the Statewide average. But if Sonoma Valley Unified got even close to the maximum proposed grant, that would push Sonoma Valley's funding per student to somewhere near $11,800 per student -- which would add more than $10 million per year to the District's budget -- and which would bring total funding fairly close to the level enjoyed by, say, Healdsburg.

 I doubt Jerry Brown's plan will be enacted as proposed -- too many wealthy suburban school districts are highly motivated to fight it. But the specifics of the plan are less important at this point in the budget cycle than the simple fact that the issue's been identified -- that the battlefield in Sacramento has been chosen, and it's funding for ELL-impacted schools.

I suspect the choice by the Governor was a good one.

Finally, since it's "Catch Up With Dan Walters Day" for me, I also noted that Dan took on the "shadow budget" in a recent column, pointing out that the general fund (~$91 billion) does not equal the budget (~$225 billion).  He argues that the practice of reporting only on the balance status of the general fund tends to deceive voters.  I agree, Dan, I agree.

Tuesday, February 5, 2013

The Philosophy of Data and Sonoma's SAT Scores.

David Brooks, with Mark Shields and Judy Woodruff
"Weekly Political Wrap," PBS NewsHour
 available at http://tinyurl.com/abgqguu.
David Brooks' column regularly features in my weekly reading list, and his segment on the NewsHour with Mark Shields on PBS is a Friday night favorite at our house.  According to Wikipedia, he's "the sort of conservative pundit that liberals like, someone who is 'sophisticated' and 'engages with' the liberal agenda[.]" Today, his column's interesting because it's all about data, but it's specifically interesting for the observation he makes that there are two things that data does really well -- it can illuminate patterns of behavior we haven’t yet noticed, and it's very good at exposing when our intuitive view of reality is wrong.


"Highest Average SAT Scores in Sonoma County"
California Schools Guide, Los Angeles Times
His two points resonate for me, because I've been looking over Sonoma Valley High's SAT scores over the past two weeks, in response to some data sent to me by a concerned friend.  

The table in question showed that Sonoma Valley High's SAT scores ranked 11th out of 16 public schools in Sonoma County. The presumption was that these schools were all comparable to Sonoma Valley High. 

 The individual wondered what conclusions could be drawn about the performance of Sonoma Valley High as a consequence. So, I took a look.

The data comes from the Los Angeles Times, as a part of their California Schools Guide. As you can see, Sonoma Valley High is right behind Piner High and ahead of Windsor High.  Santa Rosa High is at the top, Technology High in Rohnert Park's around the middle ... 

Wait a minute.  Piner High is above Sonoma Valley High? For someone who grew up in Sonoma County, that data point is completely implausible.  I just knew there had to be some real problems with the method used to create the database, given what I know about Piner High.

Number of Test Takers Versus Size of School.
Data from "Highest Average SAT Scores in Sonoma County"
California Schools Guide, Los Angeles Times
available at http://tinyurl.com/bglbet5
Thankfully, the LA Times includes the number of students in the student bodies of the schools -- and also includes the number of total test takers at each school. I gathered up the data (it wasn't exactly conveniently arranged), and ranked the schools by number of test takers.  That table's on the right.  

Technology High comes in at the top, which shouldn't really surprise anyone.  It's the highest ranked high school in Sonoma County based on API scores.  The program (it's a magnet school) is designed to send its students to college (the school itself is located on the campus of Sonoma State University).  An awful lot of seniors at Tech High are taking the SAT, and the ones that aren't may very well be taking the ACT instead. 

I'd say that Sonoma Valley should be proud that it's managing to motivate so many of its seniors to take the SAT.  Only Analy and the Petaluma schools do better, and even then it's not by much.  Piner, meanwhile, comes in nearly dead last, with only 16.7% of its students taking the SAT. 

Thus, to me, it looks like there's just a very, very serious problem in trying to draw any conclusions from ranking high schools by average test scores on the SAT, when there's a large self-selection bias taking place in the pool of test takers -- you don't have to take the SAT, after all.  You have to sign up for it (and pay for it!).  At Piner High, not many students are doing so -- in stark contrast to Sonoma Valley High.

An illustration of the Normal Curve.
From "Normal Distribution," Wikipedia
available at http://tinyurl.com/m2gx6
OK, but what about the raw scores -- can we compare the test scores on the SAT by trying to control for self selection bias?  Can we "correct" the data to try to draw conclusions? Well, if we just assume that the distribution for each school is unimodal, symmetrical, and bell-shaped -- that the distribution is normal ...

Such an effort immediately runs into a problem, which is that some high schools are unimodal, and some (like Sonoma Valley) are bimodal, and that the data is anything but symmetric. The data for the bimodal schools looks like the table at the right,  where g/t is an SAT score, and t is the number of test takers that got that score.
A Bimodal, Asymmetric Distribution.
From "Unimodality," Wikipedia
available at http://tinyurl.com/aowq5jn


Given that I knew there was an oddity in the data, I deliberately focused on only those schools that are bimodal. Thus, this comparison is for high schools where no single ethnic group constitutes more than 70% of the population -- those schools where Spanish-English dual immersion (which I happen to be interested in for my kids) is generally possible.

Pursuing that idea, I took a stab at coming up with, at least theoretically, what the 50th percentile and the standard deviation for the SAT score would be for each of these schools, presuming the sample (the self-selecting students) are all on the right end of a normal distribution (that they're more-or-less the best test takers).

Making the (heroic?) assumptions outlined above, I did what I could to estimate the score for a student who was 1 SD above average --  and correcting for different sample sizes -- again, assuming the data is normal, which it isn't. The only reason doing something like this could make any sense at all is that these schools all have the same issue with their data -- they're all bimodal and asymmetric (admittedly to different degrees). Further, while the actual 1 SD performance -- roughly the 85th percentile of test takers -- is quite possibly higher than these estimates indicate, it bears repeating that it is the relative differences I'm more interested in here.

And finally, I put in per-pupil spending for 2007 -- the last year before the real estate bubble made a lot of oddities hit these numbers, and the only year I had data for all of them -- for each of these schools.

Estimated SAT @~85% versus Spending Per Pupil,
Selected Sonoma County and Napa County Schools.
Data from "California Schools Guide," Los Angeles Times,
available at http://tinyurl.com/bglbet5, and the 
"Federal Education Budget Project," New America Foundation,
The end result of this is the table on the right. Sonoma Valley does pretty well, all things considered.  Sonoma Valley has less funding per pupil than the lowest scoring school, yet still lands in the upper half of the table.

But the story is really the spending-per-pupil. To try to measure Sonoma Valley against, say, Healdsburg, when Healdsburg has 33% more money per student, is hideously unfair.  An extra $1.2 million a year (the amount necessary to match Napa) would significantly help Sonoma Valley Unified.  And what about giving Sonoma Valley Unified an extra $12 million a year -- the amount necessary to match Healdsburg? I bet SVUSD could accomplish an awful lot with that much money ...

The whole exercise of looking at this data certainly illuminated one pattern that I hadn't noticed, which was the very significant disparity inside Sonoma County concerning school funding.  I didn't have any idea that Healdsburg was funding its schools as well as it is, and frankly, it's to Healdsburg's credit.  But the really useful part is that I think it again exposes that most people's intuitive view of Sonoma Valley High is wrong -- Sonoma Valley High, and Windsor to a lesser degree, look like they're overachieving, given what they have to work with financially.  Further, Sonoma Valley's performance is better than one of the two closest high schools (Vintage) and is in striking distance of the other.

I've been speculating why the idea that "Sonoma Valley High is a poor performer" has gotten entrenched in the community.  I was tempted to mine the Index-Tribune's archives, to perform a textual analysis to see if I can find harder evidence, in the form of a shift in the changing language used to describe the High School.  But I think the story here doesn't need that much data in order to grasp the narrative.

Sonoma's a fairly rural, agricultural place.  My hunch is that many such communities began to get a little bit skeptical of their high schools sometime in the late 1950's -- think of the charmingly quaint anti-authoritarianism of Grease.  Such grousing was probably mostly harmless until the near-revolution that took place in American Society after 1968.  When school funding really took a hit a decade later, and the decrease in funding began to bite, the slow degradation of the physical plant probably kept the idea alive in many people's minds that Sonoma Valley High was a troubled place -- now, think Fast Times at Ridgemont High.

Meanwhile, the population of the Valley became more stratified as Sonoma gained an allure as a high-end destination as a consequence of the "Judgment of Paris," and the significant population growth between 1978 and 1986 meant the High School had to grow physically while dealing with less funding per pupil from property taxes. Fast forward to the present, when the demographic profile of the school district is changing as Sonoma continues to become ever wealthier, and I suspect the older idea in people's minds that "there's a problem at the High School" gets triggered fairly easily. Even if the evidence doesn't appear to be there to support the argument, the fear now is something along the lines of Dangerous Minds, perhaps.

But the data shows that Sonoma Valley High's doing a surprisingly good job of encouraging its students to apply to college, despite the fact that it makes the school look like it's underperforming. Further, the school looks like it's overachieving next to its peers as far as performance on the SAT is concerned, despite the funding situation.  If anything, this starts looking just a little bit like a case of Stand and Deliver. Again, not the conventional wisdom -- but perhaps in keeping with David Brook's "Philosophy of Data."

Monday, January 28, 2013

So How Are Things At Sonoma Valley High?

Lorna Sheridan
Lorna Sheridan wrote a very nice article in today's Sonoma Index-Tribune about Lynn Fitzpatrick, the interim principal at Sonoma Valley High.  Lorna's a bright individual (not least because she went to Princeton) with a good deal of work experience in financial services (including American Express, one of my personal favorites).  When it comes to understanding the educational arms race that admission to an elite university has become, she's a great go-to.

Having said that, there's one line in the article that jumped out at me, and probably will for other readers as well: "... the high school’s standardized test scores lag behind other high schools in the area with similar demographics[.]"

Is Lorna right?

Statewide Rank
Sonoma and Napa County Public Schools
"Education Statistics of California," 
Google Public Data Explorer
On the left is the graph of the Statewide Rank for a set of high schools around Sonoma Valley (Petaluma, Santa Rosa, Healdsburg, Napa).  Sonoma Valley comes in at a 5 -- the same as Vintage, and the middle of the chart.  About half of the time, Sonoma Valley does a little better than Napa High and Healdsburg High, and about half the time, a little worse (we're talking about single rank moves here).  Maria Carrillo's way out in front (Austin Creek Elementary, the highest scoring elementary school in Sonoma County, feeds into Maria Carrillo). Elsie Allen (in Roseland, in Santa Rosa) and Piner trail here, which is more or less expected if you're familiar with the area's high schools.

So Sonoma Valley's the middle of the pack, right?  Well, not exactly.  The data's more complicated than that.

Academic Performance Index, "White"
Sonoma and Napa County Public Schools
"Education Statistics of California," 
Google Public Data Explorer
For all ethnicities, the scores at Sonoma Valley High are gradually increasing over time.  As an example, on the right is the API score of (their word, not mine) the "white" ethnicity at Sonoma Valley. I've also selected the other, comparable schools in Napa and Sonoma counties.  In absolute terms, the Sonoma Valley High students come in at an 807.  800 is the statewide goal, according to the California Department of Education.  Interestingly, the schools clustered around Sonoma Valley High on this graph are two levels higher on the Statewide Rank -- Sonoma Valley's 807 and a 5 Statewide, while Montgomery is an 816 and a 7 Statewide. I think there are quite a few admissions directors at elite universities that would be surprised to find that the actual test score differences are so trivial.

Ethnicity of Students, Hispanic or Latino
Sonoma and Napa County Public Schools
"Education Statistics of California," 
Google Public Data Explorer
It is entirely possible for a ranking to decline while the API scores for all subgroups increase -- for instance, if there is a significant change in the population of the schools. The graph on the right shows that the change that appears in the elementary schools in Sonoma Valley is present here, too. To repeat my earlier post, I believe this graph obscures more than it reveals when evaluating a school's performance -- on the question of whether a school is improving or worsening in its job of teaching students, the percentages of ethnic groups at the school is irrelevant.

Coming back to the main point, though, and as you might gather by now, I more or less disagree with Lorna on the condition of Sonoma Valley High.  The differences between schools that are perceived as being stronger, or better, or "whatever" in relation to Sonoma Valley are insignificant -- those schools are actually turning in performances that are essentially the same for similarly situated students.

The generalized sense that the local public high school is in "decline," which Lorna notes in passing, is, I think, really just a symptom of the budgetary chaos imposed on the schools since 1978 as a result of California's chronically dysfunctional state government.  Physically, the schools have felt like they're getting worse, due to the significant amounts of deferred maintenance at several campuses, the aging athletic facilities, and the layoffs of the janitorial staff. I think it bears repeating, though, that the data shows that the teachers have nonetheless managed to improve student performance on standardized tests over the past decade.  I am aware that this is in stark disagreement with conventional wisdom in Sonoma Valley -- but I am also aware that this improving performance is to the district's (deserved) credit, and while I was certainly prepared to concur with conventional wisdom when I started looking at this issue, the data just doesn't support those conclusions.

Friday, January 25, 2013

Autocorrecting Driving?

I'm frequently amused by the substitutions made by iPhones when they're trying to automatically make corrections.  The phenomenon has spawned a series of websites, and one of the funnier posts I've seen is on the right.

The reason I bring it up today is because of the posts I've been reading discussing the concept of "driverless" cars.  There's been some talk on the subject in the last day or two from  Paul Krugman and Felix Salmon. I've been thinking about it a bit because of the safety concerns that have cropped up here in Sonoma regarding pedestrians in crosswalks.  By some accounts, the technology is now at a point where your car can detect whether someone's in the road (crosswalk) in front of you, and slow your car down, automatically. Further, the technology has the potential to dramatically increase the capacity of roadways through platooning, which has important consequences from an urban planning perspective. 

However, while I am impressed with the potential of the new technologies, I must express a certain degree of skepticism concerning how quickly its benefits will be achieved.  After the perils of autocorrect, Siri, and Maps on the iPhone, I think caution is prudent for us all ...

Monday, January 21, 2013

The Streets Should Fit the Trees.

I was visiting family in Houston last week. Driving around, I couldn't help noticing signs like the one on the left. The Houston Independent School District  went to their voters in the last election with a pretty big ask - just short of $2 billion. I had suspected the bond election had failed -- the generally accepted wisdom regarding education in Texas is that it's underfunded and lacks support. But I was wrong -- Houston came out 2-1 in favor.

$2 billion is a lot of money, but it's good to keep it in perspective.  HISD has over 200,000 students, while my little town, Sonoma, has about 4,600 in its school district.  So for Sonoma, that'd be around a $40 million bond measure.  Interestingly, Sonoma had a bond measure of about that size in 2010 -- Measure H -- and it received almost exactly the same support as the Houston bond.

OK John, so what?

Texas' unemployment rate at the time of the Houston bond measure was 6.1%. California's at the time of Measure H was 12.2%.

6% unemployment is right about what (monetarist) economists generally think is the (suspected) value of the NAIRU, or the non-accelerating inflation rate of unemployment. This is basically the level of unemployment that keeps a lid on inflation.  In the most simple terms, monetarists argue that some unemployment is a result of people doing things like switching jobs, and not because the economy is underperforming, and that if unemployment's at 6%, the economy's probably operating fairly normally.  Thus, there aren't a lot of politicians that get terribly excited (worried?) about 6% unemployment.

12.2% is entirely different.  12.2% is considered disastrous.  American Conservatives (well, at least the Wall Street Journal) argue, for example, that the French economy is in a severe crisis, with stagnant growth, steadily rising per unit labor costs, and chronically high unemployment.  However, France's unemployment rate has never been over 11.5% -- at the time the Wall Street Journal wrote those words, it was 10.1%.  You can imagine what those editorial writers think of California's economy.

Most people reading this will see where I'm going, but it never hurts to spell it out: Sonoma's voters approved their bond measure in the middle of an economic collapse. What would the voters support if the California economy was turning in a performance like that enjoyed by the Houston voters?

This segues into a broader issue, though.  While economics and education generally aren't linked in discussions in California, improving schools and reaching out to students can only help so much when the child's parents are out of work. In 2009, Ann Huff Stevens and Jessamyn Schaller of UC Davis published a study that examined the relationship between parental job loss and children’s academic achievement. In 2011, the Washington Post's Suzy Khimm drew attention to the story.  The research  carefully controlled for distorting effects, and after doing so, determined that parental job loss increased a child's chances of being held back a grade by 15%.  After properly excluding any other possible causal factor, the study determined that the effect completely disappeared if the parent had simply been able to get a new job. While the study's metric was solely students who failed a grade, I think it is safe to conclude that the decrease in academic performance is probably far broader.

I can hear Captain Obvious in the audience, thinking to himself "John just proved that a parent losing their job can screw up their kid's life!" So what's the point?

"Jobs Since the Recession"
Cal Facts 2013
California Legislative Analyst
A lot of people sense that the economy has problems, but they're unclear on the specifics. Why does the economy have problems?

In aggregate, the problems faced by California are really linked to a couple of sectors. Since 2007, California has lost nearly 900,000 jobs in construction, manufacturing, and transportation. Perhaps 100,000 of those jobs have come back since 2010 -- but none have been in construction or manufacturing.

Manufacturing's losses are significant but I think they're not as directly relevant to a place like Sonoma as they are to, say, Los Angeles. Los Angeles (surprisingly to many) is home to lots of manufacturers --  in everything from  clothing to aerospace to jewelry.  

LA's problem is not Sonoma Valley's problem. Sonoma Valley's problem is construction.

The easy rejoinder there is "well, there's been a collapse in real estate values, what do you expect? Nobody wants to build houses."

That argument might work for the State at large, but Sonoma Valley's home prices have done better than average, and there are substantial efforts to engage in new construction, particularly concerning facilities that cater to tourism (leisure & hospitality), professional services, health services, and agriculture.  There's even some home construction (especially remodeling) going on.  Businessmen and women are trying to get things done.

Instead, in nearly every single instance, the same set of issues are coming up over and over again. As a practicing attorney, I can tell you that each project that someone comes to talk to me about has (or, invariably, will have) the same basic problem, which causes intelligent and hard-working businesspeople to throw up their hands and oftentimes abandon the entire effort.

That issue is California's thicket of minor legislation.  The problem is not a new one. In 2004, The Economist newspaper (magazine) wrote an article on the condition of California's economy, after the dot-com collapse but before the housing implosion.  To summarize their conclusions, unlike some other states in the U.S., there are different layers of overlapping government in California, and often those governments work at cross-purposes. The survey noted that places like Texas offer businesses one-stop shops, but that California presents more of an obstacle course. The authors pointed out that while costs are a problem, the bigger problem is generally unpredictability, with the sudden imposition of new rules (and charges) causing projects to slow down or to stop altogether -- the destructive consequence of the thicket of minor legislation.

I see this going on nearly every day.  From urban infill to brownfield redevelopment to simple remodel projects ... even habitat restoration efforts run into a near-impentrable maze of agencies and authorities, whether local, county, state or federal in nature.  Guiding projects through this effort is, of course, what lawyers do, but from experience, I know that business depends on understandable rules, and our government generally fails us -- that we fail ourselves -- in that department.

No one wants to make Thneedville's mistake in Sonoma Valley, or indeed anywhere in California. Our natural endowment is why many of us are here in the first place.  That's not the issue. Reducing the impact of the thicket of minor legislation when we seek to maintain existing structures or reuse already-developed sites would substantially benefit our local economy, and start strengthening the economic base that can provide increased financial support for our schools.  I can tell you that the Economist is on to something when they point to the thicket of minor legislation as the problem holding back the State -- and ultimately, holding back our efforts to improve education.

Which really brings me to my final point.  In 2009, my City decided that nearly every single tree on my street should be cut down -- no joke! If there's one issue that can really get my (and my wife's) attention, it's someone who wants to chop down trees without a (very good) reason.  After looking at the plans the City put forward, I pointed out at the City's public meeting on the issue that the City was trying to make the trees fit the street, when what the City really should have been doing was making the street fit the trees.

The problem that both my Valley and my State have encountered is that the economy and education, like the streets and the trees, need to fit one another.  We should not forget that one is more important than the other -- but we also have to pay attention to both if we want to be able to take care of either.

Monday, January 7, 2013

Dan Walters and Education Funding.

Dan Walters.
Dan Walters is a columnist for the Sacramento Bee.  While he's not exactly eye-candy to look at, his columns are a great resource for understanding what's taking place in Sacramento.  He's a bit iconoclastic and isn't beholden to any particular interest group, so I read him to get an idea where California's going politically in the next 12-24 months.

He's behind the curve right now, simply because the story has changed so much in the past two months.  But he's catching up.  His column today notes that the California budget situation has gotten much better (this after the Legislative Analyst reported the same two months ago), and that the new issues in the new year are, amazingly, going to revolve around how to spend revenue, rather than how to impose cuts.


"Significant Budget Shortfalls Since 2001"
Cal Facts 2013
California Legislative Analyst
This will not be a small fight, particularly because, as he's noted over the past week, there are also serious and substantial efforts to revise Prop. 13.  Further, at the same time, Jerry Brown is attempting to change the funding formula for the educational system.  This would involve, crucially, increasing funding for ESL programs, which would be significant change for, at a minimum, schools facing a changing demographic profile. Dan points out these changes are possible because of the Democratic supermajority in the legislature (something that may not last long).

I bring it up to note that, for many Californians, the story regarding education has generally been nothing but cuts for a decade, perhaps longer.  This has led to fatigue -- the constant crisis has frustrated many, leading to a certain sense of defeatism concerning the ability of the State, ever, to provide adequately for K-12 education.

I sympathize with anyone that thinks that -- California politics have been anything but functional for thirty years and there's a sense that's crept in that the State will always be this way, and that serious change (let alone reform) is beyond the political's system's reach.  However, this time, it really does look like it's going to be different.

Finally, Dan's good at pointing to useful reference resources for people who are interested in governmental issues.  The graph at the right is one example of that, from the Legislative Analyst's "Cal Facts" publication.  For anyone who wants to get a foundational feel for these problems, the pamphlet is a good place to start, even if only for its explanation of the major propositions that affect (hamstring?) California's government ...

Friday, January 4, 2013

Coronagraphs and SVUSD.

Arthur Eddington's photograph
of the May 29, 1919 solar eclipse.
available at http://tinyurl.com/n46h3z
The photograph at the right is probably the most important one taken in the 20th Century. It was taken of an eclipse of the Sun by Arthur Eddington, on May 29, 1919, from the island of Príncipe in the Gulf of Guinea, off the west coast of Africa.  In the upper right portion of the Sun's corona, you can make out a small dot, which is the Hyades star cluster.

The reason it was so important, was not because the position of the Hyades star cluster was wrong, although that was definitely the case. The Hyades star cluster has "moved" in this photograph when compared with all of the other stars in the sky -- but the problem was that it moved too much.  

Isaac Newton had predicted hundreds of years before that gravity could bend light.  He had even predicted how much the light would bend.  But in this rare instance, Isaac Newton was wrong -- the light from the star cluster had moved twice as much as he had expected. 

But someone else's theory was right -- a former Swiss patent clerk by the name of Albert Einstein.  The apparent movement of the Hyades star cluster was exactly the amount that Einstein's theory predicted when light traveled near the warp in spacetime induced by the Sun's gravitational field. 

OK, nice story, what's the point, John? 

This post isn't about relativity, or even really astronomy. It's about how this experiment was possible in the first place. The Moon, in an eclipse, functions as a natural coronagraph. Like any coronagraph, the moon allows the viewer to see the light from the corona of the sun (and the light from the Hyades star cluster), which would otherwise be hidden in the Sun's bright glare. Without the moon being in this position, measuring the degree of the apparent movement of the light (and the proof of general relativity) would have been well-nigh impossible. 

Why am I talking about this today? Well, over the past week or two, I've been looking at the test scores for Sonoma Valley's elementary schools, trying to develop a picture of what's taking place from the data.  In my last post, I used a very limited set of data to illustrate a point -- the overall rankings of the schools measured by API scores, and then the scores of a particular subgroup across different schools over time.  Those two pieces of data point in different directions.  

In talking over some of the comments made by some very intelligent people who reviewed the post on Facebook, it became clear that they were aware that I was using something akin to a "social science coronograph" to block out a particular piece of data, which was so bright and glaring that it would make it impossible to view what I'm really interested in. 
Ethnicity of Students, Hispanic or Latino
Sonoma Valley Public Schools
"Education Statistics of California," 
Google Public Data Explorer
I deliberately blocked out that piece of data, because, like the Sun itself in Arthur Eddington's experiment, it obscures more than it reveals.  But I don't want anyone to think I'm hiding the football here -- far from it -- it's just that this particular piece of data drowns out the signal.  

That piece of data is the graph at the right. It's the percentage of the population at each elementary school that is Hispanic or Latino. This graph is essentially the reverse of the API score decline graph I posted yesterday.  Google's setup isn't letting me graph ESL students, but the graph is essentially similar when you review it at the State of California's site. This demographic change happens to make the API scores vary wildly, which was one of the big reasons I picked a narrow subgroup when evaluating the schools over time. 

As you can see from the data, Sassarini, where I'm thinking about sending my daughter, is about 2/3rds Hispanic or Latino as of 2011. That's not a drawback from our family's perspective. My wife and I have been making an effort to ensure our daughters have the opportunity to learn Spanish, and perhaps the great overlooked point in learning a language is that you need to listen to it.  The fact that our daughter would thus be exposed to a great deal of Spanish on the playground is, frankly, a positive. I am, however, aware that there are parents that disagree.

But on the question of whether a school is improving or worsening in its job of teaching children like my daughter, the ethnic makeup of the school is irrelevant. And that's why, in making that determination, I pulled out a virtual coronagraph and blocked the glare from this graph ... it's not something that is inappropriate for consideration -- it's just that I don't think it reveals anything about the statistical performance at a particular school, and that's the thing I'm interested in measuring at this point.

There's a second issue that some of the comments were pointing out.  This was the fact that I left off the charter schools on these graphs, and that I didn't point out the scores of some of the specialty programs that exist in the school district, such as the dual immersion program at Flowery.  

The second point is more straightforward to address than the first -- the dual immersion students appear to be included in the raw Flowery numbers.  

The first point is nuanced.  The charters have unusual characteristics -- for instance, Woodland Star is a Waldorf school, and trying to measure a Waldorf school via something like an API score would be pointless and ultimately deceptive. If you want to understand a Waldorf school, you're going to need to meet the families involved.  I can tell you from experience that I am impressed with the Woodland Star families.  So, I would toss out the API score as a measure completely when evaluating Woodland Star. 

Sonoma Charter has a similar problem, but in the opposite direction. Sonoma Charter's Hispanic/Latino numbers are consistently around 15%, which is generally about half that of Prestwood.  When that issue is adjusted for, the remaining 16 point differential between the students at the two schools is modest at best and is probably irrelevant. But as a parent who takes Spanish education seriously for my kids, my evaluation of the quality of the school is impacted by the skewed characteristics of the population of the student body.  

Finally, on the point of whether test scores are a good thing for parents to look at -- I think they reveal a lot of things, many of which were not necessarily intended by the creators of the test.  Teaching to a test is always a concern, and a well rounded education is a means to an end -- the ability to function as an informed citizen in modern society.  Cram courses to learn how to fill out little bubbles on a Scantron can improve test scores but do little in terms of educating children.  

However, there are a lot of parents in Sonoma whose children will probably go on to national-level (if not international level) institutions for postsecondary education, and dealing with the reality of standardized testing is a necessary part of preparation for navigating those institutions. Scores should be taken with a grain of salt (or ignored completely when evaluating specific programs), but ignorance of the nature of the testing regime is probably more dangerous for parents and children in the long run.