To summarize what happened yesterday, I pointed out that the crushing effects on families of the continued Greater Recession/Lesser Depression is increasingly being revealed by standardized test scores in California. The I-T's article never mentioned the effect of a depressed economy on educational outcomes for students. However, I also pointed out the technical shortcomings of the reporter's analysis, including specific factual errors included in the article.
While the paper acknowledged the specific errors I pointed out, to understand the magnitude of the mistakes made means that we have to look at the explanatory PDF on appropriate comparisons of the API prepared by the California Department of Education.
 |
| "Invalid Comparisons of the API," 2012-13 API Reports Information Guide, p. 13 California Department of Education available at http://tinyurl.com/q7a8ty7 |
The sentence at issue was "[t]he [high] school’s base API was 712 in 2013, down from 723 last year and down from 735 in 2008."
- First, this sentence mistakes the 2013 Growth API for the 2013 Base API. The California Department of Education Guide never even considers the possibility of a mistake like that -- it's such a basic mistake, I think it would kind of blow their minds. This is the mistake the I-T has admitted.
- Then, the sentence sought to compare what it thought was the 2013 Base API to the 2012 Base API, which is invalid comparison Number One from the list. The I-T hasn't admitted this mistake yet.
- Then, the sentence sought to compare the 2013 Growth API to the 2008 Base API, which, coincidentally, is invalid comparison Number Two from the list. The I-T hasn't admitted this mistake yet, either.
These errors using the API aren't limited to the discussion of Sonoma Valley High -- they run throughout the discussion of all the other schools as well. That's why I noted that "most of the multiyear comparisons in the article thus don't really hold up as a consequence." The fact that the article reports nonexistent numbers is one thing, but the basic error here is that the author really doesn't understand how the testing system works.
Of course, there's yet another issue that's compounding the problems with the reporting in this article, which is that California's schools are in the middle of implementing Common Core. My opinion (and I'd really like to have completed that blog post by now, but I do actually have to run my law practice and coach soccer, too) is that the individuals behind the creation of Common Core have taken the legal doctrine undergirding the Free Software (Open Source) movement and have implemented it brilliantly to revolutionize American education.
As a consequence, the STAR testing regime has been akin to legacy software for several years now -- and the advent of Common Core renders it obsolete. It's probably time for application retirement. The Press Democrat's editorial board weighed in this week, calling for exactly that. The Superintendent of SVUSD commented this morning here, and based on that post, I think she concurs. Frankly, I have to agree -- STAR testing results get overwhelmed by demographic noise that obscures the signal concerning educational effectiveness, which has come up on this blog before.
Regarding the balance of the comment from the I-T, I'd note that neither of the URLs posted in the comment work, and stating "our print article included a huge chart" is a "defense" that explains many of the problems faced by the newspaper industry. Further, telling parents "you can go search here" rather than doing the analysis seems to miss the point of reporting.
Finally, I respect the paper when it notes that "[a]s for the reporter being remiss in not speculating why the scores went down, that isn't our role." I agree that the role, as described, is indeed the proper province of the paper. However, I have a hard time reconciling that statement with comments like the one the author of the article left on the LA Times' web site, a screenshot of which is at the right, where the reporter explicitly speculates about what causes test scores to move -- a comment that also suggests a level of partisanship one would think is inappropriate in a reporter.
If there is some relationship, business or otherwise, between the reporter and a private school here in Sonoma Valley, I think that relationship should, at a minimum, be disclosed in the reporter's byline. Whether the existence of such a relationship should render that individual ineligible to serve as a reporter on the subject of public school test scores is a question that is, however, above my pay grade.
Of course, there's yet another issue that's compounding the problems with the reporting in this article, which is that California's schools are in the middle of implementing Common Core. My opinion (and I'd really like to have completed that blog post by now, but I do actually have to run my law practice and coach soccer, too) is that the individuals behind the creation of Common Core have taken the legal doctrine undergirding the Free Software (Open Source) movement and have implemented it brilliantly to revolutionize American education.
As a consequence, the STAR testing regime has been akin to legacy software for several years now -- and the advent of Common Core renders it obsolete. It's probably time for application retirement. The Press Democrat's editorial board weighed in this week, calling for exactly that. The Superintendent of SVUSD commented this morning here, and based on that post, I think she concurs. Frankly, I have to agree -- STAR testing results get overwhelmed by demographic noise that obscures the signal concerning educational effectiveness, which has come up on this blog before.
 |
California Schools Guide, Los Angeles Times
Screenshot Taken June 7, 2013.
Screenshot available at http://tinyurl.com/p8ztwrd |
Finally, I respect the paper when it notes that "[a]s for the reporter being remiss in not speculating why the scores went down, that isn't our role." I agree that the role, as described, is indeed the proper province of the paper. However, I have a hard time reconciling that statement with comments like the one the author of the article left on the LA Times' web site, a screenshot of which is at the right, where the reporter explicitly speculates about what causes test scores to move -- a comment that also suggests a level of partisanship one would think is inappropriate in a reporter.
If there is some relationship, business or otherwise, between the reporter and a private school here in Sonoma Valley, I think that relationship should, at a minimum, be disclosed in the reporter's byline. Whether the existence of such a relationship should render that individual ineligible to serve as a reporter on the subject of public school test scores is a question that is, however, above my pay grade.