Last week, I took the Star
to task for its article about Pima County districts' AzMERIT scores in the post, To Understand Pima County Test Scores, Follow the [Parents'] Money
. (The post had more likes, shares and comments than most of my recent pieces, and a number of letters in the Star
voiced similar criticisms, meaning the Star
article bothered a lot of people.) Comparing Tucson Unified's test scores with districts whose students come from more affluent homes where parents have more formal education makes little sense, I wrote. In Arizona, around the country and around the world, children from higher income families score higher on standardized tests than children from lower income families regardless of the quality of schools they attend.
I've written often that if you want to create a reasonable analysis of Tucson Unified's AzMERIT scores, you have to compare them to scores in districts with similar demographics. Well, I've decided to put my keyboard where my mouth is. I'm beginning a rough study to see how test scores in Tucson Unified schools compare with scores of similar schools in similar districts. Why am I telling you what I'm planning to do even though I've only just finished the thinking process and haven't begun the research? To keep myself honest, for one. If I put my approach in writing, I'll be forced to stick to it and report the results as honestly as possible (which I'd try to do anyway, but it's always tempting to fudge a bit). And to let readers know what my approach is before I write about my findings so you're less likely to think I began with my conclusions and worked back to the data that "proved" what I already decided.
On my computer, I have two databases from the AZ Department of Education. One lists the total number of students in every district school in the state along with the percentage of students on free or reduced lunch. The other breaks down the 2017 AzMERIT scores of every school in detail, by gender, ethnicity, English Language proficiency and grade level. Looking at the two data sets, I can compare how schools with similar student bodies scored on the state tests, and I can compare the scores of subgroups in the schools.
Here's my methodology. Scratch that. "Methodology" is to high fallutin' a term for my crude analysis—I won't be using any sophisticated statistical tools—of a blunt instrument—a high stakes test whose validity as a measure of student achievement is questionable. So, here's what I'm gonna do.
I'm looking at districts which are similar to Tucson Unified in demographics, which means districts with a large percentage of low income and Hispanic students. I'm focusing on Southern Arizona districts, because I think it's probable Hispanic students in districts farther north have been in the U.S. longer, which means they have a higher level of English proficiency than districts closer to the Mexico border and could have higher scores for that reason. Also, I'm focusing on larger districts, those with four or more elementary schools. That means along with Tucson Unified, I'll be looking at schools in Sunnyside, Yuma, Nogales and Douglas — also Flowing Wells, though it has a larger white population than the other districts.
To make sure I'm comparing like with like as much as possible, I'll only look at elementary schools, and I'll group schools which have similar enrollment numbers and similar percentages of students on free/reduced lunch. I'll create the groupings before I look at the test scores. The next step will be to compare the scores of schools in each of the groupings.
I honestly don't know what I'm going to find. I expect the test scores of similar schools will be reasonably similar, though I know there will be variation. A few percentage points difference one way or the other isn't very relevant, given my crude analysis of a blunt instrument, but if some schools are significantly above or below others, that could mean something. I also expect that most or all of the scores will be significantly lower than in schools in high income areas. I have no idea how Tucson Unified schools will fare in the comparison.
I'm not setting a timeline for myself, so don't hold your breath waiting for the results. They'll happen when they happen.