Submitted by Phil on Mon, 10/16/2017 – 05:45
“Applying simple solutions to complex problems creates more complex problems.” -Ronald Heifetz
Recently Indiana released the A-F grades for its schools. Each year there is a chance any given school’s ISTEP+ scores and A-F grades may go up or down. This is because ISTEP+ fits statistical models and “regression to the mean” is a real thing.
The term “mean” means “average”. Here is an example of regression to the mean: compare a .300 hitter in softball with a .200 hitter. On any given day the .200 hitter could get 4 hits in 4 at bats while the .300 hitter could go 0 for 4. Over the season, the .300 hitter’s daily performance will bounce around an average (mean) of .300 so she is likely to hit well soon. Pitchers would prefer to pitch to the .200 hitter the next day, because regression to the mean says she will soon be going 0 for 4. The .200 hitter’s day of going 4 for 4 would be called an “outlier”. The issue is, in the moment, we don’t know which hitter is which hitter until the end of the season when we have plenty of data (at-bats) to look at.
The ISTEP+ is one at-bat.
It is no secret the ISTEP+ and Indiana’s A-F accountability system basically sorts Indiana’s schools by socio-economics, and every year a few schools perform outside the expected norm. Those schools are either celebrated or vilified and then within a year or two, as new data sets move through the schools, they “regress to the mean” and are forgotten as new outliers take their place. This is basic statistics, but its current application in our political discourse belittles and insults the incredibly hard work Indiana’s students, teachers and parents do every day. It also seems to be a lot of taxpayer money and educational time to spend on something that could be easily determined by simply listing school districts by median home prices or median incomes. Is this a system we want to pay for?
I want to be clear. Schools need to be responsive to their communities and meet their expectations. I also believe in the importance of assessing students. It is absolutely critical that we continuously provide students feedback on their performance so they can improve. Indiana’s upcoming shift to a computer adaptive test (computer based tests that get easier or harder for the student depending on how students responded to previous questions) will be better for our girls and boys because it should allow us to provide each child with specific information on their own performance promptly. This is great for children, their parents and their teachers. More timely, targeted information helps us all.
What I am opposed to is an incredibly expensive and ineffective system, built upon a misuse of tests and poor understanding of statistics, which has such high-stakes attached to it. There is no evidence spending millions of dollars on a test of children, tying a teachers’ livelihood to that test, labeling a community with that test, and clothing that test in the garb of “improvement through competition” and “accountability” improves student performance. In fact there is evidence to the contrary: none of the highest performing countries in the world do this.
Also, there is no evidence Indiana’s test is measuring the correct data to determine what is a “successful school”. Indiana’s current definition of success defined by our A-F system does not correlate to any definition of success in life. All “successful students” do not become titans of industry nor Nobel Prize winners. Not all titans of industry or Nobel Prize winners were “successful students”. The ISTEP+ and our A-F system certainly don’t measure what our local business leaders, teachers, or parents list as important. I was a C+ student as was my father before me. My father turned out ok. I hope I do too; so far so good, but it is early…I hope.
There are no objective decisions. The very act of setting “objective parameters” for a decision is subjective. It is based on choices. For almost twenty years, Indiana has chosen to spend millions of dollars and waste countless hours to run a system used to rank communities by socioeconomics, bash teachers, stress out students and their parents, narrow what is taught, and justify political talking points. The A-F system fails because it takes incredibly complex issues and attempts to simplify them on the backs of our children and their teachers and parents. The A-F system is an expensive waste of time that hampers economic development and harms the very communities it purports to help. Indiana, we have chosen this.
Why has Indiana made these choices? I believe sometimes political pressures force decisions to be made without taking into account unintended outcomes. We all want Indiana to be successful. We understand the need for all children to be successful. Different families have different definitions of success for each of their children and should have a say in meeting that definition. Different communities have different definitions of success. Every community has strengths. Mostly, I believe our legislature’s role is not to solve a community’s problems, rather it should be to create conditions where communities can solve their own problems. The A-F system does none of this, but it was an easy decision that sounded good on the surface.
Indiana is a small midwestern state and many of our communities are shrinking. Our problems are complex and diverse. There is no one, easy solution to those problems. Every community and every family has strengths. We need a system that begins with those strengths. We need leaders who listen to learn and ask to empower, not dictate, over-react, point blame, or cling to past poor or outdated decisions. The A-F system meets none of this criteria.
The promise of computer adaptive tests lies in its ability to individualize to each student…to learn their strengths and help teachers and parents form relationships with students based on those strengths in order to motivate. We need our legislators and those sitting on the various educational and business state-level commissions to find ways to individualize for communities. If we recognize the complexity of the state, provide individualized data to communities, respectfully listen, and help develop systems for communities to leverage their strengths, we will produce the results we need. This is hard work and would be time well spent…like parenting, and teaching. The A-F system meets none of this criteria.
Schools are mirrors of their communities. If the vision in the mirror is bleeding, putting a bandaid on the mirror doesn’t stop the bleeding. Indiana needs to look at its data and, rather than choose to jump to conclusions or force unwieldy solutions, do what all good analysts do: use the data to ask better questions and make better choices. Continuing with the current A-F system is a bad choice and it is time to get rid of it.