The Delta Issue #94

Your Test Is Driving Instruction. Do You Know Where It’s Taking You?

Hi y’all, Jessica here.

Last week, we wrote about how the job of every state education agency is to improve schools—and that requires school improvement to run through everything the state does.

For school improvement to work, you need a strong accountability system. Schools can’t improve if no one knows whether students are learning, where gaps exist, or whether changes are making a difference. Accountability systems provide that information and create the incentives that shape behavior throughout the system.

At Watershed, we think about accountability as four interconnected pieces working together: assessment, formula, reporting, and identifying schools for improvement. (You can read more about that framework here.) This week, we’re focusing on one of those pieces: assessment.

State assessments are a direct line into classrooms, and getting them right is one of the most important things a chief can do for school improvement. Like it or not, tests are incentive structures for teachers. The notion of “teaching to the test” often makes people uncomfortable, but I think we should only fear teaching to a test if it’s a bad test. 

If the summative assessment only asks students to recall basic facts, teachers will spend more time teaching basic facts. If it asks students to read complex texts, analyze arguments, solve problems, and write, teachers will spend more time building those skills instead.

So if you’re a state chief thinking seriously about school improvement, here are a few assessment dos and don’ts to consider.

DO: Actually look at your test.

It sounds obvious, but there isn’t always someone at the state level regularly looking at the actual assessment content. States should absolutely have dedicated teams responsible for assessment design, psychometrics, and quality—that’s table stakes. No chief needs to read every question on every test form, but every chief and every chief academic officer should spend time each year reviewing representative samples of the assessment.

You cannot lead assessment strategy if you’ve never actually looked at the assessment.

As a state chief, you should know what students are being asked to do. Do you know what the reading passages in your 4th grade ELA assessment look like? Are they worth reading? Does this test require students to write or just fill in bubbles? Does it reflect the standards I want taught and the rigor I expect in classrooms?

Knowing what your assessment is asking of kids is the first step. 

DON’T: Leave assessment entirely to vendors.

When chiefs inherit a testing contract from a prior administration, it’s common to assume it’s working well. But large testing companies are exactly that: large companies. Your state is one of many clients, and they have their own processes, products, and ways of doing things.

That doesn’t mean they’re bad partners. It does mean they require active oversight.

States need people whose job it is to review the work, challenge assumptions, and make sure the assessment continues to reflect the state’s academic priorities year after year. Assessment contracts are not something you buy and then walk away from.

Vendors can build tests, but they cannot decide what a state values. The state agency must set the bar for rigor and ensure the assessment continues to reflect it. 

DO: Audit your testing load—and cut the dead weight.

The concern about over-testing is legitimate, but a knee-jerk push to eliminate assessment is the wrong answer.

There are two places states should be looking for opportunities to reduce testing burden.

First, audit your own assessment. States should regularly review their blueprints and psychometric data to determine whether the assessment is longer than it needs to be. Are there sections providing redundant information? Are there places where fewer items would still produce reliable results? If the data support a shorter form, make the change. If they don’t, resist the urge to cut for the sake of cutting.

Second, help districts audit their own assessment systems. When people complain about “too much testing,” they’re often blaming the state test, but the lion’s share of the testing burden comes from district-level assessments layered on top of the state assessment.

The last landmark study on testing load was conducted in 2015—more than 10 years ago, which is itself a problem—and found that students in big-city districts were taking over 100 mandatory standardized tests between pre-K and 12th grade, spending an average of 20 to 25 hours a year just on required exams. That doesn’t include test prep, classroom assessments, or the full stack of district-administered interim assessment, meaning the real number is certainly much higher. 

States can help districts eliminate bloat by providing a framework for evaluating their testing load. Help districts identify the purpose of every assessment they administer and eliminate those that are duplicative or not being used. If an assessment isn’t giving teachers or parents actionable information and isn’t tied to the district’s academic vision, it should go.

DON’T: Solve the testing problem by lowering quality.

Some states—including my home state of Louisiana—are moving toward models where a fifth grader takes a single exam covering third-, fourth-, and fifth-grade standards all at once.

It’s not a bad instinct to want testing to consume less instructional time. The problem is that grade-span testing like this actually creates terrible instructional incentives.

For example, if science is only tested once across three grades, schools have less reason to prioritize science instruction every year. Then, when the testing year arrives, schools scramble to make up for lost time, relying on cramming and drill-and-kill test prep rather than sustained, high-quality science instruction.

This is no fault of the teacher. It’s the predictable outcome of an assessment strategy that isn’t clear-eyed about the signals it’s sending into classrooms.

It’s not just about having fewer tests—it’s about having a few high-quality tests that incentivize the right things. Before you take a hatchet to your assessment calendar, think instead about how to build assessments that reinforce the academic vision you want in classrooms.

DO: Be honest about student performance.

Assessment is one of the only ways states measure whether student outcomes are improving. But you cannot fix what you cannot see. A state that lowers its proficiency standards to manufacture good news is not just misleading parents, it’s flying blind on its own improvement strategy.

Right now, nearly nine in ten parents believe their child is performing at or above grade level, while actual proficiency rates among eighth graders are just 30 percent in reading and 28 percent in math. 

This is what’s called an “honesty gap,” a term coined by the Collaborative for Student Success to describe the disconnect between what states report as proficient and what an independent measure like NAEP shows. States have enormous discretion over where they set their proficiency bar, and over the years many have set it low so that more kids show up as proficient on paper. Wisconsin, for example, redesigned its assessment and English proficiency jumped from 39 to 48 percent overnight, without a single thing changing in a classroom. Kansas did something similar not long ago.

As a state chief, it is your job to tell the truth about how kids are doing, even when the results are politically uncomfortable. Set your proficiency cuts honestly, check them against NAEP, and hold students to a high bar because that is how they reach it. 

DON’T: Let accountability points drive assessment choices.

Assessment and accountability systems create incentives. Teachers, schools, and districts are rational actors that respond to those incentives. 

The question is whether the incentives you’re creating align with the learning goals you actually care about. If you award accountability points for weak assessments, schools will spend valuable time chasing metrics that don’t meaningfully improve student outcomes.

The worst offender of this in my book is WorkKeys. ACT markets WorkKeys as a workforce-readiness assessment tied to career pathways. But as my colleague, Jill Zimmerman Pinsky recently wrote, WorkKeys is a bad exam. No student who scores well on it is better prepared for college or a career than they would have been without it.

Nevertheless, the test persists because, in some accountability systems, it generates points. In Louisiana, ACT performance accounted for 25 percent of a school’s accountability score. WorkKeys offered districts an alternative path to earning some of those points, so schools pushed students to take it, not because it was the best measure of readiness, but because it helped move an accountability metric.

The result was absurd. We had students scoring in the 30s on the ACT sitting for what was essentially a remedial applied-skills assessment. That’s a poor use of time, money, and effort. More importantly, it wasn’t helping students get closer to college or career success.

If you want a strong assessment system, start by asking: Does this assessment measure something that matters, and will it encourage the kind of teaching and learning we want to see? If the answer is no, it doesn’t belong in the system.

Let’s Get Muddy

Want to go deeper? Here are a few pieces worth reading:

• Previous Delta: Measurement systems reveal what you value. What does yours say?
• The Case for Expanding Assessment by The Thomas B. Fordham Institute — Why more high-quality assessment data, not less, can help states better understand whether students are learning and where support is needed.
• The Honesty Gap by The New York Times — A look at the growing disconnect between what state tests say about student performance and what NAEP shows.

NAEP Expansion by The 74 Media — Why expanding NAEP could give states a clearer picture of student performance and strengthen accountability across the country.Lowercase “school improvement” is the daily work of making schools better at the core thing they exist to do: educate kids.

Capital-S, Capital-I “School Improvement” is a federal grant program created under the No Child Left Behind Act (NCLB) that has since evolved into a tangled bureaucratic web: a School Improvement office inside an agency, a cottage industry of vendors, and a portfolio of SIPs (School Improvement Plans). It runs parallel to other offices also tasked with improving schools—high school, math and literacy, and attendance—but there’s little coordination between them.

Improving schools should not sit in one office or initiative. It should be the work of every office, and the organizing principle behind how state agencies operate.

Federal School Improvement dollars are one of the clearest resources states have to drive instructional improvement in the schools struggling the most. But most states today aren’t using them that way.

This week, I want to make the case for states to stop treating school improvement like a side hustle and instead treat it like their primary mission.

If school improvement isn’t the purpose of the state agency, what is? 

During our recent webinar on “The Myth of Local Control”, we debunked the idea that states are powerless to influence what happens in classrooms because local control makes it too complicated.

As Jessica Baghian shared: Local control is real, but it’s not an excuse for states to forfeit their point of view on what strong teaching and learning looks like. 

The public expects state education agencies to lead systems, improve outcomes, and make sure kids are learning. If the job were only to transfer money from the feds to districts, five people could do it. The reason state agencies have buildings full of staff is because the work is supposed to be bigger than that.

So how did so many states end up so far from that work?

Under No Child Left Behind (NCLB), federal law treated school improvement as its own line item: states had to identify struggling schools, place them into one of a few federally prescribed turnaround models, and use dedicated School Improvement Grant funds to support those efforts. When ESSA replaced NCLB in 2015, the prescribed models went away and states gained flexibility to design their own improvement strategies.

Unfortunately, most states didn’t use that flexibility to integrate school improvement into core academics. They used it to build more around it—needs assessments, improvement plans, monitoring systems, reporting structures, compliance processes—that consumed enormous amounts of time and energy without changing what students were experiencing in classrooms.

What States Get Wrong About School Improvement 

The problem with School Improvement stems from four common mistakes:

1️⃣ States make improvement harder, not easier, for struggling schools.

Improving schools is conceptually straightforward: students need stronger instruction, teachers need stronger support, and schools need coherence around what they’re trying to accomplish.

But states have a habit of making improvement more complicated than it needs to be. If a state believes high-quality curriculum, strong teacher training, and extra support for students who struggle, leads to better outcomes, then school improvement should mean doubling down on those things, not layering on a separate theory of change for struggling schools.

2️⃣ States don’t identify enough schools.

Under ESSA, states were required to identify the bottom 5% of schools “in need of improvement” and choose from one of four federally prescribed turnaround models to get them back on track. 

But identifying only 5% of schools is not nearly enough when, in many states, far more schools are struggling to deliver strong outcomes for kids, sometimes even a majority. In Louisiana, we took improvement a step further by identifying roughly the bottom 40% of schools. Nearly every district ended up with at least one school on the list, which meant school improvement became a statewide instructional priority and landed on every superintendent’s radar.

3️⃣ States skip districts on the implementation chain.

When a school is identified for improvement, the burden usually falls directly on the school: write the needs assessment, develop the improvement plan, select the interventions, monitor implementation, and report progress. Meanwhile, the district—the part of the system that controls many of the core instructional levers—is left out of the equation entirely.

For school improvement to reach kids in classrooms, the chain has to run state → district → school, with every link doing its part. Schools have to deliver strong instruction every day in every classroom. Districts have to take responsibility for improving their schools. And states have to hold districts accountable for results.

4️⃣ States fund compliance, not improvement.

Federal School Improvement money is specifically intended to help the lowest-performing schools get better. Most states are using the money in exactly the wrong ways.

On one side, states layer extremely cumbersome compliance requirements onto identified schools that do little to move kids toward better outcomes, and that sometimes make real improvement even harder. The dollars effectively pay for the paperwork, not the work.

On the other side, the money itself comes with no strings attached. States are simply distributing School Improvement dollars without requiring anything in return—no expectation that schools adopt high-quality curriculum, invest in teacher training, or build the implementation systems most likely to move the needle for students. The result is that funding flows regardless of the choices schools make, which means the schools doing the right things and the schools doing the wrong things get treated exactly the same.

Funding should be a lever, not a blank check—and states should reserve their School Improvement dollars for schools and districts willing to make the right instructional choices.

Let’s Get Muddy

Over the coming weeks, we’ll explore what it looks like when school improvement runs through everything: how states assess schools, how districts allocate support, how leaders are trained, how success gets recognized and replicated, and how resources are prioritized. Stay tuned.

ICYMI: Watch our webinar on “The Myth of Local Control,” featuring EdTrust CEO Denise Forte , and The Thomas B. Fordham Institute President Mike Petrilli , and our very own Jessica Baghian.