Tips for Data-Informed Decision-Making: Don't Forget the Tape

It’s every gift wrapper’s nightmare: you assemble your small army of presents, gather the rolls of wrapping paper, slice off a colorful sheet, begin folding your origami masterpiece—then you freeze, mid-fold, when you remember you forgot the tape.

Strangely enough, this familiar scene captures a dilemma we see in many school districts we work with: More and more education leaders are making bold moves to put data at the center of their decision-making (something we call “data-informed,” not “data-driven,” since we believe that judgment and experience go alongside data). But the data infrastructure they rely on has crucial gaps that make it hard to actually use the data in productive and nimble ways. They’ve got the presents, the paper, the scissors—but no tape.

For example, we work with large urban school systems and states on their human capital systems. Many of our partners want to try strategies like paying raises based on effectiveness, creating leadership roles for highly qualified teachers, and providing incentives for teachers to work in hard-to-staff schools and subjects. They would like to understand the distribution of novice or highly effective teachers across schools and even across grades, or how potential changes to the benefits package will affect their compensation budget. But district leaders might discover some of the following “data infrastructure gaps”:

• Data systems are not linked: The district houses its teacher evaluation data with an external vendor, but the vendor’s system might not use the same teacher ID number as the district’s other data systems. This means that the district can’t connect teacher effectiveness data to payroll data, which means it can't determine how much it would cost to give all effective teachers a pay raise. It makes it very difficult to calculate or implement many innovative strategies—simply because the two data systems don’t match.
• Data timelines are not synced to school timelines: The district might not collect teacher evaluation data until the fall of the next school year. That means it must wait until the school year starts to know who is eligible for a leadership role or an effectiveness-based raise based on last year’s performance. This makes it hard to calculate the cost of such strategies until it’s almost too late—and just as importantly, the district won’t know who to tap for leadership roles until the year has already begun.
• Data structures don’t allow for “nimble” calculations: The payroll, HR, and school data systems often have structural limitations that make it challenging to design and project the cost of compensation scenarios. For example, payroll and HR data might not show benefits broken out per employee. This makes it difficult to calculate the full cost of compensation when the workforce changes in key ways—for example, when x% of experienced teachers leave and y% of new ones join.

These “data infrastructure gaps” are not deliberately placed barriers but often hidden obstacles that district leaders do not realize are there until they start to lean on their data infrastructure in new ways. We recommend that forward-thinking districts conduct a “data inventory” to understand the linkage, timeline, and structural gaps that could keep them from performing robust modeling and decision-making. Some areas to consider include:

• Data collection, storage, and reporting processes
• Key data sources such as payroll, human resources, teacher evaluation, school data systems, course schedule data, student assessment data, and student demographic data

By performing a “data inventory” now, district leaders will be well positioned to take on bold new, data-informed processes in the future. It's getting all the wrapping supplies together, before you start to fold.

For a greater discussion of district data sources—and how the state can support districts in using this data well—check out Spinning Straw into Gold: How state education agencies can transform their data to improve critical school resource decisions.