The following was derived from an interview and converted into this FAQ.
1. What is Accountability:
The systems and processes used for building trust with your primary stakeholders.
2. Who are the primary stakeholders?
The members of the group without whom your organization would not exist. For a school that is our students, their parents, and their communities.
3. What is Benefits-Based Accountability™ based on?
Accountability in effective organizations works from the benefits its stakeholders expect as a result of having a relationship with it. Benefits are that almost magical point that lets a non-technical stakeholder articulate what they need—improved health, readiness to be a successful adult, my plumbing fixed—that then lets the technical experts in the organization see how best need to apply their expertise to provide the benefit. At the point of benefit both sides can talk about the same thing and understand each other, even though they come to it with very different levels of technical expertise.
4. I hear you talk a lot about trust. Why is that?
Organizational accountability is properly referred to as a trust-building machine. That can work well for an organization or it can work poorly, and that’s always a choice. Because organizational accountability can never be turned off, you’re either building trust or hurting it. Those are the only choices.
5. You say organizational accountability can’t be turned off, but the states did turn school accountability off last year.
Which tells us very clearly that what states have imposed as school accountability isn’t an organizational accountability but something else. There are other accountabilities that can be turned off in a crisis, but a crisis is the very moment an organizational accountability is most needed.
6. Who does Benefits-Based Accountability™?
Virtually every effective organization out there, including hospitals, churches, synagogues, businesses, non-profits, and schools. They don’t often use the same words and they use a huge array of methods and tools, but they are constantly working to help their stakeholders understand their capacity to deliver a benefit into the future.
7. How long has the effort to get Benefits-Based Accountability™ into schools been around?
The initial idea started more than a decade ago and the more practical tools and solutions were in place roughly five years. This is still very much a new movement and we’re all learning from each other at every step. We’ve kept it at the level of grassroots efforts and to this point haven’t really pushed it. That’s changing this year—we’ve learned enough that it’s time to go full steam ahead. And remember—the frameworks have been around in lots of different organizations for many, many years longer than that. We’re not really inventing much, just learning how it applies in schools.
8. Where is this being done?
Two state consortia have emerged as the leaders in this work, first in Texas and second in Georgia. Their efforts really helped fine tune the tools and processes. Rowan-Salisbury in North Carolina has also really stepped up on this as well. When the pandemic hit, we were working on getting started with consortia in Mississippi, Pennsylvania, Ohio, and Kentucky, and we’re now working with several national organizations to extend this to a national movement.
9. Is it expensive?
No. Compliance-based systems, which have long been in place in schools are very expensive and we’ve become accustomed to the cost and burden of them, but Benefits-Based Accountability™ isn’t expensive for a whole host of reasons. The biggest reason is that compliance-based accountabilities offer real advantages if you can get away with non-compliance since you’ll get the benefit without the effort, and so the metrics they use must be constantly verified. Just getting those metrics is expensive and verifying them ever more so. But Benefits-Based Accountability™ incents the truth, with huge disadvantages anytime you veer from it. It turns out that an evidence-based truth is simply a lot less expensive to get out there than repeatedly trying to confirm that you’ve complied with something.
10. You talk about the students being the primary stakeholders, but aren’t the state and the legislature stakeholders as well? After all, they fund a lot of what happens in schools.
Of course, but they aren’t the primary stakeholder, and that distinction is huge. Students are the reason schools exist, not the state. If students didn’t exist, we would still have the state, and if the state didn’t exist or fund education that wouldn’t do away with the need. It is true that we have the schools we do because states agreed that education is important and it’s their responsibility to support it, but that importance exists independent of them.
11. What about objectivity? Don’t we need objective metrics for accountability? How else will this be fair?
I understand the question and the reason it gets asked as often as it does, but anytime you place organizational accountability in the technical weeds of an organization—and metrics and measures are always technical—the accountability system will almost always fail in helping the organization achieve what was intended. That flies in the face of everything that’s been done in education for more than twenty years and seems counterintuitive, but it’s still true.
Here’s why. Metrics are always technical, and when they are properly used, they can be useful within the organization. They can also be useful alongside other evidence to determine whether students are benefitting from their education. But metrics regularly lend themselves to two interpretations: the correct, technical interpretation, and an incorrect, non-technical interpretation. The problem with the non-technical interpretation is that to the naked eye it often feels like the one based on common sense and so it’s all too easy for policy makers to use those interpretations in their work.
Watch how easy that mistake is to make. Take graduation rates. The non-technical interpretation is that the school with a higher graduation rate does a better job graduating students. That feels so commonsensical: the school with the higher rate wins.
But the technical interpretation of a graduation rate isn’t simple. Just because one school graduates a higher percentage of students than another does not automatically mean it’s better at graduating students. It may be, but you must look before you know. Otherwise you’re making a pretty big statement without any evidence that its true. If all the students in the school with the higher rate would graduate regardless of where they went to school, that school isn’t adding much value in that regard—it would happen even if the school and those educators in it didn’t exist. That isn’t saying it’s a bad school, just that the metric isn’t very meaningful for them.
But the opposite may be true for the school with the lower graduation rate. It could be that many of the students who graduate from that school wouldn’t graduate if that school and those teachers didn’t exist, but because they do and did a great job adding value in that regard the students benefitted.
The non-technical interpretation of graduation rates in this instance is just plain wrong. The school with the lower graduation rate is better at graduating students, and if your child is unlikely to graduate and really needs some help in that regard, the choice is clear: it’s the school with the lower graduation rate, where they would be more likely to graduate than in a school that wasn’t used to doing that sort of work.
If the goal is to identify the more effective school at graduating kids, the non-technical interpretation fails to do that. And if the goal is to benefit students who are likely to struggle, the non-technical interpretation isn’t going to do that either. The further into the weeds you go with an accountability, and the more the metrics lend themselves to technical verses non-technical interpretations, the worse it gets. That’s why we stay at the level of benefit. It keeps us grounded in a way that doesn’t create that sort of problem.
The irony in this notion of objectivity is that the more objective we try to be in the name of fairness, the less likely we are to get students what they need. Simple is always good, but you must be careful with simple, because it can lead you in the opposite direction you need to go in the name of trying to do the right thing.
12. So how do we compare schools in the Benefits-Based Accountability™ world? If we can’t do that it’s a free-for-all.
The same way you compare all complex things and organizations, whether that’s hospitals, doctors, grocery stores, political candidates (especially in a primary): carefully and thoughtfully. In none of those instances would you reduce them to a single number and compare the number, which is meaningless. Instead, you look at all the benefits you expect from a grocery store or a doctor and base the comparisons on the profile that emerges.
Schools should be no different. We must look at the profile of a school to understand it, not a number. My experience in schools suggests that there are about thirty benefits schools provide students, and at any one point in time they are actively working to improve eight to ten of those. Knowing which benefits are the focus of those changes and their effectiveness at getting to where they said they need to be is a profile that will let me see clearly the effectiveness of that school and those in it. Trying to average those or reduce their work to a number prevents that from happening.
There can be no doubt that some schools will be more or less effective than others, and we should learn from the effective ones and work hard to improve those that are less effective. But the era of crudely ranking schools and imagining you can find meaning in that must end. Ending that practice doesn’t create a free-for-all if you have a Benefits-Based Accountability™ system in place. The fact is that we have now is a free-for-all that can’t identify school effectiveness and we need to treat it that way and move to something better.
13. So how will the public know that their tax dollars are being well spent in the Benefits-Based Accountability™ world?
That question presumes that they know that now when I hope I just showed that isn’t the case. What they have now are lots of non-technical interpretations of technical information that are misleading at best, even though many of those misunderstandings really are well-intentioned (but certainly not all). The truth is that taxpayers need the information in the question but don’t currently have it. That is a huge and even embarrassing problem given how much we spend on the current system.
Accountability systems based on stakeholder benefit provide this sort of information better than any other kind of system I know of. Imagine the organization of a family for a moment. If not going hungry is a benefit of being a child in a family and we go from 30% of children going to bed hungry down to 15% going to bed hungry, we will consider whatever we did to have had a positive effect. But if the numbers don’t move or went in the opposite direction it will require a rethinking.
If we try a more metrics-based approach, we would have to find a metric, say, the number of meals served each week, and draw a line that represented the difference between being hungry and not being hungry. Eating regularly is a good thing so that metric will appear to make sense. But consider how many families will be misclassified as hungry or not hungry if that metric serves as the formal accountability tool. Some families may meet the number of meals and be above the threshold, but they’re scant meals and the children still go to bed hungry several days a week. That would be a false positive. Or some families may fall below the threshold by serving only two substantial and satisfying meals most days. That would be a false negative.
Those who try and use metrics for accountability get this. As a result, they tend to make lots of adjustments in the name of trying to make the system more accurate which in turn gets really convoluted. Consider a calorie-based accountability for hunger, which would appear even more scientific and objective than the number of meals. Take the calories consumed at each meal, add them up and divide by the number of people in the household. Those who fall below a number per person are presumed to be hungry, while those above a number are not. Simple, right? Only it isn’t. You would then need to account for the fact that children require a different caloric intake than adults. You would need to check if the household had any athletes, who would consume a disproportionate number of calories given their needs and skew the view of that family, or see if anyone was overweight and on a diet, in which case a reduced calorie intake would need to be seen as a positive, not a negative.
And on and on it would go, with the hope of creating of some uber metric or index that could summarize it all. Only when that happens, after all the machinations and manipulations of the date, we would wind up in the dark as to what is really going on with childhood hunger, because we will have strayed so far away from being accountable for the benefit and moved accountability into the technical weeds of a metric.
Our current educational accountability is set up to identify the percentage of kids who pass a test and if that goes up people want to declare the money well spent. It’s logical to wonder why the percentage of children going to bed hungry and the percentage of kids passing a test aren’t the same thing, because they sure feel similar.
Here’s the difference, and its huge: child hunger is at the level of benefit and test scores are in the technical weeds of a school. Just think about what we do with test scores in our metrics-based accountability. We try and account for students who have had fewer opportunities and treat them and their schools differently. We try and account for how many days the student was in that school, and whether they qualified for services such as special education or instruction in their native language. And then someone mushes it altogether into an index and draws a line in the sand and suggests one step above you’re an effective school, and one step below you’re ineffective. That is so far removed from the benefit of learning academics that it’s impossible to know what it means, for any stakeholder or taxpayer.
Here’s the other thing: taxpayers always need to know that their dollars are being well-spent, and especially in a crisis, and yet metrics-based systems can get turned off in a crisis and frequently are. They’re expensive and burdensome and its right to give people a break from them when priorities need to be elsewhere. But you can never turn off a benefit. When do you stop being accountable for making sure children have the nutrition they need, or for seeing that learning never stops? You don’t. A Benefits-Based Accountability™ system is the best way to make sure taxpayers get what they need because it can speak directly to them at the level of benefit, even during a crisis, which will make it very clear, very quickly, if they should be satisfied or demand a change.
14. What about the timing? Schools have a lot on their plates just now.
That is an understatement. But here’s the issue I want schools and their leaders to consider. Throughout this next year educators will have unprecedented opportunities to build trust with their stakeholders. Since the pandemic hit the moral and intellectual leadership shown by our colleagues is remarkable, and yet just the other day I heard someone express concern that without testing to hold educator’s feet to the fire they’re worried schools will backslide.
School leaders need to speak with the person who made that statement, and everyone else who thinks the same, and share with them in a truthful, evidence-based manner the facts behind their efforts. Some of that will include areas of effectiveness, while some of it will include areas of ineffectiveness, but so long as it’s the truth that notion of trust will start to emerge.
School leaders will be dealing with these and many other voices with or without Benefits-Based Accountability™. What Benefits-Based Accountability™ will provide are the frameworks for having those dialogues such that the truth will be known, and trust will be built. Those dialogues are going to happen. That we do not control. That energy will be spent regardless. So why not use that energy to build a larger trust with public education? Benefits-Based Accountability™ helps organizations of all types do just that when they are in this sort of moment, and now it’s time to do it in schools.
So if not now, when?
