Tag: assessments

Is It The Tests or the Stakes?

Posted on | by Sam

As DC grapples with whether or not to adopt an accountability framework that would assign between 60-80% of a charter preschool’s overall rank to its students’ reading and math scores, it’s worth asking: What’s at the root of the problem here — the tests, or the stakes attached to those tests?

When it comes to the debate over the Early Childhood Framework (which you can access here — just click “Public Hearing” in the left column and then click “Early Childhood PMF”), I would suggest it is overwhelmingly the latter.

Here’s the thing: just about every early childhood program worth its salt is already assessing children in a variety of ways, including, and not limited to, reading and math. Usually, these assessments are administered in a 1-on-1 format, and if a teacher does his or her job skillfully, the child won’t even register that it’s a “test.” So the Public Charter School Board is correct when it says that testing children in these subjects is not new.

What their explanation glosses over is the impact children, educators and parents can expect once these assessments have high-stakes attached to them. Rankings matter to all schools, but they are particularly important to charter schools, which are always in the midst of raising money, searching for a permanent facility, and recruiting families. Here in DC, “Tier 1” status is a gateway to better fundraising, preferential treatment when it comes to facilities, and longer waiting lists.

The problem with the PCSB’s proposal, therefore, has less to do with requiring schools to choose from a common pool of math and reading assessments (indeed, the list boasts more than thirty options to choose from, many of which were submitted by the charters themselves), and more to do with attaching a disproportionate weight to reading and math. This immediately transforms those assessments from diagnostic to accountability tools. It is guaranteed to modify the behavior and strategic planning of the schools. And it perpetuates the shell game of American public education, in which we use partial information to pronounce complete judgment on whether a given reform effort is working or not. (And, more importantly, in which we allow ourselves to assign credit or blame in ways that correspond to a political timetable.)

In reality, as both common sense and research would tell you, it’s not that simple. “True standards of intellect do not lend themselves wholly to quickly collected, precise, standard measurement,” wrote Ted Sizer in his classic book, Horace’s Compromise. “We need to devise clusters of instruments to probe our students’ ability to think resourcefully about important things. Indeed, we need time to reflect deeply on what we mean by ‘think resourcefully’ and what we feel are the most ‘important things.’”

In his 2009 book Measuring Up: What Educational Testing Really Tells Us, Harvard professor Daniel Koretz makes a similar point. “Careful testing can give us tremendously valuable information about student achievement that we would otherwise lack,” he says. The question is how well we understand what standardized tests can, and cannot, tell us about American schools. “There is no optimal design,” he asserts. “Rather, designing a testing program is an exercise in trade-offs and compromise – and a judgment about which compromise is best.

Systems that overvalue one set of test scores fail to heed this advice, but they also fail to highlight a sufficiently clear picture of what is actually happening in a school. “Certainly, low scores are a sign that something is amiss,” Koretz says. “But the low scores themselves don’t tell why achievement is low and are usually insufficient to tell us where instruction is good or bad, just as a fever by itself is insufficient to reveal what illness a child has. Disappointing scores can mask good instruction, and high scores can hide problems that need to be addressed.”

So let’s vigorously debate ideas like the PCSB framework, and when we do so, let’s follow the lead of experts like Lydia Carlis, the chief of Research and Innovation for the AppleTree network of preschools. “The ongoing conversation will be so useful to all children if we focus on both/and rather than either/or when discussing teaching and (developmentally appropriate) assessments of academics and SEL in early childhood, ” she says. “The early childhood interventions that have been evaluated longitudinally and shown effective demonstrate that low-income children at risk for school failure need early access to high quality, evidence based academic and social supports. Social emotional skills are both necessary and insufficient for closing the academic achievement gap. Children who begin demonstrating cognitive gaps as early as nine months need skill and concept development on academic content and strategic support for social emotional learning. And clearly, all children will benefit from a high quality, balanced instructional program.

Works for me. PCSB, are you listening?

 

The Science of School Renewal

Posted on | by Sam

(NOTE: This article originally appeared in the Huffington Post.)

There’s a revolution underway in the scientific community, and it’s changing the way we understand both the structure and the inner workings of the universe. These insights have far-reaching implications for all of us – and none of them are being heeded by the leading voices of our current efforts of transform America’s antediluvian public education system.

This is a serious problem.  Here are three examples of what I mean:

1. The Relativity of Learning – Almost everyone is familiar with Albert Einstein’s game-changing theory of relativity – an insight that, overnight, overturned an idea that had governed human thought for more than 200 years. Fewer among us can explain the theory in any depth, but we know this much: Einstein demonstrated that time itself is not, as had been assumed by Isaac Newton and others, a fixed construct that is experienced uniformly, but rather a malleable construct that is experienced relative to something and/or someone else. This seismic development in human thought moved us away from the Newtonian notion of absolutes, and toward a deeper understanding of just how fully we experience the world in particular ways.

The lesson to be learned from this seems clear enough: we should be wary of absolutist thinking in our own lives (and, certainly, in our organizations). Yet contrast this insight with the K-12 education landscape, which is still working in absolutes, and still basing its biggest decisions on a single, standardized measure of success: basic-skills reading and math scores. This doesn’t mean our interest in these subjects is unimportant – literacy and numeracy matter greatly – but it does mean we’ve failed to learn something essential about the nature of things. Otherwise, we’d be asking a different question when it comes to school accountability: If learning, like time, is relative, how can we develop less standardized (and more customized) assessments that will help us know if we’re being successful at helping children learn to use their minds well?

2. The Quantum Mechanics of Motivation – As you may know, although our general rules for understanding the workings of the universe on a macro scale – a.k.a. classical physics – work quite predictably and neatly, those same rules mean absolutely nothing at the messier micro level – a.k.a. quantum mechanics. What quantum mechanics reveal is that relationships are the key determiner of everything. Subatomic particles cannot exist without the presence of another, and the more we try to observe and codify their nonlinear behavior into a series of linear “if/then” statements, the less relevant our insights become. It’s just too complicated – even for quantum scientists.

Similarly, we humans are nonlinear beings, and the relationships we form (or don’t form) are the key determinants of everything in our personal and professional lives. Yet contrast this insight with the K-12 education landscape, in which both elected officials and philanthropic leaders are pursuing if/then incentive programs based on the belief that pay for performance will be the missing tonic our educators need. It’s the difference between a Newtonian view of the world – which views things in straightforward terms of cause and effect – and a Quantum Mechanics view of the world – which recognizes the inherent unpredictability of the entities it is observing.

The good news is we don’t need to be so abstract. Check out these insights from three different studies of human behavior and the human responses to programs, like performance pay, that are based on extrinsic rewards:

  • “When money is used as an external reward for some activity, the subjects lose interest for the activity.” (Deci 1971)
  • “Intrinsic motivation is conducive to creativity; controlling extrinsic motivation is detrimental to creativity.” (Amabile 1996)
  • “People use rewards expecting to gain the benefit of increasing another person’s motivation and behavior, but in so doing, they often incur the unintentional hidden cost of undermining that person’s intrinsic motivation towards the activity.” (Reeve 2004)

Why aren’t we paying attention to this? Or, more to the point, why aren’t we asking a different question when it comes to issues of motivation in the workplace: How can we move from a culture of extrinsic compliance to a culture of intrinsic commitment?

3. The Ecology of Organizational Culture – Finally, there’s the changing way scientists describe the principles of ecology, a word that literally means “the study of the house.” What’s becoming apparent is that order and balance in our house (whether it’s Earth or a country or an elementary school) are not achieved by complex, overly prescribed controls, but by a few clearly delineated simple structures, and with a healthy dose of freedom for individual entities to pursue what they feel is significant. As physicist and systems theorist Fritjof Capra puts it: “In recent years, biologists and ecologists have begun to shift their metaphors from hierarchies to networks, and have come to realize that partnership – the tendency to associate, establish links, cooperate, and maintain symbiotic relationships – is one of the hallmarks of life.”

Apply these insights once again to the K-12 education landscape and you see what to do immediately: move away from the Newtonian change model of “critical mass,” and toward a more modern model of “critical connections.” Educational scholar John Goodlad urged as much following his massive comprehensive study of schooling in America in the 1970s and 1980s: “Schools will improve slowly, if at all,” he wrote, “if reforms are thrust upon them. Rather, the approach having most promise is one that will seek to cultivate the capacity of schools to deal with their own problems, to become largely self-renewing.”

These insights have profound implications for how we structure the science of school renewal – as opposed to the business of school reform – in the years and decades ahead. Instead of a push toward greater standardization and absolute constructs, we should sharpen our assessment tools to become more finely attuned to the relativistic learning needs of children. We should create organizational conditions that nurture intrinsic motivation in adults and children.  And we should be more mindful of the networks and people we will need in order to do the difficult work of systems change, and begin asking ourselves the only question that really matters: Of all the things we can do together, what must we do?