Tag: Early Childhood

DC’s Plan to Assess Early Childhood Programs: Thumbs Up or Thumbs Down?

Posted on | by Sam

In case you missed it, the Public Charter School Board (PCSB) of DC has proposed a common framework for assessing the quality of all preschool and lower elementary programs. The original proposal sparked arguments for and against the plan; led to a petition campaign of protest; and anchored a lively hourlong discussion on public radio. Lots of people wrote the board to share their own ideas and feedback, and, earlier this week, the Board unanimously approved a revised policy.

What did the PCSB get right, and where is its plan still lacking? First, here’s what they proposed (with all changes highlighted in yellow):

 

 

To evaluate these changes, I reviewed the modifications against my three central design principles of a good assessment framework.

  1. Measure the Essential Skills: On the positive side, any school that opts to measure social-emotional growth will be held to roughly equal percentages of importance (14% v. 12% in preschool, and 25% v. 20% in lower elementary). Not what I would do, but I can live with it. On the negative side, SEL measures are still not required, and the past twelve years or so of education policy would suggest that, despite one’s wishes to the contrary, a school that is only required to do A is less likely to do B, C or D with the same degree of intensity. Will a majority of schools opt in to the SEL framework? Time will tell, but I’m skeptical. I do think, however, that the addition of a mission-specific goal provides another way for schools to elevate an essential skill, such as creativity. In sum, a mixed bag.
  2. Default to the Highest Common Denominator: As I wrote previously, “One of the biggest problems with the PCSB’s framework is that even though all schools would be held accountable to the same categories, not all schools would be using the same tools to assess their progress.” The danger of this was pointed out to me by the founder of a prominent charter school in DC, who cautioned that any school that chooses a less challenging assessment in, say, math will be more likely to score higher than a school that chooses a more challenging one. “This,” she says, “creates an incentive for schools to choose less challenging assessments which may provide less actionable/useful data for teachers to use in the classroom, which is what the real point of assessment is.” As far as I can tell, this design flaw is unaddressed by the revisions. That’s a big problem, and one the PCSB will need to get right in the coming year.
  3. Identify the other elements of a healthy school culture: This is where the original proposal was closest to the endgoal, and that’s still true here. Its metric for evaluating teachers has three separate components — emotional support, instructional support, and classroom organization — and its attention to attendance and re-enrollment make sense. Here, too, the addition of a mission-specific goal, depending on what schools choose, could apply to this category (measuring school climate, for example).

Overall, then, I think the PCSB heeded much of what it heard from the public, and its final proposal still needs some small but significant tweaks. What do YOU think?

How Should We Evaluate Our Preschools?

Posted on | by Sam

Imagine, for a second, that you are in charge of more than $600 million in taxpayer money. You live in a city that has made deep investments in early education, and that aspires to provide universal preschool by 2014. You have a thriving network of public charter schools, and you want to help parents make more informed choices about where they send their children.

What would you do?

Continue reading . . .

The PCSB Responds

Posted on | by Sam | 2 Comments on The PCSB Responds

As the deadline for public comment on the PCSB’s proposed accountability framework for early childhood programs nears (August 28) — and as public reactions to the proposal intensify — PCSB’s executive director, Scott Pearson, published a formal reply to the change.org petition that is asking for greater balance in how schools are evaluated. Here’s Scott’s response:

I appreciate your sharing your concern about assessments in early childhood testing.

It’s important to note that the proposed early childhood PMF was developed in close collaboration with charter school leaders who broadly support this proposal.

Among most important features of the PMF was to incorporate the very assessments that these schools are already giving to students, and that have been used for years for school accountability purposes. Schools may choose from a menu of over two dozen assessments – a menu that includes virtually all of the assessments already used by our schools. These bear little resemblance to “fill in the bubble” tests. They are administered one-on-one, by teachers. They are authentic and designed to be developmentally appropriate and to feel low-stakes for the child.

Let me repeat. All of our schools already use these assessments for accountability purposes. The Early Childhood PMF does not impose new tests. It creates a framework for comparing the results at one school using one test with the results at another school using a different test.

Indeed, far from imposing new assessments, the Early Childhood PMF restricts the use of these assessments. For example many schools currently evaluate their youngest students for literacy and numeracy proficiency. The Early Childhood PMF now looks only at whether these students abilities have improved, not at the absolute proficiency levels of students.

PCSB has a vital role to ensure that charter schools, which are public schools paid for with taxpayer dollars, are high quality. We seek to play this role in a way that preserves the maximum autonomy and flexibility for school leaders to run their schools as they see fit. While every school is different, virtually all agree that the vast number of students who enter the third grade unable to read or perform basic math is a crisis in our city and that a basic function of a school is to prepare students to be literate and numerate.

The core bargain of charter schools is greater autonomy in exchange for greater accountability. This means there will inevitably be high stakes reviews of charter schools. Almost all charter schools prefer to be left alone to develop their own educational programs. High stakes reviews focusing on *how* they run their schools run counter to the philosophy of most of our school leaders. They want the freedom to decide the “how” and to be judged on the outcomes.

While it would be hard to find an educator who did not value the social and emotional development of children, most would also agree that valid assessments of social and emotional progress are not yet well-established. For this reason many school leaders are reluctant to have significant portions of an evaluation of their school be based on an assessment of their students’ social and emotional development. This concern influenced the work the task force developing the Early Childhood PMF.

We take the public comment phase of the process very seriously. The views of those who signed this petition, along with those who submitted comments directly to me and to PCSB, will be taken into account. Beyond this, we recognize that no system of accountability is perfect. For this reason, the committee of schools who designed the Early Childhood PMF will continue to meet to consider ongoing improvements.

We continue to discuss this issue on our blog, which can be found at www.dcpcsb.org.

Scott Pearson
Executive Director
DC Public Charter School Board

I’m grateful for Scott’s response, though I don’t feel it addresses the central complaint. As I wrote yesterday, “the problem with the PCSB’s proposal has less to do with requiring schools to choose from a common pool of math and reading assessments and more to do with attaching a disproportionate weight (60-80% of the total) to reading and math. This immediately transforms those assessments from diagnostic to accountability tools. It is guaranteed to modify the behavior and strategic planning of the schools. And it perpetuates the shell game of American public education, in which we use partial information to pronounce complete judgment on whether a given reform effort is working or not.”

I don’t see how Scott’s response addresses that central critique. Am I missing something? What do you think needs to be done?

 

Is It The Tests or the Stakes?

Posted on | by Sam

As DC grapples with whether or not to adopt an accountability framework that would assign between 60-80% of a charter preschool’s overall rank to its students’ reading and math scores, it’s worth asking: What’s at the root of the problem here — the tests, or the stakes attached to those tests?

When it comes to the debate over the Early Childhood Framework (which you can access here — just click “Public Hearing” in the left column and then click “Early Childhood PMF”), I would suggest it is overwhelmingly the latter.

Here’s the thing: just about every early childhood program worth its salt is already assessing children in a variety of ways, including, and not limited to, reading and math. Usually, these assessments are administered in a 1-on-1 format, and if a teacher does his or her job skillfully, the child won’t even register that it’s a “test.” So the Public Charter School Board is correct when it says that testing children in these subjects is not new.

What their explanation glosses over is the impact children, educators and parents can expect once these assessments have high-stakes attached to them. Rankings matter to all schools, but they are particularly important to charter schools, which are always in the midst of raising money, searching for a permanent facility, and recruiting families. Here in DC, “Tier 1” status is a gateway to better fundraising, preferential treatment when it comes to facilities, and longer waiting lists.

The problem with the PCSB’s proposal, therefore, has less to do with requiring schools to choose from a common pool of math and reading assessments (indeed, the list boasts more than thirty options to choose from, many of which were submitted by the charters themselves), and more to do with attaching a disproportionate weight to reading and math. This immediately transforms those assessments from diagnostic to accountability tools. It is guaranteed to modify the behavior and strategic planning of the schools. And it perpetuates the shell game of American public education, in which we use partial information to pronounce complete judgment on whether a given reform effort is working or not. (And, more importantly, in which we allow ourselves to assign credit or blame in ways that correspond to a political timetable.)

In reality, as both common sense and research would tell you, it’s not that simple. “True standards of intellect do not lend themselves wholly to quickly collected, precise, standard measurement,” wrote Ted Sizer in his classic book, Horace’s Compromise. “We need to devise clusters of instruments to probe our students’ ability to think resourcefully about important things. Indeed, we need time to reflect deeply on what we mean by ‘think resourcefully’ and what we feel are the most ‘important things.’”

In his 2009 book Measuring Up: What Educational Testing Really Tells Us, Harvard professor Daniel Koretz makes a similar point. “Careful testing can give us tremendously valuable information about student achievement that we would otherwise lack,” he says. The question is how well we understand what standardized tests can, and cannot, tell us about American schools. “There is no optimal design,” he asserts. “Rather, designing a testing program is an exercise in trade-offs and compromise – and a judgment about which compromise is best.

Systems that overvalue one set of test scores fail to heed this advice, but they also fail to highlight a sufficiently clear picture of what is actually happening in a school. “Certainly, low scores are a sign that something is amiss,” Koretz says. “But the low scores themselves don’t tell why achievement is low and are usually insufficient to tell us where instruction is good or bad, just as a fever by itself is insufficient to reveal what illness a child has. Disappointing scores can mask good instruction, and high scores can hide problems that need to be addressed.”

So let’s vigorously debate ideas like the PCSB framework, and when we do so, let’s follow the lead of experts like Lydia Carlis, the chief of Research and Innovation for the AppleTree network of preschools. “The ongoing conversation will be so useful to all children if we focus on both/and rather than either/or when discussing teaching and (developmentally appropriate) assessments of academics and SEL in early childhood, ” she says. “The early childhood interventions that have been evaluated longitudinally and shown effective demonstrate that low-income children at risk for school failure need early access to high quality, evidence based academic and social supports. Social emotional skills are both necessary and insufficient for closing the academic achievement gap. Children who begin demonstrating cognitive gaps as early as nine months need skill and concept development on academic content and strategic support for social emotional learning. And clearly, all children will benefit from a high quality, balanced instructional program.

Works for me. PCSB, are you listening?