Formative Evaluations

While most of the media’s attention has been focused on high-stakes summative evaluations, formative evaluations are gaining a lot of traction among teachers.

Summative evaluations are primary used to rank, sort, and label.  Final exams are a good example of these.  When a student takes a final, she does not have any opportunities to revise her work.  Her teacher does not use the results of the final to inform the next stage of the student’s learning.  The stakes of the summative assessment are usually very high.  The score on the final is used solely to rank the student in relation to other students and help give them their grade/label.

On the contrary, formative assessments are used as part of a feedback loop between the learner and the teacher.  The stakes on a formative assessment are usually low, and a teacher will use a formative assessment to see how well her student learned the concepts or skills of the last lesson.  She will use the information to determine what concepts and skills need re-teaching, or if the pace of the class needs to speed up or slow down. 

While teachers are encouraged to use formative assessments with their students, some education reformers are using high-stakes summative assessments on teachers and schools.

I wonder what it might look like is school and district administrators, state and federal legislators used formative assessments to evaluate teachers and schools.

Some reformers want to use summative assessments to label teachers as effective, average, or failing.  This is seductive because it is easy.  All of the responsibility rests on the teacher.  “Your tests scores are low, fix it.”  Could you imagine a teacher behaving this way with students?  I can too.  I can see a teacher presenting information all week, month, or semester then giving a summative assessment.  The kid gets an “F” and the teacher says, “Do better.”

We wouldn’t accept this kind of behavior from a teacher.  Why should we expect this kind of behavior from school administrators or politicians?

As an alternative:

·      School administrators should use formative assessments for their teachers 

·      These assessments should be a blend of observations of the classroom and an analysis of student work

·      The school administrator should use the data she sees in the observations and analysis to identify strengths and shortcomings in the teachers work

·      Then, supports and interventions should be used to assist the teacher improve his work and, consequently, improve the work his students are doing

In parallel:

·      Local and state educational officials should use formative assessments to look at schools

·      These assessments should also include school observations and analysis of school outcome data

·      Schools data could include many things that are far more important that mere scores on a one-shot tests

·      We should look at graduation rates, drop-out rates, and success rates of the schools recent graduates

·      Rather than look at such data and then label a school as “failing,” local and state officials should look at the strengths and shortcomings of the schools and identify supports and intervention to help school improve.

The problem with formative assessments is this: they are hard.  It is easy for teachers to identify strengths and shortcomings of their students, but it is hard to identify the kinds of supports or interventions a child needs to improve.  It is very hard to find the time and resources to implement those supports and interventions.  Likewise, it is easy for a school administrator to identify shortcomings of a teacher, but it is hard to imagine the kind of professional development that the teacher may need to improve.  It is really hard to find the resources to provide extra professional development.  It is easy for government and reformers to point their fingers and say, “Schools are failing.  Look at those test scores.  Shape up or ship out!”

The problem with summative assessments is that they do nothing to help solve the problem.  Teachers can’t help their children after they do poorly on a final exam.  Administrators can’t help their teachers after labeling them as “ineffective” on a review.  Politicians and reformers can’t help schools by just labeling them as “failing.”  As the iconic farmer would say, “You can’t fatten a chicken by weighing it.”

Instead of our obsession for labeling and finger pointing, we need to work together to actually help schools do better.  It’s hard work, but it is the only way we’ll make any progress.

The day I was attacked by a student.

Watching the chair fly through the office door, I thought to myself, “This isn’t going to be good.”

I had not come to Vice Principal’s office looking for a fight, or even an argument.  I wanted to talk about a new chart that we could use to track the daily progress of our kids who were in danger of failing their classes for the semester, and hopefully get some of them back on track.  Someone had other ideas.

I hear that it started suddenly.  Josh (not his real name) was in the office with his mom to talk about his re-entry to school after participating in a large-scale fight.  The conversation wasn’t going well and the Vice Principal had said that perhaps today was not the best day to have this meeting and that the three of them should try again tomorrow.

Josh was unhappy.  We could tell right away because he swept all of the binders off of their shelf and onto the floor, before grabbing the chair he had been sitting in and flinging it through the door and into the introduction to my story.

As he fled the office, Josh ran straight into me, his arms flailing, one fist connected with my face, the other with my shoulder.  Turning around, Josh spied the guillotine paper cutter and grabbed it off of a table.  My teacher readers know well the equipment I am writing about and their minds have probably already gone where my mind went when I saw him heave the paper cutter.  A guillotine paper cutter is a fourteen-inch blade attached to a base on one end.  The blade is sharp enough and heavy enough to cut through twenty or thirty sheets of paper in one “ker-chunk!”

In my mind’s eye, I pictured the huge gash on Josh’s hand.  The Vice Principal imagined the paper cutter hurdling through the air directly at the other two children in the room, inflicting horror-movie injuries.

While his mom screamed, “How can you be doing this?  I didn’t raise you like this!” Josh raised the cutter over his head.  I snatched it from his hands and set if behind me.  As Josh sunk into a chair, The Vice Principal and I began talking softly to him.  “It’s going to be okay, Josh,” we said over and over.  “Calm down; just breath; we can start making this better now.  You’re going to be okay.”

Security entered the room to see Josh sitting calmly in a chair, head in his hands, crying.  The Vice Principal and I were still talking calmly to Josh, thinking that the worst was over and that we were on the road to recovery.  Our attendance officer was calming mom down.  The police had been called and were on the way.

Josh calmly got up.  Reentered the inner office.  We thought he was moving to shake off some of his adrenaline, but we were wrong.   Instead, he exploded again.  He swept the coffee maker from the table, smashing it against a bookcase.  He tore the printer off of the desk and threw it to the ground.  When on of the security guards and I grabbed him, he leapt into the air and kicked the desk lamp, crushing it against the wall.

Another security officer joined the two of us in wrestling Josh to the ground.  “Let me up!” he shouted.  “No.” three of us replied.  We got a handcuff onto on wrist, turned him over, and cuffed his other hand behind his back.

“Ring!” went the bell; and I had to leave to teach my classes.

Later in the day, I found out that Josh continued his violent outbursts until the ambulance arrived.  It finally took seven adults to strap Josh onto the gurney for the ride to the hospital.  This had not been the first time he had been admitted because he posed a danger to himself and others.  “It’s been like this every time he looses his temper,” mom explained, surveying the destruction.  “My house looks just like this.”  Later, we found out that Josh had been severely beaten since early childhood.  His father is currently serving time for the abuse. 

Days later, once I finally got over the shock, I began to wonder about Josh; about his childhood; about his illness.  I wonder what his test scores are going to look like this year.  I wonder how his teachers might be evaluated if the school used Josh’s tests scores for the Value-Added Measure.  I wonder what the so-called reformers would say if they had been in the room with Josh instead of me.  Would they still claim that poverty, or living in violent neighborhoods is no “excuse” and that a great teacher could compensate for Josh’s circumstances and help Josh succeed on the high-stakes test?  Would they disagree with me when I think that Josh has bigger issues to deal with this year than the Pythagorean theorem?

I can’t help but wonder: WWRD?  What Would Rhee Do?

Of Course Money Doesn't Matter in Education

Forget the fact that:

Oakland Public Schools spend                           $4,945     per student, and
Secular Private Schools spend on average         $20,100   per student, and
California spends                                            $224,712    per child in juvenile detention.

Nothing to see here.  Move along.... move along...

(Oakland Public Schools and California Juvenile Detention numbers from Harper's Index, November 2010;  Secular Private Schools average a Washington Post article, "Per Student Spending Gaps Wider Thank Know, August 2009)

Subversive Teaching and Jewish Vampires

Teachers know their kids better than anyone else in the education system.  They know what academic strengths and needs each student has.  They know where their interest lies and how to bridge student curiosity to academic goals in order to make learning meaningful and fun.

When teachers receive guidance from above, from site or district administration or from state and federal governments, they embrace this guidance if it meets two conditions.  First, the guidance needs to be congruent with what the teachers know is in the best interest of their children.  Second, the guidance needs to allow for teachers to be flexible and creative, engaging their students in meaningful learning and igniting their passions and their sense of wonder.

“Mr. Orphal,” a fourteen-year-old freshmen began.  “What if a Jewish person turned into a vampire?  Would that vampire be afraid of a Star of David like regular vampires are afraid of a cross?”  That question launched a ten-minute conversation about the concept of vampires in legend and their use as a literary device, so popular in modern novels, television and film.   We talked about the purpose of vampires in 18^th century literature as monsters that story-heroes can fight and defeat.  We talked about the evolution of vampires into darkly sexual creatures, and eventually into social outcasts trying to fit in with everyone else.

The conversation ended with me encouraging the student to write that story.  As the author, he could decide how the non-Christian vampire would react to religious iconography.  Through his story, he could explore his thinking about religion and the nature of God.

A short story like this would be a wonderfully entertaining way to assess this boy’s current mastery of language.  What vocabulary would he choose to use?  How would he describe the setting of the story?  Who would be the protagonist?  What perspective would he use? How would he use grammar and diction to make his story compelling?  We could even look at his spelling and his editing choices.

A short story like this would also be a great way to examine his ideas and his critical thinking skills.  What choices would he make about his Jewish vampire?  Would he choose to affirm an exclusive rightness for Christianity, keeping with traditional notions of vampires and crosses?  Would the Jewish vampire cringe at a Star of David?  If the vampire did fear Jewish religious symbols, what would this mean? Would this mean that he had shifted exclusive validity from Christianity, as in many traditional vampire stories?  Alternatively, would he choose to explore an idea of morality and universal goodness where Hindu, Buddhist, or Muslim vampires would cringe at symbols from the religions of their previous cultures?  Would these vampires even try to negotiate their current undead reality in light of a religious morality the characters were trying to maintain?  Each idea becomes an intriguing path to deep and critical thought.

Unfortunately, I doubt this student’s English teacher will be more than superficially supportive of his idea.  This should not be read to cast blame and shame at his teacher.  Rather, this should be seen as an example of how teachers, especially teachers of English and mathematics, are hand-tied by the need for their students to score well on up-coming high-stakes tests.

It takes a subversive teacher to engage in such meaningful and fun learning these days.  A teacher would need the courage to forgo her district and state mandates to allow this child to explore his ideas.  A subversive teacher may be able to treat district mandates as a sort of “paper work game” as Linda Darling-Hammond describes in The Right to Learn.  In her book, she quotes one such teacher, “A lot of the goals of the school district can be taken care of by putting some words on paper and then going ahead and doing the things the way you want to.”

Current education reforms such as former DC Chancellor Michelle Rhee would tend to argue the words of the above-quoted teacher, but not the intent.  She would say that accountability guidelines are meant to give good teachers exactly the kind of flexibility I describe in this essay and reflected in the teacher’s statement.  She would say that great teachers do exactly this; they use frameworks to build rigorous, engaging learning experiences for their children.

Where her argument runs off the rails is the moment the high-stakes tests come into play.  Creative writing, deep thinking, and wonder are not assessed on any of the current testing tools, and they are unlikely to be so in the next generation of assessments currently being designed.  This is because deep thinking and creativity are really hard to test.  Instead, current exams measure low-level sub-skills that translate well to bubble-in-the-answer questions.  They look at questions where there is one right answer that can be hidden among three red herrings. 

Unfortunately, it will be the score on such a test that will be used to judge the student, teacher, and school.  The student’s score, compared to what Value-Added statisticians imagine he would have scored with an “average teacher,” will be used to label his teacher as effective or not.  If Ms. Rhee has her way, this score would then be used to decide whether or not the teacher remains in the classroom.  Under such pressure, it is hard to imagine a non-subversive teacher allowing hours to be spent exploring vampirism in non-Christian communities.

This is too bad.  I wish Ms. Rhee and other reformers would be willing to read some essays by this child from last year and then read the story of the Jewish vampire.  I can imagine these pieces of data would point to some great learning on the part of the student and some great teaching on the part of his teacher.

Until then, so-called reformers will continue to point at test numbers.  They’ll do so because test numbers are easy to point at, not because they even remotely reflect the totality of so complex a process as learning.  The story of the Jewish vampire will have to wait until after school.

The Factory Model of Education Reform

 The current education reform movement is based on the factory model.  In that model there are a few at the top of the organization who are the thinkers and the planners.  The rest of us are unskilled workers, expected to obey the mandates from on high and implement the plans from our bosses.  As Fredrick Taylro said in 1911 and quoted from Darling-Hammond’s 1997 The Right to Learn, “One type of man is needed to plan ahead and an entirely different type to execute the work.”

In the education reform movement, the one type who plans ahead are the men and women in administrative positions, far away from the students.  They are the politicians, the state and local superintendants of education, the policy wonks located in foundations and think tanks.  Teachers are treated as the uneducated workers who are expected to “execute” the reforms faithfully, unquestioningly and thoroughly.

A part of the problem in the education reform movement is the seemingly hypercritical nature of reform.  Simply put, our actions do not match our rhetoric.

We talk about how much money is spent in education in general and spent on personnel in particular.  We assume that this money is spent on classroom teachers, ignoring the reality that nearly half of school employees are not in contact with students on a daily basis.  Instead, these employees are busy filling out the accountability forms, ensuring that teachers are, in fact, doing more with less.  Current educational reformers want the public to think that public school teachers are becoming wildly rich over the course of their careers, ignoring the reality that the average public-school teacher makes only $55,000 a year.  So-called reformers have even been brazen enough to compare teacher retirement programs to the multi-million dollar golden parachutes of corporate leaders.

We talk about how they want kids to deeply understand mathematical principals, but the tests that we use to determine if this is happening cover only low-level sub skills that are easy to test.  We say that we want kids to write and to appreciate great literature, but schools are held accountable on how well students can identify the definition of a word or edit basic grammar.  We say that we want kids to explore historical trends and themes, but we test them on memorized facts.  We say we want them to experiment, but test them on memorized science facts.  We say we want them to become decent human beings, but play time and learning experiences about acceptance and conflict solution aren’t on any tests and consequently are deemed unimportant, cut first in hard economic times, and sacrificed on the alter of ever-higher test scores.

Even when it comes to our academic standards, we seem to have a split personality.  The new Common Core Standards look great on paper.  Classroom teachers who have studied these new standards are typically hopeful, seeing how their classrooms would return to exploration, experimentation and authentic learning as they meet these standards.  The problem is that the several state standards that the Common Core are replacing look good on paper as well.

The problem is the tests that we use to assess the standards and the tremendous import we place on the scores on these tests. 

Even the most pro-test educational reformer admits that the tests are flawed, assess only low-level skills, and give only a snapshot of student performance.  Despite this acknowledgement, pro-test reformers want to ascribe even more import to the scores.  They want teachers ranked based on these scores.  They want schools graded on these scores.  They want public education judged on these scores.  “They are not perfect, but they are all we have,” say some reformers, painting a picture that the test scores are near perfect rather than near catastrophic.  The analogy isn’t pouring sand on a fire for lack of water; it is more like using gasoline.

The crucial work about the Common Core Standards is happening right now.  Two consortiums of states are designing the assessments that will be used to judge student, teacher, and school success on the Common Core Standards. 

The rhetoric surrounding the next-generation assessments is hopeful.  Secretary of Education Arne Duncan says that the next generation of tests should not be fill-in-the-bubble based.  The consortiums are placing a lot of hope in emerging technologies that may be able to marry the speed and cost-effectiveness of scan-tron assessments and the need to assess essays and projects.

I am wary.  Perhaps the next-generation assessments will be significantly better than the last generation.  Perhaps, the assessments will look at skills more complex than memorization of facts and procedures.  Unfortunately, the desire to ascribe great import to the next series of test scores look to be even greater than kids are currently expecting.

I imagine that a decade from now, English teachers will decry how student essays are locked into the formula and patter that the computerized essay scorer recognizes as proficient.

Unfortunately, I am confident that classroom teachers will have very little influence over the look of the new assessment and the new curriculum that will be designed to meet the Common Core Standards.  Teachers are not the “type of man is needed to plan ahead,” but considered by so-called reformers to be the “entirely different type to execute the work.”