First Post from TEDxSFED: The Freeze!

Using Student Performance Data to Evaluate Teachers

Two hundred teachers sat in the hotel’s conference room in Oakland, California, drinking coffee and getting ready for a two-day discussion on Effective Teaching.  By the end of the summit, our hope was to have some agreements about what effective teaching looked like: an idea on how the schools and the district could measure it; and a plan on what to do to highlight teachers who were are most effective and help those of us who are struggling to be more effective.

As we greeted one another and talked about the coming days, the Jazz band from one of our middle schools set up their instruments and began to play.  I turned to my colleague seated next to me and said, “This is what, ‘using student performance data’ to evaluate teachers should look like.”

Imagine for a moment that Jazz band.  There they sat, performing.  None of the students was concerned about getting an “A.”  They were concerned about being perfect.  In math, English, social studies, or science, a student may be trying to get 90% of the questions on a test right, but not the band.  If a member of the band missed only one note in ten, that wouldn’t be an “A-“, it would be a catastrophe.  Imagine the hours of “homework” each child put into this class, preparing for this test.  I can imagine there were many hours of “homework”, far more than the number many of them put into their other classes.  One of the secrets to this level of dedication was that the “homework” was actually called “practice” and that the “test” was a performance, in front of a real audience.

I think of this story as I read the report published by the Denver group of the Center for Teaching Quality’s New Millennium Initiative (NMI).  In the interests of full disclosure, I am a member of the NMI group based in the San Francisco Bay Area. Over the last year, while the Denver group grappled with teacher evaluation models, our group discussed what a better teacher induction and career advancement model might look like. 

The Denver report identified four key areas of work that the group believes will lead to a better teacher evaluation system:

•  Developing meaningful measures of student growth (including in non-tested areas) to comprise 50 percent of a teacher’s evaluation, as required by state law
• Defining qualifications and training for evaluators
• Determining how to account for school conditions and student factors in a teacher’s evaluation
• Designing an evaluation system that informs both employment decisions and professional growth and learning

This post will focus on the first key area, that evidence of student growth should comprise 50% of a teacher’s evaluation.

Frankly, I worry about so much of a teacher’s evaluation being tied to student performance.  I worry, because I fear that “evidence of student growth” too often is code for the single score on a fill-in-the-bubble test.  Over the last ten years, I’ve seen deep and rich curriculum in California become shallow and bland as schools lost their focus on student learning as the obsessed over tests scores.  I’ve seen teachers push research projects, student portfolios, and service learning to late may and June, after the high-stakes tests were over and done.  I’ve heard principals in many schools change their tunes from, “We’ll never let the test dictate what we teacher” in the 1990’s to “We already know what materials are really important; the state has put that material on the tests.”

So how do we navigate these treacherous waters?  I, like many of my colleagues at the Center for Teaching Quality, want teaching to evolve into a results-orientated profession.  We’ve chosen a new version of the old adage, “It’s not what you say; it’s how you say it.”  Instead, we say, “it’s not what you say; it’s what they hear,” meaning that it is less important what a teacher teaches and more important what a child learns.  More and more teachers are getting on board with this message.

In the June 2^nd edition of the Public Agenda Alert, “Teachers and the Tests,” they mention, “The conventional wisdom among many education reformers has been that teachers resist all kinds of evaluation, but in fact they're open to a number of ideas, according to the research we conducted with Learning Point Associates. Nearly all teachers (92 percent) rated the level of student interest and engagement as an excellent or good indicator of teacher effectiveness. Teachers also gave excellent or good ratings to how much their own students learn compared with other students (72 percent) as well as feedback from principals and administrators (70 percent).”

State departments of education should take a cue from the Oakland middle school Jazz band.  We can evaluate teachers based on what children learn, but those assessments have to be composed of meaningful work.  Imagine an alternate scenario, where the jazz band read a textbook about the theory of music and instructions about how to play an instrument, then gathered in the hotel’s conference room to “wow” us by filling in the correct bubbles about music theory and instrument play.  How many hours do you imagine children would spend studying for that test?  How well would a test like that tell us about what the children are learning and how effective the music teacher is?  It’s a ridiculous scenario, for sure.  However, it is exactly the way we have been evaluating schools for the past decade and how New York and Los Angeles are currently ranking their teachers of English and math.

So how to we navigate these treacherous waters?  I think the Denver NMI group is taking the first steps; they are getting the voices of teachers into the debate.  My hope is that the education policy makers in Colorado are ready to hear.  Teachers are in favor of having our jobs evaluated, at least partly, on how well our children are doing.  I can imagine the Oakland music teacher being willing to allow his evaluators listen to a performance in October, then another in May, and then stand behind the progress his students have made.

Other teachers, in other subject areas, could take the same stand.  While I still say, “NO” to having myself, my school, or my children judged based on how many bubbles they filled in correctly, I stand behind their research essays.  “Read one of their essays in October,” I told my administrator last year.  “Read another one in May, then you’ll know how well I’m teaching my kids.”  If I were an English teacher, I would say the same thing about my kids’ essays or poetry.  If I were a science teacher, I would say the same thing about my kids’ experiments and lab write-ups.  If I were a math teacher, I would say the same thing about my kids’ projects.

We really can use student-learning data to see how well students are learning and how well teachers are teaching.  The trick is to be more concerned with the quality of that data than the ease with which we gather the data.

Growth Grading

 Imagine a typical student who is failing a class.  This student is earning low scores on his quizzes, exams, papers, and projects.  For the sake of this thought experiment, I’m going to stick to several imaginary assignments that each have 100 points possible as a perfect score.  In the real world, teachers know that all assignments are not equally weighted, but the concept works with any assignment, and I want to keep the math easy for this article.

In this scenario, the student earns 20, 22, 28, 35, 45, and 60 on each of the assignments.  In this imaginary class, the scores break down along the traditional 90-100=A, 80-89=B, 70-79=C, 60-69=D, and <60=F.  So this student has earned an F on each first five assignments and then a D on the sixth. 

In a traditional grading system, I can imagine this student giving up on this class.  As you can see, even though the student has improved on each and every assignment, he sees very little benefit from his increased efforts.  I can imagine many of my students ever even completing the forth, fifth, and sixth assignments at all, because they have given up hope.  I can hear the child say, “Why should I even bother!  No matter how hard I try, I’m still failing!  I give up!  I hate school!”  With this imaginary student, and hundreds like him at my school, I can see a dropout in the making.

With a growth model for grading, this child can find reward in his increased efforts and hard work, and his improved performance.

Under a growth model, after the first assignment earning a 20 the teacher would have a conversation like this.  “You earned only a 20/100 on this assignment.  Now, you and I both know that this is an F.  What I would like us to agree to is this: you come in for extra help, and make some commitments to work harder.  Our goal is a 10% increase on your next assignment.  So you’re shooting for at least a 22 on the next assignment.”  The child asks, with typical teen hope, “Will a 22 get me an A, then?”

The answer is, “no.”  Here is how growth grading works.  On the next assignment, the student earns the 22.  A 22/100 is still and F on content.  But the student hit is growth target perfectly.  On his growth goal, he has earned an “A.”  I the growth model, these grades have equal weight, so the total grade for assignment two is a C.  Then the teacher and student set the next growth target, another 10% or a 24. 

On the third assignment, the student exceeds his growth target, earning a 28 instead of the hoped-for 24.  His growth grade on assignment 3 is an A+, but his content is still an F.  This averages to a C+ on the third assignment.  Because the student’s growth was so impressive, he and the teacher set the growth bar a little higher, say 20% growth this time, or a 34. 

On the forth assignment, he does well again.  Exceeding his target slightly with the 35.  Again, this is an F for content, but another A for growth, averaging to another C for the assignment.  Together, they set the bar higher, hoping for 25% improvement on the next assignment. 

Again, the student hits his target with a 45.  Again the content and growth grades average to a C.  Again they set a growth target, 30% this time, aiming for a 60

On his sixth assignment, he does it again.  This time, while his growth grade is another A, his content grade has increased to a D.  The average of this assignment is a B, perhaps the first B this child has earned in this class all year.

Can you see the reward for perseverance?  Can you see the child realizing that hard work and study pays off over the long haul?

With this student, once his content grades have climbed to a C or a B, the growth grading ends.

I know this model flies in the face of some practitioners who are strong advocates for strict standards.  They may argue that the content grade is too important to dilute with a growth grade and that an F is an F. 

I know this model also flies in the face of fairness.  If this student can earn a C for only 45/10 correct, what does that say about the student who is earning a 75/100 and earning the same letter?

I also know that this model of grading will hurt my school’s score on the high-stakes tests in May.  This is because this child is not learning the material well enough to do a great job on that test, and the State of California is far less concern with a single child’s growth than they are about being able to rank my school based on a singe test score.  Additionally, growth grading will hurt my school because, rather than abandoning hope and dropping out in the fall, this student will still be with us in May, along with his lousy test score.

I know all of this, and I don’t care.  I don’t care because I also know that this child is learning more of my class content earning a 22, 28, or 45 than he would have had he dropped out.  I don’t care because I would rather have this student in my school dragging down our test scores than on the streets dragging down his future.  I don’t care because I care more about learning and hope than strict standards and fairness.

The trick is to see every child as an individual and find a way for each child to learn as much as she or he can every hour they spend with me.  Growth grading is not for every student, but for those students who may lose hope, who may drop out, it can be a great tool to keep them connected to school and teach them that hard work and effort really can pay off.

Zombie Horde of Bad Teachers

I disagree that there are hordes of "bad" teachers who, like zombies, are after our children's brains.

I work with a bad teacher.  Everyone at our school knows this teacher.  Most all of us agree that this teacher should not be in the classroom.  We talk about this teacher among ourselves, and to our non-teacher friends and families.  I know that kids are talking about this teacher too.

That's nearly 100 adults at my school talking about this 1 teacher.

That's over 1500 teenagers at my school talking about this 1 teacher.

In our stories, this teacher is no longer just one teacher, this person is now 1600+ lead characters in 1600+ stories.

This is how a small number of bad teachers can seem to grow to a horde.

What Coaching Should Look Like

It seems to me that teacher evaluations are personality driven. 

I’ve known administrators who were fully bought into the adversarial relationship between teachers and principals.  These administrators seemed to relish taking a “tough stand” on “bad teachers.”  The teacher evaluations were an elaborate game of cat-n-mouse.  I saw teachers, whom I had never seen with a lesson plan, busy copying materials and checking in with colleagues do see if they had differentiation, multiple learning styles, and other key buzz words, “covered” in their lesson.  Conversely, I’ve seen good teachers dinged over trivial issues, such as the daily learning target stated in the old-fashioned “Students will be able to…” rather than using the students voice by writing, “I will…” that is currently in vogue.

The arrival of these “gotcha” evaluation signals the opening of the second round of the fight.  Not a year goes by without one of my colleagues coming to my door to ask, “Can you believe this?   He gave me a ‘2’ on this!  I got a ‘1’ on that!  (We use a 1-4 point scale at my school, with 4 being excellent.)  He was only in my room for ten minutes!”  Trying to take advantage of a potential learning opportunity, I always ask, “Are any of the remarks things you think you should work on?”  Typically the answer is a colorful version of, “No.”

I myself was the victim of a different form of bad evaluation.  It was my first year at my new high school, even though I walked through the door with twelve years of experience teaching history, as far as my new placement was concerned, I was a new teacher.  It took three meetings for my evaluator and I to settle on my goals and objectives for the year.  Silly me; I thought they were my goals and objectives, but clearly I has misunderstood.  The vice-principal couldn’t just tell me which criteria that I would be measured by, because technically they still were my goals and objectives for the year.  Instead, like a good teacher with an unruly child, she would guide me to the correct goals that would be mine that year.  After three meetings, I had her signature on my pre-observation form, but it had cost me my enthusiasm for having a second set of eyes looking at my craft and offering me insights.

The scheduled observation never happened.  Month after month passed.  Three times I scheduled opportunities for her to visit my class.  Eventually, I stopped rescheduling, telling myself that the responsibility now rested with her.  I finished out that year never seeing receiving any feedback.

It doesn’t have to be this way

This year I’ve seen an administrator who is personally invested in coaching.  He sees his role as being an extra set of eyes for his teachers. 

Yet again, a teacher came to my door to talk about her observation, but this time the conversation looked different.  The teacher still received some 2’s on her evaluation, but they were in the areas that she was asking for help in.  She and the administrator had talked about those areas.  The amazing thing for her was this, instead of going over what went wrong or how to fix it, the administrator asked “What do you think you need this year in order to work on this?  How do you think I can help you?”  These were two questions she had never heard before.

As cool as that experience was; as invested in teacher coaching as this vice-principal is, it is still a symptom of the problem I’m laying out here.  Teacher evaluations are personality driven.  One principal sees himself as the new sheriff in town running the “bad” teachers out on a rail.  A vice-principal is too overwhelmed to find the time to even visit the classroom of the teacher she is supposed to be observing.  The third administrator does a great job, but only because he sees himself as a coach.

Formative Teacher Evaluations

Instead of paying “gotcha,” every teacher observation should be a partnership between the teacher and her/his observer.  There needs to be enough trust to have hard conversations if necessary.  Both the teacher and the observer have to be allies in the goal of helping the teacher be even better than s/he was the year before.

I can envision a future where the observer could be an administrator or a teacher-leader who spend a part of her job mentoring and coaching her colleagues.  I use the term “teacher leader” because in my mind’s eye, the people who are observing, evaluating, and coaching teachers are teachers themselves.  Gone will be the days when a teacher might push-back on a observation saying, “It’s been x years since the VP have taught kids, he doesn’t know what I am dealing with.”

In my fantasy world, every observer would behave like the “coach” vice principal at my school.  The observer would have conversations with the teacher being observed.  Together they would identify areas where the teacher thinks s/he is struggling, or additionally, areas where s/he is good and interested on getting better.  I envision post-observation sessions where both the teacher and the observed can evaluate the teacher’s performance together.  They can look at what’s good and what needs improving.  Them together, they can create a plan by which the teacher gets the support and time s/he needs to make the improvements s/he wants.

The problem with formative assessments is this: they are hard.  It is easy for a school administrator to identify shortcomings of a teacher, but it is hard to imagine the kind of professional development that the teacher may need to improve.  It is really hard to find the resources to provide extra professional development.  It is far too easy to point fingers and say, “Look at those test scores.  Shape up or ship out!”

Differentiated Professional Development Could Help

Professional development (PD) at my school has been a mess.  At the best of times, we gather all of the teachers into the library to hear a day-long speech from the guru that some one in the principal’s office or downtown had met at a conference and felt, “This will be the game changer at our school!”  In the library, some of the teachers agree.  They also think that the guru is the bee’s knees.  For them, the lessons and ideas of the workshop will live in their classrooms, for a while.  For most of the converts, the lack of on-going discussion, collaboration and support will result in a gradual fading of the new ideas and a return to business as usual.

Unfortunately, most teachers wont connect with the workshop.  They wont see the relevance, or it will look like the reform idea they saw tried-and-died years ago.  These teachers may try to make the most out of the professional development day by grading papers, but some may form affinity groups to chat among themselves or snark at the guru.

At their worst, PD at my school devolves into administrative meetings.  Last year, I was the one responsible for this disappointment.  Lat year, my school engaged in our Western Association of Schools and Colleges (WASC) self-study.  This huge undertaking ate up nearly all of our so-called PD time.  Speaking for myself, I learned a lot from this process.  Being one of the people responsible for writing the report allowed me to take a glimpse at everyone else’s classrooms and pedagogy.  I saw some great things that I then wanted to implement in my classroom.  I saw some of my own mistakes mirrored by my colleague, making it easier for me to see them.

While I was leading our “PD” about our WASC self-study, I couldn’t help but think of all of the first-year teachers in the audience.  I was asking for their help in evaluating the school when many of them needed to know where the photocopier was.  They were helping us create a five-year plan for improvement when they didn’t yet have a handle on their own classroom management.  I kept thinking to myself, “They should be in these meetings.  They have entirely different needs!”

What if our school had several “Master Teachers”?  These would be the veterans who are widely viewed as having great lesson planning skills, great instruction, great classroom management, and so on.  What is the school designed a Beginners Teacher Institute where 1^st – 3^rd year teachers held their PD sessions with those Master Teachers.  For example, I’m widely seen as having good skills in conducting a parent-teacher conference, getting even angry parents to see the value of joining forces with their child’s teachers to form a success-team dedicated to getting improvements done. 

After their third year, new teachers could be designated as “Professional Teachers.”  Professional teachers could form their own Professional Learning Communities (PLCs) around themes that are relevant to them.  One PLC could self-organize a year-long professional development series about integrating more Web 2.0 technology into their classes to enable students to collaborate on their projects.  Another PLC could form to further their work on classroom management because they see themselves as struggling in this aspect of their teaching.  An administrator, using formative evaluations, could help a teacher identify this need and connect with other teachers in the school who share this concern.  A third PLC could spend a year doing action-research about their new project-based learning unit.  The ideas just go on and on. 

Master teachers would lead hybrid professional lives.  In my mind’s eye, I see these great teachers having a foot in two worlds.  They spend a part of their day in the classroom teaching a reduced load of children.  In an elementary school, this could be accomplished by having Master teachers share a class, one teaching Monday through Wednesday, the other taking over Thursday and Friday.  In a secondary school,  I see Master teachers in front of children for two or three periods a day, having the other two or three periods for research.

Some master teachers would be paid by their school districts to create professional development lessons and then facilitate beginning or professional teachers who want or need that PD.  Other master teachers could work one-on-one with new hires, mentoring and training them, ensuring that fewer new teachers leave the profession before they have taught for five years. 

The money to do this is already in the system.  Every year, I hear once again how our states are facing a teacher shortage.  Policy makers look for creative ways to fast-track new recruits into the classroom.  This is a false crisis.  The reality is, we have a teacher-retention problem.  Over ½ of all new teachers are out of the classroom before their fifth year.  Thinking about a teacher-recruitment crisis is like thinking about filling a bathtub that has a hole in the bottom.  Some folks are trying to get more water into that tub faster and faster.  All we really need to do is fix the leak.

Huge sums of money are spent recruiting, training and placing new teachers into our schools.  That money would be better spent supporting and mentoring the new teachers who are already there. 

The personnel are already in the system.  In California, over ½ of people who are paid with educational dollars are NOT in front of children.  They are site administrators, professional development coaches, trainers, curriculum writers, managers, and policy wonks.  Many of these positions could be converted into hybrid roles.  Classroom teachers could spend a part of their day being local and state educational leaders.  Conversely, educational leaders would see their standing with teachers skyrocket if they could tell stories about their own, recent, successes and struggles in their own classrooms.

Teacher leaders.  It’s a radical idea whose time has come.

What is the Value-Added of a Gunshot Wound?

Recently, I was asked, "Dave: As a more experienced teacher who has a track record of helping kids reach high academic standards, what are the pros and cons of using VAM to assess both student performance and teacher effectiveness."

Here is my answer:

As a classroom teacher, I dont see any pros to me specifically vis-a-vis the latest Value-added metrics.  The whole point of Value-added is to attempt to isolate the many non-academic issues that follow a child into the classroom so that one can feel that, when they look at comparative tests scores, they are looking only at what the teacher has control over, or the teachers "value added."

The reason why I don't see VAM as helpful is because I am already in relationships with my kids.  I already know that Kevin was shot twice in the leg this year when he was caught up in a scene of random violence that is all-too-common in his neighborhood.  I already know that Shanice ran away from home for two months this fall and was living on the streets of Oakland doing God-knows-what to survive.  I already know how these incidents and hundreds like them are effecting how much and how well my children are learning.  I am already making adjustments to help these children heal from the too-adult wounds they have been afflicted with; help them move on; help them move back to something more resembling a childhood; and help them reconnect with school and learning.

In terms of the larger educational reform debate that is playing-out across the country, I think VAM is a useful tool.  While I continue to rail against any idea that a single test score can be conflated with anything as complex as "learning," I do see VAM as an attempt to acknowledge that great teaching looks different in the suburbs than it does in the ghetto.  I'm concerned that the usefulness of VAM could "rub off" onto the tests themselves, which are appalling.  The tests we have been using to "grade" public schools have been measures of only the isolated facts and simple sub-skills that are easy to measure.  They have not been the skills and knowledge we say we want children to know and be able to do when we adults sit around a table and discuss the purpose and goals of public education.  I would like VAM to be used with a far better set of assessments. 

For example, my school district, Oakland Unified, wants to know that every sophomore and junior can write an essay answering a deep and complex historical question using primary source documents.  This is a test I strongly support.  It is assessing real-world skills of document analysis, bias/perspective understanding, argumentation using evidence, and writing.  I would be happy to see a part of my evaluation as a teacher based on how well my students essays progress from Fall to Spring.  VAM may help an outside evaluator understand why Kevin and Shanice didn't improve on their writing this year, as we were working on far more important issues.  Ultimately, this is who VAM is for, it is not for the classroom teacher, it is for those who wish to evaluate the classroom teacher.

Ultimately, as it is used currently, VAM is useful to the person who is located far away from my classroom.  VAM is useful to someone who does not want to see and hear and feel the messiness of these children's lives.  VAM is useful to someone who really, really wants the job of seeing if children are learning to be as easy and convent as looking at a spreadsheet.  VAM is useful for someone who want to honestly acknowledge that thousands of American children are living in violent and impoverished neighborhoods with having to really know what that really means.  

Book Review: To Teach: The Journey, in comics

 Greatness in teaching engages students, interacts with them, draws energy and direction from them, and offer reasons to plunge into classroom life.  Greatness in teaching is always in pursuit of the next challenge, the next encounter… greatness demands an openness to the new and the unique.  For great teachers, it must always be, ‘Here I go again.’

Bill Ayer’s reworking of his book, To Teach: The Journey, in Comics is a wonderful, inspiring read, reworked into a comic medium (don't call it a genre) with illustrator Ryan Alexander-Tanner.  One can only imagine the twists and turns of Ayer’s journey from being on the FBI’s wanted list as a member of the Weather Underground during the height of anti-Vietnam protests to being a kindergarten teacher and now professor or education.

I first met Mr. Ayers when he keynoted the Humboldt State University inaugural Education Summit.  As part of the organizing committee under my then-mentor Eric Rofes, I was gifted with hours of conversations with Mr. Ayers over breakfasts and dinners.  He is an amazing storyteller and a wonderful teacher.

Most of the story is about Bill thinking about teaching and about how his children learn.  There are great stories about exploration, creativity, perseverance, and wonder. 

Sprinkled through out are vignette from other teachers that Bill clearly admires.  One such story talks about an elementary school teacher who has her class fill up a bookshelf every year as they learn about something that she, the teacher, knows nothing about.  Together the teacher and the students become co-learners, exploring where their collective curiosity leads them.

Another story is about a high school teacher who is trying to “teach a really good kindergarten class with 18-year-olds.”  In this story, we learn about how scary it is to allow students to control the learning environment, and how rewarding it can be.

Periodically the “specialists” form the central office show up in Bill’s classroom to explain to Bill how his children should be labeled as deficient or “at risk” and what standards he should be covering with his kindergartners.  When they leave, Bill jumps upon his soap box to talk about how the standards movement, and it’s enforcers, get in the way of the very human relationship that is teaching and learning.  After one such meeting, when his children are diagnose as ADHD, learning deficient, and “at risk” he states, “Focusing on what I can’t do diminishes hope and limits possibility.  It pays no attention to what I can do.” 

When the “specialist” leave, one of Bill’s children asks, “Are they coming back?”  When Bill answers in the affirmative, the child speaks with what many teachers may agree is their own voice…  “They’re weird.”