The MET project: Measures of Effective Teaching
New Study Evaluated by NYU Dean
By Dean Mary Brabeck
Teaching is a complex and challenging task, so it is not surprising that finding good measures of teaching is just as complicated. Consider an observation tool to measure the effectiveness of doctors, dentists, or lawyers, and you can begin to see the complexities. The authors of Measures of Effective Teaching (http://www.metproject.org/) have done an enormous service by critically examining the five current observational tools that are most used to assess what effective teachers do. The MET study is the first of its kind: researchers used 7,491 videos of 1,332 teachers who each taught between four and eight lessons, and those videos were rated by 900 trained raters using the five teacher observation measures.
Using a value-added model of standardized test scores in mathematics and literacy, all five measures were associated with student achievement gains. Teachers performed better on tasks like classroom and time management than on higher-order skills like questioning, analysis, and problem solving. This information is useful for those who prepare teachers, and suggest where we should put our efforts. However, subjects like science, history and the arts were not studied, and only teachers in grades 4 through 8 were included.
Researchers reported that a combination of scores on the observation measures, student feedback, and student achievement gains (value-added) are better in predicting student performance than graduate degrees or years of teaching experience. These findings contradict other research and beg for programmatic study of the kinds of graduate experience that promote student achievement and schools that promote ongoing teacher development.
The authors do caution, however, that reliable ratings of teacher practices require multiple observations and suggest that any high-stakes decisions should combine observations of teachers with student achievement data and student feedback. They emphasize training in the proper use of the observation tool, including impartial second raters for at least some data, and ensuring checks on the accuracy of those who are using the measure.
Perhaps if New York City had followed this advice before releasing data based on value-added modeling of student standardized tests, the public would have more trust in the results, and many teachers would not have been undeservedly shamed.
There is agreement that “teachers matter,” as President Obama proclaimed during his 2012 State of the Union. In the same speech, the president noted that we still lack data on what it is that effective teachers do that matters. We are closer to answering that question because of the MET study on the reliability and validity of the five teacher observation measures. #
Dr. Mary Brabeck is the dean of the Steinhardt School of Culture, Education, and Human Development at NYU.
