Measuring the Effects of Effective Teaching

Standardized tests are not the answer to the question of how to increase student performance. They may, however, -- properly developed and analyzed -- be a tool that can help us find the answer.

I've always been skeptical of society's oft-held belief that student scores on standardized tests are an indication of teacher quality. So many factors besides the teacher -- socioeconomic background, family support, intellectual aptitude, personality, familiarity with test formats, self-confidence, previous instructional quality, even the weather -- can influence student test scores! I believed that even a terrific teacher, faced with a classroom of low achievers, could look ineffective in the reflected light of uniformly low raw scores. I was wary of ever using such scores as a basis for evaluating teacher performance.

Last week, in State Tests Don't Make the Grade, a StarrPoints column on standardized testing, I wrote, "Perhaps the fault lies not in the teachers, but in the tests."

Look What She Starr-ted! In today's StarrPoints, columnist Linda Starr says that standardized tests can be an accurate indication of teacher quality. What do you think? Should teachers be evaluated based on their students' performance on standardized tests? Share your reflections on a StarrPoints message board. Linda Starr, a former teacher and the mother of four children, has been an education writer for more than a decade. Starr is the curriculum and technology editor for Education World. Previous StarrPoints

That line prompted a response from David N. Shearon, executive director of the Tennessee Commission on Continuing Legal Education and Specialization and of the Tennessee Lawyers' Fund for Client Protection. Shearon, who served on the Nashville (Tennessee) School Board from 1998 to 2001, said, "The problem is not in the tests ... it is in the analysis of the scores."

In Tennessee, according to Shearon, every student in grades 3-8 is tested annually in five subjects using a commercial standardized test. High school students take end-of-course tests in several subjects, and every high school student takes ACT, the Scholastic Aptitude Test (SAT), or the WorkKeys test.

In Tennessee, however, unlike in many other states, neither the students nor the teachers are evaluated simply on the tests' raw scores. Each student's score also is analyzed in terms of year-to-year gains and compared to test norms. That comparison makes it possible to tie student performance gains to the effectiveness of a particular teacher.

The results of Tennessee's analysis, Shearon said, "have been overwhelming."

• Math scores of fifth-grade students assigned for three years in a row to the most effective teachers averaged in the 85th to 95th percentiles, and math scores of fifth-grade students assigned for three years in a row to the least effective teachers averaged in the 35th to 45th percentiles.
• Students who scored in the bottom quarter of state math exams in fourth grade had a 62 percent chance of passing the math portion of the high school competency test if they had the most effective math teachers in grades 5-8 but only a 16 percent chance of passing if they had four straight years of the least effective teachers.
• Students who scored in the top quarter of the sixth-grade math exam averaged anywhere from 19 to 26 on the high school ACT math test; the variations correlated with the effectiveness scores of their high school math teachers.

William L. Sanders, director of the Value-Added Research and Assessment Center at the University of Tennessee at Knoxville, corroborates Shearon's observations. According to Sanders's analysis, "On average, the least effective teachers produce gains of about 14 percentile points among low-achieving students; the most effective teachers posted gains that averaged 53 percentile points. High-achieving students gain an average of only 2 points when taught by the least effective teachers but an average of 25 points when taught by the most effective teachers. Middle achievers gain a mere 10 points with the least effective [teachers] and in the mid-30s with the most effective."

Other states have reported similar results. Researchers for the Dallas Independent School District, for example, studied the correlation between teacher effectiveness and student performance on formal assessments. They found that

• the average reading scores of students assigned to three highly effective teachers in a row rose from the 59th percentile in fourth grade to the 76th percentile by the end of sixth grade, and students of similar ability assigned to ineffective teachers for three consecutive years fell from the 60th percentile in fourth grade to the 42nd percentile by the end of sixth grade.
• the average math scores of students assigned to three highly effective teachers in a row rose from the 55th percentile in third grade to the 76th percentile by the end of fifth grade. The scores of students of similar ability assigned to ineffective teachers fell from the 57th percentile in third grade to the 27th percentile in fifth grade. Students of similar ability and performance in third grade, therefore, were separated by nearly 50 percentile points just three years later.

"Teaching," Shearon said, "is critical."

Apparently, effective teaching is also measurable. Although I still maintain that a test is only as valid as the standards it's based on, test results of the magnitude described above make it clear that standards-based test results can provide a strong indication of teacher effectiveness. They also make it clear that teacher effectiveness is not just the most important factor in student achievement; it is the factor that can overcome most other impediments to learning.

Standardized tests, of course, are not the answer to the question of how to increase student performance. They may, however, -- properly developed and analyzed -- be a tool that can help us find the answer.

"Urge school and political leaders to make sure they are analyzing gains, not just raw scores," Shearon said, "then forget about the scores! Focus on effective teaching. The scores will take care of themselves."