RAND Report Questions Texas Test Gains

Reports about education come and go -- but they normally don't find their way onto the front pages of nearly every daily newspaper. Nor do such reports lead both morning and evening television network news shows. This week, however, an issue paper released by RAND, an independent think tank headquartered in Santa Monica, California, made headlines. The report took aim at the validity of standardized-test score gains in Texas. When Texas takes a hit, so does Republican presidential candidate, Texas Governor George W. Bush.

As one of recent history's closest presidential races nears its conclusion -- polls are too close to call, and the Oval Office is still very much up for grabs -- political analysts say that every vote will matter. A report from a reputable nonprofit research organization questioning the academic gains boasted by Governor Bush could make a difference inside the voting booth. The campaign teams for both candidates know it.

Staffers for the Bush campaign say the report isn't fair or accurate; the Gore camp trumpets its education platform; and teachers unions and other education experts say "I told you so."

MINORITY GAINS INFLATED

The RAND paper that caused this week's media frenzy, What Do Test Scores in Texas Tell Us?, is the result of an investigation into whether the dramatic gains in math and reading scores on the Texas Assessment of Academic Skills (TAAS) represented actual academic progress. In particular, the researchers focused on the achievement gap between whites and minority students, particularly African American and Hispanic students. The researchers found large discrepancies between the TAAS scores and scores by Texas students on the National Assessment of Education Progress (NAEP). The NAEP is a congressionally mandated test, commonly known as "the Nation's Report Card," that assesses what U.S. students know and can do.

According to the RAND paper, Texas NAEP results showed very large achievement gaps between whites and students of color and showed that those gaps are increasing slightly. The TAAS scores tell a different story. Those scores indicate the gap is much smaller and is decreasing greatly.

RAND cites several reasons for the high TAAS scores. Those reasons include the following:

• Teachers are teaching to the test.
• Higher dropout rates among minority students, estimated by the National Education Association to be about 50 percent, may inflate test scores.
• Students who "top out" (get nearly perfect scores) artificially narrow the gap between whites and students of color by raising the score averages.
• There is an increase in the number of students with disabilities exempt from the TAAS.
• Scores generally improve in subsequent testing years because students practice how to answer the specific types of questions that appear on the yearly TAAS.

DANGERS WITH HIGH-STAKES TESTS

Although the RAND issue paper is critical of Texas, it also criticizes high-stakes testing in general. The authors suggest that other states learn from "the danger of relying on statewide test scores as the sole measure of student achievement when these scores are used to make high-stakes decisions about teachers and schools as well as students."

A key point in the RAND report is that although TAAS scores did rise in Texas, it was because teachers teach to the test. That practice could hurt kids in the long run, the authors warned.

The RAND report doesn't surprise another independent research and public policy institution. The Applied Research Center (ARC) in California studied Texas testing last year. "It's what we suspected all along," said Terry Keleher, senior researcher at ARC. "We are very concerned with the way the data in Texas is manipulated.

"There is a significant racial disparity, and it's pretty alarming," Keleher continued. "About 85 percent of Mexican American and African American students, who make up nearly 40 percent of Texas high school seniors, could not get their diplomas even though they had passing grades. That is a real injustice."

Keleher agrees with RAND that the emphasis on high-stakes tests hurts students because they lose gains in higher-order learning and more-complex reading and writing development. "The major candidates do not have an effective position on high-stakes testing. These kinds of high-stakes tests can be unsound academically, unfair, and actually racist."

TEXAS TEACHERS RESPOND

That's not news to some teachers in Texas. "Teachers don't have any choice but to teach to the test in Texas," said Annette Cootes, spokeswoman for the Texas State Teachers Association, an NEA affiliate. Scores are published in the newspapers and then the papers line up the scores, comparing one neighborhood school to the next, she said. That puts a lot of pressure on the teachers and the students.

All teachers are required to drill their students for the test, Cootes said. Teachers complain that they lose teachable moments. "You live by the TAAS. You die by the TAAS. But the TAAS is not the answer. We have felt that for so long because we are shortchanging the kids. They aren't getting a broad education.

"Teachers feel that no one listens to them down here," Cootes added. "The Texas Education Commission office has been a fairly unresponsive, lumbering giant for more than 20 years."

TEACHING THE CURRICULUM, NOT TO THE TEST

"Our state test is based on our curriculum, so we should not be surprised that teachers are teaching the curriculum," said Debbie Graves Ratcliffe, senior director of communication for the Texas Education Agency. "The teaching and the test are going to match. Some schools do too much on drill and review. Mindless drill is not necessary."

In response to the criticism that children are losing out on a broad education, Ratcliffe said Governor Bush and the education commissioner, Jim Nelson, both believe schools must focus on reading because children who can't read struggle in other areas. The Texas legislature enacted a law, however, that will require tests to include science and social studies in three years.

As for the high minority dropout rates -- according to Texas figures, about 20 percent of minority students drop out, not 50 percent. Ratcliffe said that Texas does not include students who earn GEDs as dropouts, which lowers the dropout rate.

Another reason academic gains were inflated was that the number of exemptions for special education students rose, according to the RAND issue paper, which Ratcliffe confirmed. "When the legislature began counting special education students' scores and including them for school accountability, the number of exemptions for special education students went up." In 1998, before those scores were calculated for school accountability, 62.7 percent of special education students took the TAAS. In 1999, when their scores counted, the number of special education students taking the test dropped to 47.2 percent.

Ratcliffe pointed out, however, that the RAND issue paper cites researchers opposed to high-stakes testing, which taints the objectivity of the report.

Texas commissioner of education Jim Nelson agreed with Ratcliffe that the RAND issue paper is tainted, calling it "shoddy research." Nelson cited a previous RAND report, released last July, that said Texas was a top performer compared to most states.

RAND stands behind the July report too. That 271-page report, Improving Student Achievement: What NAEP Test Scores Tell Us, which took three years to complete, placed Texas at the top of the heap nationally for gains on the NAEP.

Read about that report in an article from the Education World archives. See Does High Teacher Pay = High Achievement? A Study Says No!

David Grissmer, lead author of the RAND report on improving student achievement, named Texas and Indiana as leading performers in student achievement gains. RAND compared students in those states with students from similar racial and socioeconomic backgrounds in 42 other states. Grissmer credited improved teaching conditions, reduction in class sizes, increased pre-kindergarten enrollment, and more teacher discretion in resources as the primary reasons for the improvement in the scores.

The findings in the main report and the issue paper are so different because the main report differed in scope, methodology, and focus, said James A. Thomson, RAND president and CEO. It sought to explain why student achievement scores vary so widely across the states even after demographic adjustments are made. "The team that researched the new issue paper, on the other hand, focused on Texas and its statewide testing program," Thomson said in a written statement. Texas was studied in the issue paper because the state exemplifies a national trend toward using statewide exams as a basis for high-stakes educational decisions. RAND says it is scrupulously nonpolitical.

That isn't how the Bush campaign sees the RAND issue paper. They say the report is unfair and accuse the lead researcher, Stephen P. Klein, of being politically motivated for releasing the report only two weeks before Election Day. Margaret LaMontagne, Governor Bush's senior education adviser, criticized Klein for not requesting the complete testing data from the Texas Education Agency for his study instead of using data provided on the agency's Web site.

The Gore camp jumped at the opportunity to take advantage of these most recent RAND findings, calling the "Texas Miracle" a myth. "The same independent group Bush once pointed to as proof that he has done an effective job on education has now devastated his claim," said Dagoberto Vega, a spokesman for Gore. "Without a record to stand on or a real education agenda, Bush is essentially turning in an incomplete assignment to voters."

However, both candidates' platforms continue to promote standardized tests as a way to hold schools accountable for their performance.

Diane Weaver Dunne
Education World®
Copyright © 2000 Education World

Related Articles from Education World

• Before You Pull That Lever ...

Please check out our featured theme this week: