What’s Next for Standardized Testing?

EducationWorld is pleased to present this article contributed by C.M. Rubin. The piece is part of Rubin's online interview series Global Search for Education, in which she joins thought leaders as they explore big-picture education questions. For this series and her other online series "How Will We Read?", she received a 2011 Upton Sinclair award. Rubin is also the author of three bestselling books, including The Real Alice in Wonderland.

What if educators were able to assess the 21st-century skills that thought leaders talk about week after week? What if this 21st-century replacement for the SAT were able to measure high school students’ critical thinking, analytical reasoning, problem solving and written communication—in other words, all the essential skills that both educators and employers have said students need to succeed in college and the workplace?

Until we are able to assess critical thinking, it probably won’t get taught. Program for International Student Assessment (PISA)’s international problem-solving standardized test does assess this, but according to Harvard University’s Dr. Tony Wagner, when it comes to the forward-thinking model for American schools and colleges to watch, “The College and Work Readiness Assessment (CWRA/CLA) is really in a class by itself.”

Roger Benjamin (left) and C.M. Rubin

Wagner sees CWRA as an essential part of what he calls “Accountability 2.0.” But he adds it “should be accompanied by audits of students’ digital portfolios which show evidence of progressive mastery of the skills that matter most, such as critical and creative thinking, communication, and collaboration.”

Naturally, I was curious to learn more about this promising candidate for the gold standard in the future world of assessment. I recently got the opportunity to chat with the President of the New York-based CAE (Council for Aid to Education), home of the Collegiate Learning Assessment (CLA), Dr. Roger Benjamin.

Prior to his role at CAE, Roger was a senior research scientist at RAND from 1990 to 2005 (director of RAND Education from 1994-1999). 

Do you think that standardized tests should be used to evaluate the abilities of high school students for admission to college? Do you think that GPA alone would be sufficient?

I do think there is a role for standardized tests because of the grade inflation in high school GPAs, widely disparate metrics used to evaluate students, and the advantages students from affluent high school districts have compared to students from disadvantaged schools. There are students everywhere in the United States who, when given the chance, demonstrate extraordinary potential. 

Standardized tests are therefore an important additional tool for admissions officers. However, standardized test protocols must become better aligned with the education reform movement underway, including the Common Core State Standards movement, which calls for open-ended tests instead of the heavy reliance on multiple-choice tests, which are not worth teaching to. 

In your own words, would you describe for me what you believe the current SAT measures, versus your own performance assessments?

The SAT measures the aptitude high school seniors have for doing well in college through reasoning and verbal abilities tests. The CWRA measures high school students’ critical thinking, analytical and quantitative reasoning skills, problem solving, writing mechanics and writing persuasiveness skills that educators and employers believe high school graduates need to have to succeed in college and work.

Why is the College and Work Readiness Assessment a better option than the SAT?

Photo courtesy of Tech Valley High School, Rensselaer NY; New Tech Network and Traverse Bay Area Independent School District, Michigan/Mancelona Public Schools/Joanie Moore.

I would replace the word “better” with the word “different.” The CWRA is more congruent with the requirements of today’s “Knowledge Economy,” in which it is more important to be able to access, structure and use information than to only accumulate facts. 

Multiple-choice questions require the ability to recognize a painting. In comparison, performance tasks require the student to paint. Definitions of learning have shifted to "the ability to apply what one knows to new situations." Performance assessments capture this change. We are not sure whether the CWRA+ will compete directly with the SAT or be thought of as additional important information for college admissions officers. However, it is becoming clear that schools, colleges and employers are eager to move beyond reliance only on multiple-choice tests. 

Students learn best when they are being taught to learn as opposed to being taught to pass a test. Can we overcome the problem of teaching to a test? 

A strong point of the performance tasks (case studies and realistic problems) of the CWRA+ and CLA+ is that it is very difficult to teach to these tests. There are not simple “right” or “wrong” answers. The focus is on how well the student reasons, not on whether they get the “facts” right. Indeed, all the facts needed to answer the tasks are provided to the students taking the test. 

The SAT is a multiple-choice test with the exception of the writing test. Grading of the multiple-choice test is easy. The answers are either right or wrong. The CWRA assessment is more about judgment. How do you assure grading consistency across the exams taken in different locations and at different time periods?

Performance assessments such as CWRA have been around a long time. Teachers have not liked multiple-choice tests, but we could never figure out how to take to scale the more complicated problem-solving assessments that constitute the CWRA. However, with scientifically designed rubrics, we can now train human scorers to score open-ended essays just as reliably as they score multiple-choice tests. Moreover, computer-assisted scoring, built on our human scoring protocols, score these responses at levels that are as reliable as those for human scoring, which further reduces error and cost.    

What would be your best argument for keeping the SAT?

It is relatively cheap and easy to administer and score. It has a significant number of reliability and validity studies that corroborate its efficacy in predicting a student’s GPA in the first year of college.

You speak a lot about measuring critical thinking skills. How much do these new CWRA assessments require students to have honed their creative skills?

That’s a good question. We know that students in the arts and sciences at the college level do better than students who are in vocational or applied subjects. We think that is probably because they do more analytic-based writing and are involved in more open Q & A sessions. However, there is a lot of work to be done on how creativity plays into doing well on assessments like this. So that is an interesting question. 

Do you believe that strong creative skills are important to the process of identifying problems and finding solutions?

There are important soft skills such as creativity and collaboration, but the challenge is how to measure them at the same level of scientific reliability as the skills that we are currently measuring reliably. Hopefully we will succeed in measuring these additional skills that are also very important.

How many schools nationally are using the CWRA performance assessment, and how many college admissions offices are asking for it? How many will accept it instead of the SAT?

This year, 120 high schools are using it. We are now talking to college admissions officers and leaders of colleges who are aligned with high schools that feed into them, and are also CWRA+ users, about accepting the CWRA+ results in addition to SAT or ACT results. There is much controversy in education at present about standardized testing, with many believing all standardized testing should be eliminated during primary and secondary school, with just one test given at the end of secondary school. 

What are the logistical issues you face in getting broader distribution for the CWRA assessment?

We now have a new version of CWRA called CWRA+ to be used in the college admissions space. We have started to market this new assessment through partnerships with colleges that will accept the CWRA+ results as part of admissions records. The Obama administration's Race to the Top program launched the biggest testing educational initiative ever funded by the federal government, and we won the right to develop a number of assessments. 

These newer assessments are becoming mainstream almost overnight. It’s because of the “Knowledge Economy” – content is important, but when you can Google for facts, you’ve got to be able to think about what it is you’re going to be Googling for. Multiple-choice tests have been the dominant type of test in this country since World War I, but that is about to change.

Education World®    
Copyright © 2013 Education World