High-Stakes Testing: Gaming the System or Getting It Right?

Preschool teacher Melissa Preece is blunt when asked about standardized testing.

“[It] felt like punishment,” she says about her own experience in Austin-area schools.

During elementary school, she remembers becoming so exhausted during one test that she started doodling on the exam page. In middle school, she recalls feeling pressured by the principal during an all-school rally to do well on the test so that the campus would be recognized as a top school.

“Normal lessons were pushed aside so we could be prompted on how to take the test,” she says. “We were told that if we reached a certain score, we would be treated to a pizza party. The school seemed to be more interested in status than my education.”

Ms. Preece graduated in 1992 — before the No Child Left Behind Act ramped up the intensity of standardized testing. In fact, her experience seems almost quaint compared with what kids today face.

And as an educator herself now, that’s what has her so concerned.

***

The No Child Left Behind Act, which was passed in 2002, requires schools to demonstrate adequate yearly progress in order to receive federal funding for low-income students. Schools that fail to meet adequate yearly progress for five years can be shut down — which happened this past July with Pearce Middle School in Austin. The Act requires schools to administer standardized tests each year to measure student progress. Each state handles this differently. In Texas, the accountability system evaluates each school by the percentage of its students who pass the state tests, according to Jennifer Booher Jennings, a doctoral researcher at Columbia University who studies high-stakes testing. “Imagine that students must answer 70% of the questions correctly to pass. Schools get no credit for moving a student from a 15 to a 69, or from a 70 to a 95. Yet if educators nudge a student from a 69 to a 71, the school's passing rate increases,” she explains.

Given this, the stakes for schools are enormous—and so are the ways to game the system.

Chasing the Bubble

Given the pressure to perform, it’s no surprise that many educators aim for low-hanging fruit: focusing their attention on those students closest to passing the test, says Ms. Jennings. Dubbed “bubble kids”—because their scores put them on the “bubble”, or cusp, of passing—these students give schools the biggest bang for the buck, test-wise. So, it’s no surprise that schools expend most of their resources on them, to the detriment of both low-performing and high-performing students, says Dr. David Hursh, a professor of teaching and curriculum at the Warner School of Education at the University of Rochester and author of the book High-Stakes Testing and the Decline of Teaching and Learning.

“Because teachers and administrators reason that getting a student who is close to passing a test up to the passing level, or keeping a student who is just above the passing level at that level, has the best payoff for the school or district, students who are the furthest behind are likely to be slighted or ignored completely,” Dr. Hursh tells Parent:Wise. “High-stakes testing results in leaving some children further behind.”

Researchers who study high-stakes standardized testing seem ambivalent about its effectiveness. In 2002, Dr. David Berliner and his colleague Dr. Audrey Amrein, both of Arizona State University, examined 18 states that employ “severe consequences” for non-passage of standardized tests, including Texas. In the resulting study, High-Stakes Testing, Uncertainty, and Student Learning, the two concluded that, while test scores may improve, the increase can be attributed to test preparation and the exclusion of certain students — not to genuine learning. What’s more, they further discovered that overall student learning also didn’t improve once high-stakes testing was implemented: at best it remained the same; usually, it got worse.

This conclusion seems to buoy the argument, proffered by testing opponents, that high-stakes testing encourages teachers to “teach to the test”. Dr. Hursh says there is some validity to the assertion. “Whenever an exam becomes ‘high stakes’,” he says, “it distorts what and how teachers teach.”

Think of it this way, he says: If a teacher is told to prepare students for questions she thinks will be on the test, the most effective way to do it is to target her classroom curriculum, focusing on information she believes will be on the test and eliminating anything that will not. In elementary and middle schools, says Dr. Hursh, this has lead to the reduction or outright elimination of instruction in the arts, sciences, and social studies. A February 2009 Government Accounting Office study seems to bear this out. Although the study found that most schools maintained their arts and music education programs, many relegated such instruction to after-school hours. Overall, the study found that art and music education decreased 7% from 2004-2007, largely due to budget constraints and “Competing demands on instruction time… due to state education agency or school district actions taken to meet NCLBA [No Child Left Behind Act] proficiency standards.”

The GAO report prompted Rep. Lamar Alexander (R-TN), the ranking Republican on the Children & Families Subcommittee of the Senate Committee on Health, Education, Labor, and Pensions, to express concern that schools seemed to be narrowing their educational focus in the wake of federal mandates.

“I hope that Congress will reexamine the role of Washington mandates,” he said in a press statement, “and provide flexibility for state and local leaders to ensure that our students can read and write, understand mathematics, know science, have a knowledge of U.S. history, and also have an appreciation for music.

Fudging the Numbers

For many schools, limiting curriculum and concentrating on “bubble kids” doesn’t always result in higher test scores. When it doesn’t, some schools, desperate not to be sanctioned under No Child Left Behind, resort to fudging the numbers.

In their book Collateral Damage: How High-Stakes Testing Corrupts America's Schools, Dr. Berliner and Dr. Sharon Nichols, an assistant professor of educational psychology at the University of Texas at San Antonio, detail examples of test data manipulation throughout the country — and especially in Texas. In some cases, incorrect answers were erased and replaced by correct ones. Other times, administrators mislabeled information on test sheets and booklets, causing those tests to be thrown out. Coincidentally, this seemed to happen only with the low scoring tests. “From the time the high stakes testing went into effect in Texas, and on into the No Child Left Behind years brought to us by a former Texas governor, we have seen Texas be among the most inventive of the states in producing phony scores,” Dr. Beliner tells Parent:Wise. “At one point, it was among the leaders in the nation in the number of children not counted for testing.”

Perhaps the most blatant, and subsequently well publicized, example of cheating happened at Houston’s Wesley Elementary School in 2004. For nearly a decade, the school received national honors for teaching low-income students how to read. It was quite literally a beacon of scholastic success: the school propelled its district to a $1 million Broad Foundation grant and basked in national glory during an appearance on The Oprah Winfrey show about schools that “defy the odds.” But Wesley wasn't defying the odds at all. A Dec. 31, 2003 investigation by the The Dallas Morning News blew the lid off of the school’s elaborate cheating scam. Ironically, TAKS test results tipped the paper off: In 2003, Wesley's fifth graders performed in the top 10% of students in the state on the reading portion of the TAKS. The very next year, as sixth-graders at Williams Middle School, the same students fell to the bottom 10% in the state. Confronted with these data, Wesley teachers admitted cheating was standard operating procedure.

Although Wesley was the most well known school the paper exposed, it was by no means alone. Other schools had similar testing anomalies. Certainly, such anomalies are possible without cheating, Dr. Berliner says, but Texas had some 400 instances. The situation was so alarming that then-Commissioner of Education Shirley Neeley called a press conference to promise she would take whatever actions were necessary “to maintain the integrity of our testing program”. Despite this, the Texas Education Agency didn’t fully investigate the matter, Dr. Berliner asserts.

“Texas is like the poster child for all the unethical and immoral ways a state can find to increase test scores without increasing genuine learning,” he says. “The whole state looks to be corrupt in terms of its test reports.”

Unequal by Design?

In some ways, it may not be surprising that a school like Wesley, which is dominated by low-income students, would resort to cheating. Historically, low-income, minority, and other high-risk kids have not done well on standardized tests. Some researchers believe that’s because high-stakes test results reflect a student’s socio-economic status and not their learning.

“In every state, in every town, in every city, the zip code predicts how well students will do on high-stakes tests: the wealthier the district, the better the children do on the tests,” says Dr. Dale Johnson, professor of literacy education at Dowling College. “This has nothing to do with intelligence and everything to do with advantages that wealthier children have such as private tutoring, travel experiences, books in the home, and more.”

In his book Stop High-Stakes Testing: An Appeal to America’s Conscience, Dr. Johnson makes a strong case for this socio-economic disparity. He cites one example in which children from wealthier backgrounds did not know what food stamps were: some thought they were the little stickers found on produce. The implication in this example is that children from dissimilar socio-economic backgrounds have somewhat different vocabularies. High stakes tests do not take this into account, says Dr. Johnson: they tend to use words that are stacked in favor of the affluent.

In the late 1990s, the Mexican American Legal Defense and Education Fund, known as MALDEF, attempted to put a stop to this testing disparity. The group filed a lawsuit on behalf of Latino and African American students, alleging the state’s TAAS test violated their rights under the Constitution’s Equal Protection Clause of the 14th Amendment. The federal district court ruled that, while the test did have a disparate impact on minority students, it did not violate the Equal Protection Clause because it was not developed with the intention to discriminate against minority students.

A decade later, minority students still substantially under-perform their Anglo counterparts on Texas’ standardized tests, says David Hinojosa, MALDEF’s staff attorney.

“Virtually no educator disagrees with the notion that all children — regardless of race, color, or ethnicity — have the same capacity to learn, and that given the appropriate guidance, resources and instruction, all children will perform at roughly the same level,” Mr. Hinojosa says. “However, the playing field has never been leveled for minority students.”

Defending Standardized Tests

Despite the consternation over standardized testing, it does have strong defenders. Perhaps the most influential is Dr. Eric Hanushek. In addition to being a distinguished professor at Stanford University’s Hoover Institution, he also chairs the Board of Directors of the National Board for Education Sciences and serves as the chairman of the Executive Committee for the Texas Schools Project and as a research associate of the National Bureau of Economic Research. His research has shown that strong academic accountability systems —most often those that utilize standardized testing— are associated with greater student learning and achievement.

In an interview with Parent:Wise, Dr. Hanushek explained how states that introduced accountability through standardized testing during the 1990s showed greater improvements on the National Assessment of Educational Progress than those that did not introduce such accountability measures.

“Accountability works to focus everybody on achievement,” Dr. Hanushek says.

Yes, this may mean that teachers “teach to the test.” But proponents of standardized testing don’t see this as a bad thing. All of the nation’s schools are supposed to offer a vigorous academic curriculum. Tests administered under the No Child Left Behind Act measure how well that curriculum is being absorbed by students. Given this, proponents assert that standardized testing is especially good for kids from disadvantaged backgrounds because it can pinpoint whether their schools are actually teaching the enriched curriculum mandated by the No Child Left Behind Act.

“Standardized tests promote fairness by providing a level playing field regarding what is tested and the conditions under which the test is administered,” says Dr. Stephen G. Sireci, director of the Center for Educational Assessment at the University of Massachusetts at Amherst “In fact, having these tests in place helps us ensure the teachers will be teaching the material they are supposed to.”

This is good news for parents, he says, not only because it ensures that schools do their jobs, but also because the test can function as a strong advocate for parents to demand better for their children. While a teacher may say a child is doing well in school, a standardized test provides “independent and unbiased information about their child's achievement,” Dr. Sireci says

Debunking Stress

Proponents of standardized tests recognize that testing can be nerve-racking for kids, but they say the traumatic effects have been overblown.

“Sure, high stakes tests can be stressful,” acknowledges Dr. Richard Phelps, author of the book Correcting Fallacies About Educational and Psychological Testing. But, he points out, a completely stress-life is a pretty dull life. Besides, for decades student surveys have shown little or no evidence of student stress — indeed, boredom and a lack of challenge have been more frequent complaints, he contends. Moreover, the high stakes tests we give in North America tend to be low level, he says, so low that virtually any student who makes an effort can eventually pass.

Perhaps the problem the public has with high-stakes testing is one of public relations. Standardized testing has become so politicized that research on its negative effects is 30 times more likely than research on its positive effects, says Dr. Gregory Cizek, a professor of educational measurement and evaluation at the University of North Carolina. Worse, he says, much of the research into the negative effects of testing doesn’t adhere to the scientific method and is riddled with flawed data and analysis — something the public isn’t necessarily aware of. He and other proponents of standardized testing worry that this skewed viewpoint could result in educational policy that isn’t good for kids.

“It is very important that we make policy based on the best available evidence, and not uninformed opinions,” says Dr. Hanushek.

Getting it Right

Even the strongest proponents of standardized testing warn that exams never should be used as a sole measure for determining a child’s promotion to another grade or graduation from high school, or whether a school is making adequate academic progress. Instead, they say, students should undergo multiple, on-going assessments that give feedback on what they know so that educators can adjust their teaching and quickly remedy any lack of understanding.

This is exactly what Texas Rep. Rob Eissler (R-The Woodlands) hopes to accomplish. He is the author of House Bill 3, which the Texas Legislature passed during the 2009 session. The bill, which Gov. Rick Perry signed, will usher in a new accountability system in Texas beginning with the 2010-11 school year: the state no longer will rely solely on high-stakes testing to measure schools’ academic success. Instead, schools will be allowed to earn distinctions in several areas: outstanding academic achievement in the core subject areas; growth in student achievement; closing academic gaps; fine arts; physical education; 21st Century Workforce Development; and second language learning. Further, these distinctions will be awarded using multiple measures of success, not just state assessments.

Rep. Eissler told Parent:Wise that the two most important components of the new accountability system are the focus on postsecondary readiness and the inclusion of individual student growth. The new system will require schools not only to have a certain percentage of students meeting a passing standard or demonstrating significant growth, but also to have a percentage of students meeting a college readiness standard or demonstrating significant growth toward that standard.

“As a state, we need to look beyond minimum skills testing to focus on preparing more students for college and career,” he says. “These changes in our accountability system will make Texas the first state in the nation to require schools to prepare students for life after high school graduation.”

The new law stemmed from conversations Mr. Eissler had with people throughout the state. After talking with parents and teachers, he quickly realized everyone was tired of “unnecessary and unneeded pressure,” as he put it, especially when it came to testing 3rd graders. He was surprised to learn that both little kids and their parents believed taking the TAKS test was a do-or-die moment: if they didn’t pass, they didn’t get to go to 4th grade. The TAKS test was never meant to used that way, he says, so HB3 makes clear that 3rd grade students should be promoted based on a variety of factors, not just performance on a single test”

This likely will be greeted by most parents as good news. Ms. Preece, the preschool teacher, is cautiously optimistic. She strongly believes that standardized tests should not be used the way they currently are. And while HB3 holds promise to remedy that, she remains wary about standardized tests in general.

“I feel standardized test are a waste of time, frustrate students, and do not truly show what a student knows,” she says. “Educators need to come up with a better way to assess what students are retaining and teach well.”