Tests Are Broken and We Shouldn't Fix Them

In 1969, Bowdoin College became the first school in the United States to give applicants the option of omitting test scores from their applications. In the fall of 2021, more than half of the country’s four-year colleges will do the same.

The test-optional movement has gained considerable momentum over the past few years in particular. Schools around the country are scrambling to recover from a litany of recent admissions-related issues: staggering student inequity, high-exposure admissions scandals, enrollment decline, soaring tuition costs — and the list goes on.

In some ways, the switch to test-optional successfully addresses that dysfunction. Test-optional schools report higher diversity among admitted students and equal (or higher, at some schools) graduation rates. The switch also helps mitigate self-selection by encouraging more teens to apply to a wider variety of schools.

"I looked at the average test scores of colleges, and if my scores didn't fit that range, I just nixed them. That was the first time that I was presented with the idea that SATs could be optional. I didn't know. I would never have thought to apply to liberal arts schools in the Northeast." —The Test and the Art of Thinking (Davis, 2018)

To be clear, the profound crisis in higher education demands more than just going test-optional. But the movement does indicate a growing recognition that the way we evaluate our kids is faltering and needs to change.

The world desperately needs to overhaul how we measure talent and potential. Tests are falling short, and the solution isn’t to rewrite, update, or adapt them for incremental change. We need new methods that leverage the 21st century’s powerful tools to tackle its equally powerful challenges.

Tests Are Broken and We Shouldn't Fix Them

One of the most widely criticized aspects of standardized tests is that they reward socioeconomic status (SES), leading to skewed opportunity access along the lines of race and class. We couldn’t agree more: resolving testing inequity is an urgent priority. But that bias is a symptom, not a source. To make lasting structural change, we must uncover tests’ deeper, inalienable design flaw.

Stringently timed, high-stake tests have an adverse impact against racial minorities, women, those with low socio-economic status, non-native speakers of English, older applicants, and people with disabilities. —Ruth Colker, law professor and national expert on discrimination

Tests are susceptible to “hacks,” a flaw articulated by author and computer scientist Paul Graham. In theory, tests measure raw merit: everyone sits down with the same paper, study materials, and #2 pencil to prove who knows the most, who works the fastest, and who thinks the best. But in practice, you can rig your score without touching what they purport to measure.

Bluntly, test scores reflect race and income. Scores increase remarkably among white, Asian, and wealthy test-takers, thanks to advantages such as the following:

  • Spare time
  • Access to coaching
  • Funds to retake the test
  • Familiarity with models of success
  • Previous experience in high-stakes environments

If we want to level the playing field, we need to do away with measures designed to favor the few, not tweak them bit by bit. The same could be said about other hackable measures, like GPAs, personal essays, and letters of recommendation; they let advantaged students reinforce their leg up by “proving” their talent and potential. This sets the precedent for a trend toward individualism. In school, we’re taught the pernicious idea that making a great case for your accomplishments is more important than the accomplishments themselves. When certain groups have better tools to make a great case, they’re given more opportunities. The rich get richer, the poor get poorer, and actual change becomes an afterthought.  

This opportunity gap isn’t new. In fact, the knot that ties together race, wealth, and test scores hasn’t loosened in more than 50 years. If we step back to get a global view, the picture is even bleaker: according to meta-research, the global SES opportunity gap between the wealthy and the poor is widening each year.

The fallibility of standardized tests and similar assessments has severe consequences for socioeconomic mobility across students’ entire lifetimes. These paper-thin methods determine whether a 17-year-old will be able to attend college, one of the clearest (and few) pathways out of poverty. If pursuing upward social mobility is climbing a steep and narrow path, a high-stakes assessment is like a locked gate at the trailhead. But imagine building an escalator instead.

A product of history: When (and why) we started testing our kids

Tests feel inevitable because they’re sewn deep into the fabric of our lives, but history tells another story. In the United States, for example, standardized tests emerged in the 19th century to solve problems of the time. Those problems have faded away; tests should, too.

Written, standardized tests came bundled in a “factory model of education,” a shorthand phrase to describe the American approach to schooling post-Industrial Revolution. Before, schooling in the United States was largely parochial. That meant students were educated by their local churches, which espoused regional, ethnic, and religious fealty. Once industrialization reached the United States, the country began remodeling its education system after Prussian education reforms made to sustain a booming manufacturing economy.

American public schools were instated, featuring standardized, secular classrooms. The aim was to instill a strong sense of national identity and produce better (i.e., “punctual, docile, and sober,” according to economic historian and professor Joel Mokyr) factory workers.

Standardized testing was also implemented as a sort of scalable quality control. It was a simple, repeatable way to demonstrate that state-funded schools were producing a quality product: young laborers. Tests also provided an “objective” ranking of students, which was a useful tool to sort young people into defined education, military, and labor tracks.

This educational approach continues to be the basis for our schools today. Of course, we’ve made improvements to the model, with additions like group work, project-based learning, peer-to-peer instruction, and learning through play. These updates are undeniably positive; they chip away at the rigid emphasis on results, which is mission-critical for educational progress.

However, the factory model remains a heavy anchor to our past, a problem that echoes internationally. Around the globe, school systems are tangled in similar histories and struggle to make meaningful progress beyond their own high-stakes evaluations. Incremental improvements aren’t keeping up with today’s rapid innovation, and our students’ education and futures are falling through that gap.

Two hundred and fifty years ago, the Industrial Revolution triggered an upheaval of education and testing, making schools unrecognizable from their parochial predecessors. Today, technology is driving us into the Fourth Industrial Revolution, generating new economies, tools, and challenges. History set a precedent for new educational models. We owe it to our children to build them.

The next (r)evolution: 21st-century problems need 21st-century solutions

We’re already years deep into an era of innovation. Take a look around: our world is powered by mass digitization, artificial intelligence, hyperconnectedness, and long lists of innovations that echo the science fiction of yesteryear. There are two major takeaways. First, standardized testing is outdated. Second, and this is crucial, we have more tools than ever before to discover and develop talent.

One of the original functions of standardized tests was to funnel teens into distinct career tracks. In the context of our current job market, tests don’t fulfill that promise. Technological developments have made soft skills (such as learning ability, creativity, communication, and collaboration) more valuable than technical skills in the job market. Three-quarters of employers agree that identifying candidates with those skills is difficult. We try to evaluate those skills in written tests, but, unsurprisingly, those efforts aren’t sufficient.

Global development is only going to continue to outgrow tests. According to IBM, more than 120 million workers in the global workforce will need to be retrained by 2022 because of automation. Another prediction claims that over one-quarter (27%) of available jobs in 2022 don’t yet exist. By 2030, the job market will be so unrecognizable that estimations jump to a staggering 85%. Written tests measure rote memorization under time constraints, but jobs that use those skills will continue to be done by robots, not graduating students.

The founder and executive chairman of the World Economic forum urges leaders to “together shape a future that works for all by putting people first, empowering them and constantly reminding ourselves that all of these new technologies are first and foremost tools made by people for people.”

We must embrace ways to identify individuals with the potential for creativity, design thinking, and social good. Those skills will power our future economy, but, more urgently, they stand between us and an unstable future. Solutions to breakneck globalization, wealth disparity, and the climate crisis might come from someone who has perfect SAT scores and a flawless GPA on their record. But the stakes are too high for us to overlook kids who don’t.

Fortunately, these unprecedented challenges are matched by unprecedented innovations in technology. In 2020, there are an estimated 3.5 billion smartphone users across the world, and an estimated 3.6 billion people tapped into social media. Generation Z, the first generation born into an internet-enabled world, also displays a unique commitment to equity, social good, and ethical consumption.

These factors give us new opportunities for insight into what kids are already doing and what they are capable of. Before, our best methods to discover talent and distribute opportunities had close ties to wealth, access, and networks. Technology, on the other hand, has the potential to be a hugely democratizing force. It provides new opportunities to reveal creativity and collaboration, and it provides a platform to share some pretty incredible stories that never get told on a transcript or a Scantron.

New alternatives to discover and develop talent

More than half of four-year colleges will offer a test-optional admissions process in the fall of 2021. Ten years from now, what will they use instead?

A few guiding principles emerge in contrast to the old approach:

  • Minimize prep time. Kids should be celebrated for how they think, not how well they cram for a high-stakes test.
  • Make daily life visible. Tests take kids out of their routine for evaluation, but the real signs of talent and potential  are scattered throughout the day-to-day.
  • Make creative work visible. Technology provides a whole new stage for kids to show off creative work. Whenever they do, it’s our job to notice.

The possibilities for testing alternatives are exhilarating. Could we identify outstanding thinking like we discover athletic and musical talent? The skills of young athletes and musicians aren’t quantified on a high-stakes test; their ability to perform speaks for itself. What could peer evaluations reveal about young people that transcripts and test scores don’t? What if kids could use TikTok, Snapchat, YouTube, and Instagram to show off their interests and creativity?

We don’t need to wait for institutional change to transform evaluations. At Hello World, we’ve set out to answer some of these questions. Specifically, we’re using smartphones and video to identify and nurture overlooked talent around the globe. We’ve already connected with some incredible kids, and we have big dreams for what’s to come.

If you want to see what this looks like in practice, download our app on iOS and Android. We also want to connect with people who believe in our mission to discover and develop the potential of teens. If this is you, say hello! I look forward to meeting you.