I recently had the opportunity to serve on a jury in a case in which a young man was accused of ‘operating under the influence,’ i.e. driving a car while under the influence of alcohol. There was only one witness, the police officer who arrested the young man. In order to establish that the young man was under the influence of alcohol after pulling him over, the officer had conducted what is known as a field sobriety test, which usually has three components:

1. The horizontal gaze nystagmus test, in which a person’s eyes have to track a moving object.
2. The walk-and-turn test, in which the person walks heel-to-toe for nine steps, pivots on the left foot and walks back heel-to-toe for nine steps.
3. The one-leg stand test, in which the person stands on one leg and counts to 30.

Each test has its criteria for success and failure. For example, you may fail the one-leg stand test if you have to use your arms to balance yourself, or if you put your foot down before counting to 30.

In the jury room, we six jury members found that the prosecutor’s evidence was based almost entirely on the results of the field sobriety test, which the young man failed. We had been instructed by the judge that we should presume innocence and find the man not guilty unless the prosecution could prove ‘beyond reasonable doubt’ that the young man had been driving under the influence of alcohol.

We jury members were not permitted to do any research on the field sobriety test, and none of us had any information about how useful it was, though we knew that it is widely used. Most jurors had doubt about the evidence, but one juror believed we should base our verdict on the result of the field sobriety test.

I found myself in the position of explaining that we cannot assume that tests give us useful information and I used the notions of validity and reliability to explain it. In this case:

1. Is the field sobriety test valid? Does it measure what it purports to measure? (Note that the officer conducted only the walk-and-turn test and the one-leg stand test.) Were there other reasons why the young man – or anyone – might fail these tests other than being under the influence of alcohol? (It was claimed by the defense that the young man had ‘an issue’ with his left leg.)
2. Is the field sobriety test reliable? Can we be sure another officer would have reached the same conclusion based on the young man’s performance on the test?

Given these questions, could we determine ‘beyond reasonable doubt’ that the young man had been under the influence of alcohol?

I am writing about this because in language teaching, we tend to place a lot of trust in the tests we use, whether made by an individual teacher, a program, or an international testing organization. Any really useful test should help us determine whether the test taker has attained certain knowledge or is capable of performing certain functions in the language. Think about any test you know – the TOEFL, TOEIC, IELTS, or those home-grown tests used in your program. To what extent are you able to say ‘beyond reasonable doubt’ that the person taking the test has acquired certain knowledge or skills? We know for sure that plenty of students with high TOEFL scores are not well-prepared linguistically to succeed on an American college campus, and this is a test that has a huge research effort behind it.

So I would ask that you advocate for more evidence than a single test to determine students’ language proficiency. Decision-makers such as college admissions staff should seek multiple sources of evidence that converge on a conclusion, not just a single test. They should have more than one person or entity reviewing students’ language ability. We should maintain a healthy doubt about the tests we use.

What was the verdict in this case? In the end, no matter what the truth was, we could not return a guilty verdict because the state had not proved beyond reasonable doubt that the young man had operated a motor vehicle under the influence of alcohol.

Naturally I rushed to my computer after the trial and googled ‘reliability of the field sobriety test.’ What do you think I found?