Language tests being misused, UK study suggests

Dr Fulcher, and Professor Fred Davidson of the University of Illinois Department of Linguistics, argue that generic tests are being used for multiple purposes because some policy makers go for "cheap and simple solutions" to complex problems, such as immigration.

Their article, entitled Test architecture, test retrofit, which has won this year's Best Paper Award from the International Language Testing Association (ILTA), uses architectural references to illustrate how language tests are being misused.

The researchers will be presented with a check and a plaque at the Language Testing Research Colloquium, being held at the University of Michigan from June 23-25, 2011.

Dr Fulcher, from the University of Leicester School of Education, said: "The use of language tests as instruments of public policy has spread exponentially over the last ten years. Most recently, for example, the UK Border Agency has introduced the use of language testing to reduce the number of students entering the United Kingdom; and a policy of restricting immigration rights to the non-EU spouses of UK citizens who do not speak English is now in place.

"In the world of professional certification language testing is also becoming increasingly important. The press has recently been full of stories about the horrors of allowing professionals to practice without the necessary language and communication skills to function safely. These have ranged from calls to introduce language testing for all medical staff who trained in non-English environments to the standardization of language testing for air traffic controllers and pilots.

"These are all cases of "high stakes" test use situations, where the consequences of poor decision making are serious."

Dr Fulcher cited the example of tests of academic English that are used for spouses: "The barriers to immigration may be set too high, and test use therefore may be a form of discrimination. Similarly, it is hard to draw inferences about a pilot's ability to communicate with air traffic control, or a doctor with a patient, both contexts which require specialist vocabulary and terminology, on the basis of a test designed for first year university students."

Dr Fulcher added that testing is a lucrative industry -- but it is expensive to research and to develop new tests that are fit for purpose.

"Some testing companies encourage the use of a single product for as many uses as they can. Our article explicitly draws attention to the tension between prioritizing test purpose and score interpretability on the one hand, and commercial pressures to increase test volumes, on the other.

"We need to show policy makers and testing agencies that the consequences of their decisions are potentially very serious. It is possible for someone to be certified on the basis of a generic test and then fail in their real world tasks -- like an air disaster because the air traffic controller could not communicate clearly enough to keep two aircraft apart in an emergency. Many such language related instances are on record."