Disinterested testing allows us to discern what works from what doesn’t.
In searching for the source of any given problem, we must learn to distinguish actual causes from ostensible ones.
Let’s take randomized trials as an example. Randomized trials place subjects into groups at random and give each group a different solution or treatment to be tested. These trials are effective because they help us to distinguish between a problem’s apparent cause and its actual one.
Many such trials have taken place within foreign development aid. To test the hypothesis that textbooks improve test scores, one Dutch charity in Kenya delivered textbooks to a random selection of schools. Surprisingly, textbooks had very little effect on the students’ scores.
Then, the same charity gave the schoolchildren a treatment against intestinal worms. While worms weren’t expected to have an influence on learning and academic performance, this program was actually more successful than the textbook experiment, because it reduced the number of absences due to illness.
However, certain problems seem impossible to solve using randomized testing. Yet, even then, there are still ways to draw useful insights about a problem’s causes.
These intractable problems are known as fundamentally unidentified questions – or FUQs. Since such problems have a complex mix of causes, it’s very difficult to pinpoint a single cause to test. It may also be impossible to measure such causes until after the fact, like the role of carbon dioxide emissions in climate change.
The same is true in measuring corruption. Take learner drivers in India. Researchers approached several learners, promising some of them cash for passing their test, while subsidizing the lessons of others.
After their test, the researchers surprised the drivers with a second, independent test.
Compared with the learners who’d had their lessons subsidized, those who’d received the cash prize were more likely to have passed their initial test. However, they were less likely to be able to drive sufficiently well in the second test.
Why? It turned out that those learners who won the cash prize had bribed the examiners!
This type of experiment serves as an identification strategy: a way for us to see what can cause FUQs (such as corruption and bribery).