A Review of:
Broussard, M. (2018) Artificial Unintelligence: How computers misunderstand the world, The MIT Press, Cambridge, Mass.
Smith, G. (2018) The AI Delusion. Oxford University Press, Oxford.
The purpose here is to highlight for readers a rich, correcting literature that runs parallel with the ‘AI’ definitional market mayhem we encounter daily. By now most commentators see AI as a term that can be used without much definition. We all know what it means—developing machines that are already taking over cognitive human work, and might well eventually replace much of the work humans use their brains for. I have seen the phrase used for technologies and uses that are definitely not AI—robotic process automation (RPA) for example. Meaning inflation has its vested interests, of course, and makes us lazy with words. AI is used as if we are on the edge of achieving ‘strong’ AI with general intelligence capability. What we really have at the moment—outside the military, government and very few big tech companies—is at best ‘narrow’ or what I have called ‘weak, weak’ AI.
This has been regularly pointed out by more discriminating observers. Vendors, consultants and marketeers have responded by inventing the term ‘intelligent automation’ (IA). This is used to cover RPA coupled with more cognitive tools including image processing, natural language programming, machine learning, deep learning, neural networks, algorithms, all backed by impressive developments in memory and computing power. For good measure ‘AI’ is also often included as part of IA—just in case, I guess. Incidentally, none of these technologies as currently used exhibit intelligence in any normal meaning of the term—but that is about par for this particular guff course!
How to counter the avalanche of ‘AI’ guff? Here we focus on just two books that will lay the ground for future reviews. Meredith Broussard understands technology and shows that there are fundamental limits to what we can and should do with the technology. Gary Smith understands statistics and demonstrates that, as so much of today’s cognitive computing relies on statistical method, poor statistical proficiency can lead to an AI delusion—the results, often devastatingly, are not what we think they are. Both viewpoints lead to a fundamental questioning of that often posited, seamless, accelerating glide to omnipotence of what are in fact our own, and therefore all too imperfect, ‘AI’ creations.
Brussard is very good on how computers and AI misunderstand the world, and how we misunderstand AI. In fact, these technologies do not actually understand anything. Brussard points out the heavy anthropomorphism in the language we use to describe the capabilities of these machines. She herself has an eye for the telling phrase. General AI is the Hollywood kind of AI. ‘Narrow’ AI is statistics on steroids. She quotes Zachary Lipton on AI technologies: ‘they are no more sentient than a bowl of noodles.’
Machine learning is a metaphor only. The machine can become more accurate at performing a single task according to a specific metric a person has defined, but it does not acquire knowledge, wisdom or agency. The dirty secret of big data is that all data is dirty. Data is socially constructed. The numbers camouflage important social and cultural contexts. Narrow AI involves quantitative prediction. It gives the most likely answer that involves a number. But not everything is a calculation, and calculation is not consciousness. Not everything that counts is counted. Machines cannot identify quality nor do they have the cognitive capacity to reason about the future—a core limitation of AI.
A great strength of the book is that she underpins these position statements by accumulating detailed evidence on how computers and developers work, and how programs and algorithms are formulated. This is very much an insider view. Developers cannot predict every situation, data does not arrive neutral, the world changes, the best we can do is create a necessarily simplistic model of how things seem to work, and how decisions should be made. The results are not always impressive, and she devotes four chapters to examples when computers don’t work well, with case studies on why poor schools cannot win at standardised tests, people and machine learning problems, and why the car won’t drive itself. She is convincing on the engendered history of computing, and how this passes into biases into data collection, analysis and use. Broussard also reveals the potential deficiencies in statistical practice, an illuminating example (and well worth reading) being her detailed analysis of survival rates in the Titanic disaster. It is clear from this that the computer can only analyse what is input—a real problem when life and death depend on the resulting analysis—as with ‘self-driving’ cars, for example.
Her solution is greater recognition of software and ‘AI’ development as a craft, and the application of knowledge and experience this implies. Also, a jettisoning of so much of the misleading terminology that leads us to see machines as intelligent, sentient, learning and able to replicate and surpass the workings of the human brain. Not surprisingly, Broussard recommends using human-in-the-loop systems: audit algorithms, watch out for inequalities, consciously reduce bias in computational systems. It’s a never-ending task. Above all, avoid magical thinking about technology.
Gary Smith also addresses magical thinking—this time about the power of mathematics and statistics when utilised in computer systems and ‘AI” contexts. Like technology, numbers have a mystique we can be in awe of, not least because much of the time we do not understand how they are really arrived at. They also invoke some things we crave for—precision, certainty, objectivity, absolute value in a world experienced as dynamic, risky, relative and ambiguous. But of course, as Gary Smith demonstrates, much of this projection on to numbers is wish fulfilment. Hidden in the ‘AI’ are fragile, humanly-constructed data and statistical processes marked by frequently recurring deficiencies. The strength of Smith’s book is to make these explicit, taking in many illustrative examples along the way.
Smith’s early chapters support Broussard’s picture of the ‘AI’ world. Computers are great at many things including record keeping, information retrieval, number crunching and fast processing. But they are essentially unfailing rule followers. They do not read, feel, think, understand. They are not intelligent. As an example, in playing Jeopardy!, IBM Watson found words but did not understand them, nor ideas and concepts. The computer’s forte is doing without thinking. Computers are poor at taking time into account, crafting theories to explain and understand the world, emotions, critical thinking. A computer program does not understand, in any human sense, what it is doing, why it is doing it, or what the consequences are of doing it. For Smith, in the age of Big Data, the realistic danger is not that computers are smarter than us, but that we think they are smarter, and so let them make decisions, and tell us what to do.
Data, especially Big Data, as symbols without context, are particularly problematic. Computers do not have real world experience to guide them, so rely on incoming data, a digital database, pre-set rules and algorithms to find statistical patterns. This is helpful, but also fallible. Big Data is not always better data. Indeed, can a computer recognise bad or incomplete data? There is self-selection bias. One study found traffic fatalities were higher in 50 mph zones than 75 mph zones. Conclusion: raise speed limits? Well no, there is a statistical problem. The State surely chose to put 75 mph limits on the safest highways (straight, well lit, with light traffic). Famously, correlation is not causation. Does factor A lead to factor B or vice versa, or is a third or fourth factor more determining? What difference does time make? If marriage increases and beer consumption increases over time, which influences which, if at all! Spurious correlations are everywhere. Humans are sometimes fooled, but computers are always fooled. Survivor bias relates to not seeing things that no longer exist. Allied warplanes returned from bombing missions with holes in most places except the cockpit, engines and fuel tanks. Where should additional protective plating be placed? Well the cockpit, engines and fuel tanks. (Work it out).
For all the publicity celebrating Big Data, a small amount of relevant data can be more useful than a mass of obsolete or irrelevant data. It can be more productive to collect good, representative data focused on the questions a study is intended to answer. There is a misperception that data patterns uncovered by computers/AI must be meaningful. With Big Data, massive data sets and a loading of more and more variables, patterns and correlations are inevitable. But the bigger the data, the more likely it is that a discovered pattern is meaningless. Smith particularly dislikes statistically abusive data mining practices. Data mining algorithms are programmed to look for trends, correlations and other patterns in data. A theory is then invented to explain interesting patterns, or the data is left ‘to speak for itself’. Unfortunately, the numbers cannot speak for themselves. This is not helped when a black box approach is taken where input is fed into an algorithm, which provides output without human users knowing how the output was determined. Smith concludes that data without theory is a treacherous philosophy. Ask why, not what.
Yes, data is very problematic indeed, and if you torture the data statistically for long enough, as Ronald Coase said, it will confess—you can get results, even the ones you need. But beware some typical errors. Smith details all too many. I was struck particularly by ‘The Texas Sharpshooter’ fallacy. This has two variants. In one the sharpshooter (statistician) puts up a range of targets, fires a gun randomly, then points to the targets hit successfully, omitting the ones that were missed. In the second, our sharpshooter shoots a bullet into a blank wall, then draws a target with the bullet at its centre. While the results of such operations are likely statistically ‘significant’ they make for dubiously useful information, and the exercises are rarely, if ever reproducible, undermining the scientific basis of such ‘research’.
Then there is the role of noise. Smith describes a study that used an MRI machine to study the brain activity of a salmon as it was shown photographs and asked questions. A sophisticated statistical analysis found some clear patterns. The punchline is that the salmon was dead. With so many voxels, the MRI recorded some random noise, which might be interpreted as the salmon’s reaction to the photos and questions. The reality is we can always find patterns, even in randomly generated data, if we look hard enough. Seek and you shall find! Smith concludes that the Dead Salmon study is wonderfully analogous to someone data mining Big Data, except that Big Data contains far more data, and can yield far more ludicrous relationships.
None of this would matter greatly if the results were restricted to largely unread and unapplied articles in esoteric academic journals. But even there we now hear much talk about a reproducibility crisis across disciplines. More seriously, Smith draws his illustrations of ‘AI’ statistical fallacies from a wide range of applications, from healthcare, stock trading, autonomous car design, insurance, personnel selection, to predicting presidential elections. What we are now doing with ‘AI’ and statistics can have very profound real-world consequences.
All this becomes intensely worrying. We are increasingly placing our faith in problematic technologies and data processing techniques, precisely when we are becoming much more dependent on them for decision-making and how we run our work and social lives. The convenience—speed, answers, predictions—too often wins out against a more sceptical and cautious approach towards what we engage with, and are being told and sold. But, as Broussard says, if it’s artificial it’s not intelligent, and if it’s intelligent it’s not artificial. Ultimately it is our intelligence that must be applied to making these machines and methods work for us. How well do you think we are doing?