The machine stops, a cautionary tale

The Machine Stops is my favorite work of science fiction. Written in 1909, it’s more relevant today than ever before. Thousands of years in the future, human civilization is dependent on a machine designed and built generations earlier. The complexity of the machine is beyond anyone’s complete comprehension:

Year by year [the machine] was served with increased efficiency and decreased intelligence. The better a man knew his own duties upon it, the less he understood the duties of his neighbour, and in all the world there was not one who understood the monster as a whole. Those master brains had perished. They had left full directions, it is true, and their successors had each of them mastered a portion of those directions. But Humanity, in its desire for comfort, had over-reached itself. It had exploited the riches of nature too far. Quietly and complacently, it was sinking into decadence, and progress had come to mean the progress of the Machine.

The machine fulfills all human needs and then some, but it stops working. Unable to repair the machine, all of humanity perishes:

They wept for humanity, those two, not for themselves. They could not bear that this should be the end. Their silence was completed their hearts were opened, and they knew what had been important on the earth. Man, the flower of all flesh, the noblest of all creatures visible, man who had once made god in his image, and had mirrored his strength on the constellations, beautiful naked man was dying, strangled in the garments that he had woven. Century after century had he toiled, and here was his reward. Truly the garment had seemed heavenly at first, shot with colours of culture, sewn with the threads of self-denial. And heavenly it had been so long as man could shed it at will and live by the essence that is his soul, and the essence, equally divine, that is his body.

Most analyses of the novella focus on the clairvoyance of the author, E. M. Forster, in predicting specific technologies of the then distant future. The story includes things like handheld screens, video calling, and air travel, all decades before they were invented. While the predictions are inspired, the inferences drawn from them are what most interest me. The Machine Stops is techno-dystopian, but not in the usual ways. Most techno-dystopian visions emphasize the immediate first order consequences of a new technology when it is introduced. Instead, The Machine Stops explores the more subtle, long term second order consequences of depending upon a successfully adopted technology.

While The Machine Stops is a hyperbolic illustration, it is not far removed from the current state of some modern systems, just a few decades old. Samuel Arbesman’s book, Overcomplicated: Technology at the Limits of Comprehension, is full of examples of such systems in virtually every domain of human activity. Forster wrote this well before the advent of modern software systems, which are particularly prone to incomprehensibility, but he saw the risks of depending on complex, centralized systems. Readers familiar with Nassim Nicholas Taleb’s work may see parallels with his themes of tail risks and fragility, crystallized in The Logic of Risk Taking:

About every time I discuss the precautionary principle, some overeducated pundit suggests that “we cross the street by taking risks”, so why worry so much about the system? This sophistry usually causes a bit of anger on my part. Aside from the fact that the risk of being killed as a pedestrian is one per 47,000 years, the point is that my death is never the worst case scenario unless it correlates to that of others.

The Machine Stops is a cautionary tale for any system that trades greater complexity or centralization for increased efficiency. The machine reduced the volatility of individual human outcomes at the expense of increasing correlation across individual outcomes. When the machine failed, it did so systemically and irreversibly. It’s especially pertinent in a time in which technical, sociopolitical, and commercial systems are becoming increasingly centralized and complex. I strongly recommend reading the whole thing.