Uncontrollable Power

Historian Yuval Noah Harari, the bestselling author of Sapiens, explores the symbiotic relationship between information networks and human societies, emphasizing that humankind’s capacity to collect, store, and transmit information drives its evolution.

Power

In a world with artificial intelligence (AI), a fundamental misunderstanding of power could precipitate humanity’s undoing.

In 1797, early in the Industrial Revolution, Johann Wolfgang von Goethe wrote The Sorcerer’s Apprentice, a poem that captures a persistent view of the relationship between information and power. Goethe depicts a young boy using his master’s magic to animate a broom to do his job of fetching water. The boy quickly loses control of the spell. He chops the broom in two, but the splinters turn into more enchanted brooms, hundreds of them, all fetching water, overwhelming the terrified boy. The sorcerer returns and undoes the spell. You may remember Disney’s recreation of this legend, starring Mickey Mouse as the apprentice and featuring ranks of marching brooms, in both the 1940 and the 2000 versions of the classic movie Fantasia.

In Harari’s view, the story’s lesson — “Never summon powers that you cannot control” — applies to humanity’s experience of religion and of AI.

The main premise of this book is that humankind gains enormous power by building large networks of cooperation, but the way these networks are built predisposes us to use that power unwisely.Yuval Noah Harari

In the story, the sorcerer fixes the apprentice’s mess, implying that a heroic figure can solve the problems that arise when people summon powerful forces beyond their control. However, power derives from social cooperation, not individuals, and no save-the-world sorcerer awaits in the wings today.

Harari explains that the ability to tell and believe in shared myths fosters unity. For example, believing in divine figures or the legitimacy of kings empowered early societies to mobilize their resources, organize armies, and maintain social order.

Information Networks

Harari sets out several foundational ideas about information networks. He stresses the concept that information itself is a tool for creating networks that promote cooperation. And, he notes, only human beings store, process, and share information within complex networks. 

Information networks have historic origins, rooted in ancient times when primitive communication methods, such as storytelling and shared mythology, supported large-scale human cooperation. Throughout history, people have depended on a blend of accurate information and fictional or mythic narratives to form large, cohesive societies. 

Shared stories were the first large-scale information networks. Stories provide the emotional and ideological glue that binds societies. Shared documents establish the logistical framework and the bureaucracies that enable complex societies to function. But there is a pitfall: Throughout history, human societies have overestimated the reliability of their information networks, often succumbing to fantasies of infallibility that sprang from religious dogma, the scientific knowledge of the time, technology, or political ideology.

Religion can fulfill many different functions, like providing solace or explaining the mysteries of life. But historically, the most important function of religion has been to provide superhuman legitimacy for the social order.Yuval Noah Harari

Democracies rely on decentralized information networks that enable informed decision-making and allow ideas and opinions to flow freely. Totalitarian regimes centralize the control of information and use it to fuel propaganda and censorship, while leaving the citizenry uninformed.

The transition from human-mediated to machine-mediated information processing transformed the landscape of information, allowing systems to operate at unprecedented speed and efficiency. 

Computers and AI mark a profound departure from previous information technology, such as the printing press. While earlier technologies amplified human capabilities, Harari explains, AI can make decisions and generate new forms of knowledge without human intervention. With each advance in computing and AI, experts predict that machines will never be able to do the next big thing, whether defeating a chess grand master or driving a car. And then, machines do it.

Political and social implications

Artificial intelligence makes information networks more complex, powerful, and pervasive. It embeds an unprecedented “alien intelligence” into their processes and functions, such as making decisions and analyzing and disseminating data. Like the humans who create them, all AI networks are fragile and susceptible to corruption.

Harari cautions that people may struggle to grasp the consequences of integrating AI into their everyday lives. For example, any reasonable expectation of personal privacy could evaporate as AI combines information gathered by pervasive surveillance with tools that process, analyze, and convey collected data at lightning speeds.

Today’s continuous systems process data around the clock. People find themselves inundated without realizing that their flawed information networks could be delivering junk, leaving them increasingly misinformed or disconnected from reality. AI often provides deceptive, incorrect, or excessive information.

AI and digital technologies also disrupt a society’s political discourse, Harari warns. This disruption gains even more impetus when social media platforms and algorithm-driven news feeds targeting a specific audience or belief system come to dominate public communication. Social media itself often creates echo chambers that distort public perception, undermine democratic ideals, and weaken the self-correcting mechanisms of political structures, including the press and the legal system.

The problem with computers isn’t that they are particularly evil, but that they are particularly powerful.Yuval Noah Harari

A “Silicon Curtain” might emerge, dividing the world not by ideology but by access to and control of technology. Some countries or regions could gain dominance while others fall behind. As Harari sees it, humanity stands at a critical juncture for avoiding a deeper digital divide and preventing totalitarianism at the hands of human bad actors or AI.

But AI has positive potential as well, Harari concludes, with a big if: If people integrate AI into their lives based on ethical frameworks and human intelligence, they could make it possible to channel its power to foster understanding, cooperation, and a more equitable society.

Cooperation and organization

In Yuval Noah Harari’s interpretation, people’s greatest strengths — cooperation and organization — stem from the information systems they create and come to depend upon. Harari warns that, unlike earlier technologies that helped organize information systems and networks, AI can operate autonomously. Thus, it presents an existential challenge to human control over information. The root of this challenge, Harari makes clear, is the interface among ever more powerful — and impenetrable — technology, the credulity of its users, and the malign intent of those who seek to seize and maintain power. Harari recognizes that, throughout history, totalitarian leaders or regimes have sought to control the nature of and dissemination of information, a crucial fuel for free people. He is not optimistic that AI might thwart those impulses.