Techno Hubris Cometh Before the Fall
How we're optimizing and scaling our way to misery and irrelevance
“There is no path to happiness. Happiness is the path.”
—A.J. Muste1
Things You Can Touch
I’ll begin this essay with a pop quiz: What do Neil Gorsuch’s recent Reason Magazine interview, the measles outbreak in South Carolina, the dramatic increase in the price of gold, and Donald Trump’s ballroom all have in common?
If you selected any of the first three answers, then you got it mostly right. However, none of them fits perfectly—the fly in the ointment being the Gorsuch interview.
For those who haven’t perused social media in recent weeks, the interview drew the attention and consternation of many prominent right-wing populists (see below), mainly on account of the Justice’s description of America as a creedal nation (see clip here).
The common thread that ties the reactions to Gorsuch’s comment together with the measles outbreak, the ballroom, and the rising price of gold is an emerging and widespread contemporary trend among Americans to reject the abstract, the conceptual, and the universal in favor of the tangible, the concrete, and the particular. Whether it be the supplanting of liberal democracy and rule of law by patrimonialism and charismatic authority, the jettisoning of constitutional conservatism by many rightists in favor of blood-and-soil nationalism, the flight to gold in conjunction with the loss of confidence in fiat currency (especially the dollar), or the growing public distrust of vaccines—huge swaths of Americans now seem exhausted and disillusioned with remote, impersonal, decentralized, trust-based systems, and instead are seeking more accessible alternatives that they can more easily see, touch, or feel.
The most visible expression of this pre-modern impulse has of course been the emergence of right-wing populism as a major political force throughout the West over the past decade. In an earlier essay addressing a related topic, I argued that contemporary populism’s most alluring and essential feature lies in its capacity to offer legibility to citizens who feel cognitively overwhelmed by the demands of the modern world by offering a dumbed down, fantasy-driven, crowdsourced facsimile of our political reality which they find more comprehensible and personally empowering.
While I’ve made no effort to conceal my contempt for most aspects of this movement—the primitive tribalism, the conspiracy theories, the incoherent and wildly excessive use of the word “they,” the embrace of “do your own research” junk science, the legions of grifters, opportunists, and political entrepreneurs who’ve made it their meal ticket—that doesn’t mean I see no value in trying to understand it as something beyond an epidemic of mass stupidity. Though my earlier characterizations of right-populism have tended to emphasize its more absurd and farcical features—treating it as something akin to a population-level Truman Show—it can also be regarded as a civilizational auto-immune response gone haywire: An incoherent, self-cannibalizing, bottom-up societal reaction to a widespread perception that we’re losing our way not just as a country, but as a species, due to the downstream effects of the technologies and incentives that are shaping the modern world.
Behind the Eight Ball but Ahead of the Curve
In addition to the many other colorful adjectives and descriptions that I’ve applied to the MAGA faithful, here’s something else that they are: ahead of the curve. What the “forgotten man” ultimately represents is the first round of casualties in a technological and social revolution that’s leaving humans behind. I suspect that this is part of what’s behind the appetite to replace impersonal, opaque, trust-based systems with things you can touch: the desire to return to a more legible and human-centric world in which human creativity, human labor, and human relationships are not all subordinated to economic imperatives and technological accelerationism.
I’m not the only person to see a connection between the growing opposition to Big Tech and the populist right’s rabid, foaming-at-the-mouth hostility towards anything that vaguely smacks of “globalism.” Here are some observations from a couple of accounts that I often enjoy:👇
I don’t necessarily disagree with these statements in a narrow sense. But the broader sentiment they express gives me pause. While the on-the-ground activism in opposition to data center construction may be full of goofy, misinformed people with no understanding of basic economics, public opposition and concern about technological acceleration (particularly as it pertains to AI) should not be cavalierly dismissed.
We’re on the cusp of what is perhaps the most consequential, destabilizing, and dangerous technological breakthrough in human history. And the key decisions are being made by a very small number of people without public buy-in. The rationales given for why we have to approach such a monumental step in such a frenzied and reckless way that’s so dismissive of popular concerns tend to take two basic forms:
— “If we don’t build it, then China will.”
— “You have no idea how much economic value is being created you low-IQ idiots. So just shut up and let the adults decide.”
The first rationale is straightforward, utilitarian, and hard to dismiss. Though I don’t think it should automatically be accepted as dispositive. The second, in addition to being imperious, has both practical and philosophical implications.
Subjecting the social, political, and cultural spheres to the forces of online virality and algorithms designed to feed us an endless stream of rage-inducing content that keeps us online has produced massive, largely unanticipated, negative externalities. The impacts have been felt both at the individual level in terms of our emotional and cognitive health, and also at the societal level by contributing to crippling (and potentially civilization-threatening) institutional and political dysfunction throughout the West.
Whatever the balance sheets of Silicon Valley’s most valuable internet companies might say, you don’t have to be a degrowther to wonder if a lot of what they’ve done has really been a net positive for humanity. Whether their founders began as idealists or were cynics all along, the logic of the attention economy (which exists largely as a result of decisions that they made) has transformed their industry into the new Big Tobacco—only with far higher stakes. Taking all of this into account, it seems reasonable for ordinary (as well as extra-ordinary) citizens to refuse to be cowed by those who’d argue that they should simply passively accept that the same powerful people who’ve unleashed forces that have already nearly brought society to its knees must be given carte blanche to rush headlong into a push for AGI that’s likely to have far more severe consequences than anything they’ve done to date.
I chose to refer to MAGA devotees as the “first round of casualties … ” (see first paragraph of this section) because the same forces that have caused some of our more cognitively limited fellow citizens to lose the thread entirely and seek refuge in an idolatrous cult revolving around a deranged fabulist are coming for all of us. Hanania et al. can dismiss these sentiments as low IQ and economically illiterate. Yet I think beneath their economic determinism lies something else: a broader contempt for humanity. I’m not entirely unsympathetic to this sentiment, by the way. On some level I share it. When compared to the technological wonders of our era, human emotional needs can start to seem insignificant, pedestrian, and even silly. But to me this is part of the problem. We’re building a world in which human consciousness is poised to be treated as little more than a rounding error. Whether you regard all of this as worth opposing isn’t merely a matter of economics or forecasting. What’s ultimately at issue is a much more fundamental question: 👇
What Are People For?
The title of this section is a line from Kurt Vonnegut’s 1952 novel Player Piano. The novel, Vonnegut’s first, marked one of the earlier contributions to the genre of science fiction dramas that featured a dystopic society of the future in which technology had rendered most human beings useless and without any clear social or economic purpose. The question “What are people for?” is posed to a supercomputer named EPICAC by a religious leader who goes by the title The Shah of Bratpuhr when touring the fictional industrial city of Ilium New York.
The question isn’t exactly original. In fact it lies at the foundation of nearly every major religion and school of thought going all the way back to the dawn of civilization. The demand for satisfactory answers forms the basis for the entire suite of academic disciplines that make up the humanities. It has simple answers, some of which have value, and far more complex answers. Among the simplest and most glib set of answers to the question “What are people for?”, the answer that’s reigned supreme for thousands of years, at least since Cain’s famous conversation with God, has been “other people.”
One of the more enigmatic qualities of many techno-optimists is that they embrace the notion of revolutionary technological change while at the same time arguing to their detractors that in the realm of human affairs, there’s nothing really new under the sun. Their standard defense goes something like this:
—Technological revolutions have always produced winners and losers.
—They’ve always destroyed some industries while creating new ones.
—They’ve always produced social and political upheaval.
—But on net, society has invariably emerged immensely better off.
As a historical synopsis, this may be a reasonable first-order approximation of reality. Yet I think there’s ample reason to believe that as far as the current iteration of rapid technological innovation is concerned, there is something new under the sun.
While our society has evolved various mechanisms for assimilating labor-replacing technology, it’s developed no such immunities to the downstream effects of technology that makes people increasingly obsolete as social companions. This is arguably where the impact of the latest generation of technological innovation has been most insidious, which once again returns us to that same question: What are people for?
The Canary in the Coal Mine
I remember about twenty years ago a friend and I had a running conversation going where we both agreed that the world suddenly seemed “very dead.” Some time between the late 90’s and mid 2000’s it felt as though there’d been a subtle yet profound change in the culture—the kind of thing Gen Z would later come to describe as a vibe shift. Young people seemed more atomized, nightlife had died way down, and novelty and adventure were increasingly being abandoned in favor of a convenient, predictable, technologically-mediated domesticity. Over the course of about five years, the cultural touchstone of the ecstasy-soaked rave was supplanted by the image of a young person binge-watching reality TV in their underwear while doing bong hits and instant-messaging with friends. All of a sudden, it seemed, people were abandoning transcendence and excitement en masse and opting instead to be anesthetized and amused.
This is all anecdotal of course, merely reflecting the impressions of two disgruntled, not especially well-traveled twenty-somethings in one northeastern city. At the time, I recall that when I shared these observations with others I tended to get blank stares and shrugged shoulders. Very few among my cohort seemed to perceive any problem.
In the cultural realm as well, there didn’t seem to be an abundance of reflection about the direction society was headed in. The most popular movies and TV shows often tended to be geared towards escape, offering little in the way of insight into contemporary societal ills. This was the era when fantasy franchises like Harry Potter and X-Men were first coming to dominate at the Box Office, while movies depicting real people living in the here and now ceased drawing large audiences.
Putting It All Online
Speaking of movies about real people living in the here and now, I remember when I first saw the movie The Social Network. It was the beginning of 2011 and it had just become available on HBO on Demand. I’d never been a fan of Aaron Sorkin, and had always found his characters and dialogue unbearably smug and self-aggrandizing. In this case, however, it all seemed to work. Perhaps this was because the main character, whom the audience was being invited to deplore, was meant to come across as unbearably smug and self-aggrandizing. In any case, I recall finding the movie so engrossing that I watched it twice all the way through without stopping.
What most captivated me was that it combined a character arc of personal economic triumph at the cost of social and spiritual health, a la Citizen Kane, with a Revenge of the Nerds theme that captured some of the essential features of a fundamental regime shift in which social capital was becoming increasingly untethered from interpersonal skills and instead accruing to those most adept with algorithms and symbol manipulation. The most conspicuous depiction of this phenomenon in the movie was Zuckerberg’s triumph over the Winklevoss twins, which seemed to feature them as proxies for a dying social order, and Zuckerberg as the personification of its replacement. In the end, his inability to escape his personal emptiness despite his otherworldly success mirrors the contradictions of the technological and social revolution in which he’s played a starring role; a revolution which has harnessed technology to vastly expand the efficiency and reach of our social networks while paradoxically leaving us more isolated and alone.
There was a particular scene near the beginning of the movie that’s always stood out in my mind. Zuckerberg is at a party with Eduardo Saverin (his best friend and future CFO) and explaining to him the idea that would eventually blossom into Facebook. The key line in the exchange was when Zuckerberg explains to Saverin “I’m talking about taking the entire social experience of college and putting it online.” 👇
My initial reaction to hearing this line was to ask (to nobody in particular)—“Why on earth would you want to do that?” Fifteen years later I’ve yet to hear anything that seems like a satisfactory answer.
Part of the reason I found this scene noteworthy is because you of course can’t put the entire social experience of college online. At least not without losing an awful lot. The complexity of social life can’t be captured 140 characters at a time. Or through pics and bios.
Facebook and the many other social media companies that have emerged over the past 25 years may have been created and marketed as tools for enhancing and facilitating human interaction. But through a combination of economic incentives and network effects, they’ve rapidly evolved into much more of a substitute. And this is where that same question “What are people for?” once again becomes central. Technological revolutions have been disrupting labor markets and contributing to social upheaval for hundreds of years. Yet throughout recorded history, humans have remained social animals. The lowest dimensional valid answer to the question “What are people for?” hasn’t changed—until now.
Consciousness in the Crosshairs
Matt Yglesias recently published an article titled Why Kids Don’t Go Anywhere Anymore in which he attempted to address some of these issues. In one section called Staying Home Has Gotten More Fun, he points out that the many forms of entertainment and engagement that can now be accessed with the click of a button online have turned boredom into an anachronism, thus reducing the premium on face-to-face human interaction. While I agree with this argument, I don’t think Yglesias goes far enough in exploring its full implications.
When I was first attempting to talk to people about the issue of social life increasingly migrating to the online sphere and trying to explain what I thought was potentially being lost in the process, I tended to get a bunch of different variants of the same basic response: Nobody was forcing anyone to spend time online. Thus, those who found the online world less than maximally stimulating always had the option of opting out. At the time the term network effects had yet to assume a prominent place in the popular lexicon, so my attempts to explain the relevance of the concept to others were often frustrated by the absence of appropriate terminology.
Network effects aren’t the only vicious cycle associated with the transition to life in the cyber realm. As the alternatives to human contact continue to become more accessible, more addictive, and more ubiquitous, the incentives to make ourselves fit company for other human beings plummet. In this sense the “end of boredom” is potentially far less innocuous than it sounds, as it contributes to a feedback loop whereby people end up self-optimizing for a world in which the value we place on other human beings is in perpetual freefall.
The sprint towards human irrelevance and social atomization that’s accompanied the transition to life online, and which has become progressively more apparent in the era of Web 2.0, has been further intensified in recent years by the rapid progress in AI. As large language models have grown ever more sophisticated and prevalent, they’re trained on data from an internet that’s increasingly populated by AI-generated content. This has at times contributed to a degradation in the quality of their output, which can become generic, derivative slop, descended from other slop, which itself descended from slop, and on and on it goes … Industry insiders even have their own term to refer to this phenomenon: model collapse.
With people adapting to a less human-centric world and human creativity being drowned out by synthetic content, consciousness itself becomes an irrelevant detail. An indecipherable rounding error submerged in noise. A private affair that’s no longer worth cultivating, sharing, or exploring. The nightmare scenario that arises out of this isn’t merely a super intelligence that disposes of humanity. It’s a world in which human consciousness has atrophied and been devalued to the point that it no longer feels like preserving it even matters.
The Mother Functor
Bookended by 9/11 and the financial crisis, the 2000’s seemed like the decade when America discovered its limitations. It turned out that our geographic isolation didn’t insulate us from attack in the way that we thought, and the pace of economic growth we’d come to expect was increasingly being sustained by a massive wave of securitization and deregulation of financial markets that culminated in the 2008 collapse. In between, we also discovered that our military, while preeminent, was not omnipotent. And it was not very effective at transforming tribal societies into thriving liberal democracies. Yet while Americans were busy learning that we were not in fact invulnerable and all-powerful, there was another story of great import developing that was largely passing beneath the radar. The internet, which had previously functioned primarily as a tool for facilitating commerce and collaboration, had begun colonizing domains that had hitherto never been associated with ideas like economies of scale and algorithmic optimization.
As social life migrated online during the 2000’s and tech companies began to embrace an ad-based profit model oriented around monetizing attention, our relationship to the internet fundamentally changed. It went from being a tool to being the primary interface through which we interact with the rest of the world. This certainly has its advantages if you’re looking to find a ride, or order a meal, or schedule a flight, or just look something up. But there are certain parts of life that are meant to be lived rather than optimized, and that can better be likened to a dance than a race: friendship, adventure, sport, courtship, and perhaps most importantly of all—childhood.
Speaking of childhood, I’m reminded now of a famous scene from a movie of my youth. In the original Jurassic Park, as one of the dinosaurs is about to be fed a live goat, the Dr. Grant character played by Sam Neill remarks to Jeff Goldblum “T-Rex doesn’t want to be fed. He wants to hunt.” While we’ve built a world of technological wonders, we remain the same species that we were thousands of years ago. As such, lives in which most forms of struggle, friction, and uncertainty have been mediated out of existence tend to lack a certain spark.
In the first section of this essay I described the people attracted to right-wing populist parties as the first casualties of a technological and social revolution that’s leaving humans behind. Some, like Richard Hanania and Jeff Giesea, see the resistance to the AI data center build out and some of the broader concerns around AGI as growing out of a similar impulse. And I think they may be onto something there. But that isn’t automatically discrediting. This is where that same pesky question “What are people for?” once again reenters the picture. And this time it’s accompanied by a related question—“What is technology for?”
Are our inventions meant to serve us? Or is it the other way around? Are people really for other people? Or are we meant to serve some higher purpose? To get Elon Musk to Mars? To help Peter Thiel achieve immortality? To invent an intelligence that’s superior to us in every way so we can hand it the baton? Are people the end game or are we just one step along a much longer path? Whatever the answer, it seems that the decision should be left up to more than just a tiny handful of self-worshiping billionaires.
When I described the people belonging to populist-right parties as being the first casualties of a technological revolution that was sucking the meaning and purpose out of our lives, I also mentioned that the same forces were eventually going to be coming for all of us. Well, recently they began coming for me. 👇
I once had a professor who used to tell us that someday all of math would be captured in a single functorial arrow. And all we’d be able to do is sit back and marvel at it. He called it the Mother Functor. The problem, however, is that we don’t want to just sit back and marvel at it. We want to hunt!
The author of this quote, A.J. Muste, was a Christian pacifist and peace activist. The quote is often misattributed to Gautama Buddha.
Note: This post has been lightly edited for stylistic clarity.
I just cross-posted this with this note: Our reader Josh Rosenberg--whose work I've posted here before under the "What went wrong" rubric--wrote this during my four-day *ekstasis,* as one reader described it, or my adventure in art therapy, as another reader put it more prosaically. Josh had no knowledge of what I was about to write. (Nor did I.) It's spookily germane--especially his point about the impulse to retreat into the concrete and the tangible. I was carrying out the upscale version of the maneuver he describes here. This is an important insight: I might not have recognized it to be a really good insight, though. if I hadn't just spent my week the way I did. (I actually have more to say about this but no space. I'll leave it in the comments on his piece.) For those who don't subscribe, I'm talking about this: https://claireberlinski.substack.com/p/under-the-roman-goddess. It's a description of the utter joy I felt when my Internet went out for four days.
The word I've been drawn to in considering what you're describing, Josh, is "infantilization." And what have I been doing if not infantilizing myself? I've been finger-painting, for God's sake! But you're right to say that it's not quite infantilization. Or perhaps it is, but I think you're right: it's prompted by the utterly unnatural way we're forcing our brains--which aren't adapted to it--to operate. It's unbearable, and it's making us nuts.
Josh -- I like the article.! It reads quite fluently about the forces disrupting our society (hence how we found ourselves in Trump II) & the necessity to step back from allowing AI & the tech bros to wipe out our humanity on the way to making themselves all trillionaires. Moreover, you are on target about these trends germinating in the early 21st century, leading on up to something awful for humankind as we head toward mid-century.
A few caveats: I wish you would find less jarring synonyms for stupidity & ignorance. As to the movie, "The Social Network," please humanize it by mentioning that a very young Jesse Eisenberg convincingly played the nerd king, Zuckerberg. Finally, I was glad that at the very end of the article you referred to the economic trends (i.e., the tech billionaires) that have played a critical role in creating this Brave New World that I (in my old age) feel myself propelled along all the way into its very core -- minus any real agency on my part. Nevertheless, as you are a mathematician, I do wish that you had discussed some of the economic factors propelling the techno hubris earlier in your article.
Overall, quite a solid, thoughtful, cogent article! I am glad to have read it & will send it on --