An Embarrassing Confession
Could this be the "most important moment in history?"
This is the one moment in ten thousand ages.
— Li Si1
Introduction
I have a really strong feeling; I’m embarrassed to express it, because it isn’t in my usual register, it isn’t good historical writing, and it involves some fallacious thinking. Still, I think it is important to share. So, I’m going to share this feeling, then spend the rest of this essay elaborating on the problems with this kind of thinking and why I still think I’m right.
Here it is:
In 2022, I believed I was living through a relatively normal period in human history. I probably would have predicted that my life span (say 2004-2077) would be defined by a gradual declension of American power, an increase in global life expectancy, and the growing ubiquity of information technology. I would have guessed that the 21st century would be less “important” than the 20th, and that my parents lived through a more transformative period. I would have guessed that science would continue within the normal paradigm with which I was familiar, and we would make gradual progress toward curing cancer and understanding physics.2
I am confident that this is all wrong. For better or for worse, the next century will very probably present greater risks and rewards for humanity than any period in human history. We have probably created a “technology” which is not comparable to any easy historical reference point. It simultaneously poses the extinction risk of an atomic bomb and promises to usher in a new scientific paradigm, where an artificial superintelligence (ASI) solves most of our problems. It may be a new sort of intelligence entirely. I think we might either emerge from the 21st century a changed species, or not emerge from it at all.
My impulses as a student of history tell me that this framing is extremely problematic. First of all, importance is not a property of periods; it is a relation between a period and the observer doing the ranking. Observers are often really bad at being unbiased: I, a history student who studies 17th-century British philosophy, think that 17th-century Britain is one of the most important places and periods in history. Duh.
Imagine if you could ask the character of Hermione Granger in Deathly Hallows which event in the Harry Potter story was the most important thus far. Would her response be valuable or accurate? Would you trust her to give a good answer? Probably not! She doesn’t know where events are going, and moreover she doesn’t have a very good perspective from inside the story. We have the same problem. We are inside the causal network of events, actions, people, creations, etc., that link the past and future.
Another objection: to say “you are living through the most important period in history to this point” is to perform a speech act, an illocution, more than to make an observation. It doesn’t really mean anything to say that this is the most important period in history — we just can’t know that. Instead, it is an appeal to the reader to take whatever action the speaker thinks is an appropriate response to the significance of this historical moment. In my case, that might be to suggest that a smart young person go into AI alignment or mechanistic interpretability.
Despite the hedging I’ve done, I still think this feeling is worth taking seriously. It’s pretty clear that some historical moments do end up transforming the future more than others; historians can only relativize so much. In this essay I will discuss our tendency to behave as if our historical circumstances are normal and sustainable, how we might try to understand AI and a potential future ASI in a historical context, a popular objection to this view, and what AI might mean for our human history.
Our Delicate Worlds
Human beings tend to act as if the circumstances of their lives will not change dramatically. Even as I write this, I feel on some deep level that things won’t really change. This despite the overwhelming evidence provided by human history that the circumstances of our lives are tenuous, and things fall apart. Our behavior is shaped by our model of the future. Our models of the future are shaped by our very delicate present circumstances, and therefore by a powerful normalcy bias.
This phenomenon is almost ubiquitous, and when you go looking for it you find it everywhere. People tend to behave as if things will always remain the same, even when they profess to (and sincerely do!) believe otherwise. I think we do this because we have very little confidence in our capacity to predict the future accurately. Fair enough, but it’s equally not rational to behave as if the present will continue. Anyway, we base our actions on what we understand and feel is certain: the present. We rationalize by dismissing the impact of new technologies or small-but-dangerous political ideologies, because they don’t change our lives right now.
This human flaw leads people to believe that things will not get vastly better or worse in short periods of time. John Maynard Keynes describes the Londoner of 1913 who believes that his world is not hanging in delicate balance:
The inhabitant of London could order by telephone, sipping his morning tea in bed, the various products of the whole earth, in such quantity as he might see fit, and reasonably expect their early delivery upon his doorstep…But, most important of all, he regarded this state of affairs as normal, certain, and permanent, except in the direction of further improvement, and any deviation from it as aberrant, scandalous, and avoidable.
The point being, of course, that these Londoners’ world was balanced on a delicate (and perhaps untenable) web which collapsed in 1914. Though by our nature we may tend to optimism, we are also similarly susceptible to behaving as if things cannot radically improve. In 1916 it was essentially certain that you knew someone who lost an infant child; 10% of all infants died. It might then have been difficult to imagine that ~ 95% of these deaths would have been prevented within the century.3 We simply have emotional blocks and cognitive biases which make us bad at imagining how things might radically change in our lifetimes.
To be very clear, I think that people within AI spaces are not the primary perpetrators of this kind of thinking. In fact, people who study AI are members of a very small group that are more likely to behave as if this technology might completely transform our lives, and might also kill us. It is the rest of us who have been painfully slow to face the music. 1,319 employees at leading AI research labs recently signed the “Pacing the Frontier” open letter, which says “AI could help create a dramatically better future, but that outcome is not guaranteed.”
I’m not advocating for following Aella’s lead.
More “Fire” than “Floppy Disk”
The past 200 years have seen the creation of countless transformative technologies.4 The modern era was the era of invention, mechanization, and automation. The computing and information revolution is perhaps a continuation of the modern industrial era, and it is ending. I think we are probably at the start of something we will look back on as a new “revolution.” That means right now you have an exceptional, outsized ability to influence future human lives and wellbeing. Imagine the relative transformative power you would have to transform the course of human history if you could travel back to the Fertile Crescent 15,000 years ago, to London in 1760, or to Silicon Valley in the 1970s. Getting in on something with major world-historical consequences early on gives you a lot more influence on the causal chain, generally speaking, than getting involved later. The alignment work that might save us all from doom needs to be done right now.
I strongly believe that our ability to create mathematical systems that behave intelligently is categorically separate from modern human inventions. It is a paradigm/civilization/species-shifting creation. It is more “wheel” than “windmill,” and more “fire” than “floppy disk.” That is not to say that artificial intelligence is simple; it is to say that simple is relative to the technological changes ushered by a technology. Human society was changed by the windmill and the floppy disk, true, but the wheel and fire have become fundamental parts of our identity as humans. They underlie all human history, and I expect the same will be true of AI.
(After writing this, I happened to stumble across this LessWrong post by Gwern where Douglas Hofstadter and Geoffrey Hinton compare AI to fire and the wheel, respectively. Maybe I would also suggest the invention of written language as a comparable development.)
We don’t yet understand the ramifications of creating and growing a non-human intelligence. We also don’t know if AI is an intelligence at all! What we can probably stipulate is that the distinction between a truly intelligent AI and one adept at mimicking intelligence won’t matter if we can’t find any meaningful distinctions, or if we are all dead. I suspect that we will gradually begin to think of AI models as intelligent as (or if) their abilities become less spiky.5
In Gödel, Escher, Bach, Douglas Hofstadter asked:
“Do you really understand Bach because you have taken him apart, or did you understand at that time you felt the exhilaration in every nerve in your body? And probably no one will ever understand the mysteries of intelligence and consciousness in an intuitive way. Each of us can understand people, and that is probably about as close as you can come.”
Imagine we do crack open the black box and see exactly how LLMs work: we set out looking for consciousness, personhood, and find…what? We don’t even know what we are looking for. We feel that we can be pretty sure other people are conscious because they are the same sort of thing as us, not because we have identified some quality or a part of the brain which creates consciousness.
This means that we are going to be the first humans to have to reckon with non-human intelligence, personhood, consciousness, moral patienthood, etc. I think we get a future where we have to deal with these questions even without ASI, just through the gradual improvement of models and the diffusion of AI through society.
Contra MacAskill, Kind Of
Will MacAskill, in his 2020 paper “Are we living at the hinge of history?” wrote some objections to Derek Parfit’s assertion that we live during the most influential period of human history. He is primarily concerned (for understandable reasons) with whether or not we should be passing resources forward to the future or using them right now, as Parfit’s view suggests we should. MacAskill argues that influence rises with knowledge, so people 10, 20, and 100 years from now will be better suited to make decisions and therefore should be allocated resources. He also says, given that you and your contemporaries are a fraction of the trillions of people who will ever live, concluding you’re extraordinarily situated is itself evidence your reasoning has gone wrong.
My big problem with MacAskill’s argument is that it would never allow a group of people to conclude that they really were living through a hinge-of-history moment. Clearly it’s possible for there to be a moment in history where we face either destruction or flourishing depending on the decisions we make. In MacAskill’s framework, how would one ever conclude that one was living through that moment? How would we act on that conclusion on time, if MacAskill wants us to keep passing the buck down the timeline to a point where we have more knowledge? Moreover, investing and passing forward is a strategy that presupposes the continued existence of the species!
MacAskill directly addresses the view (attributed to Bostrom and Yudkowsky) that we might be on the verge of value lock-in. This is where an ASI ends up more capable than the rest of humanity combined, and its values, aligned or no, become enshrined forever. On this view, he says, influentialness goes vertical now and flat for the rest of time. MacAskill naturally observes that believing this requires tons of evidence we don’t have, and so we can’t accept that the hinge is now.
Here’s my point: I think we need to behave differently when we have abnormally high evidence for a lock-in scenario or a potential existential risk inflection point. Importantly, that doesn’t mean statistically compelling evidence, just unusually strong evidence. I think it would be silly to travel to, say, Los Alamos in 1945 and insist that the physicists developing the bomb not behave as if they are at a hinge. It is better, they tell me, to be safe than sorry. As I pointed out earlier, the alignment work that needs to be done has to be done right now, or at least at some point before we get superintelligence (I’m assuming that comes within our lifetimes.)
Conclusion
It is embarrassing to write these things. I accept that I could be completely wrong about this. I might look at this post in 6 years and cringe: “how could I not realize that the creativity/compute/capital bottleneck would result in AI becoming a completely normal technology!” One thing I will not do, though, is look back and regret thinking about these things. I will not regret saying this: we exist in a very, very important moment of human history, and we have an obligation to go out and do as much good as we can.
Footnote 25 in MacAskill’s “Are we living at the hinge of history?”
Other people were of course more bullish about technological innovations in the 21st century, however I think it is fair to say that most people would have agreed with me.
In the US, infant mortality is about 0.5%.
The programmable computer, electricity distribution networks, and antibiotics, vaccines, cars, airplanes, etc.
Importantly, even in the scenario where models don’t generalize well they can still pose an existential risk.
What are your thoughts on the open source vs privatization debate currently happening with the release of the China’s new Kimi model? Do you think either of these avenues would be particularly more influential to the ways AI will change our day to day lives?