167 - We're Not Going to Die: Why Eliezer Yudkowsky is Wrong with Robin Hanson

and one of the moves that often AI people make to to spin scenarios is just to assume that AIS have none of that problem AIS do not need to coordinate they do not have conflicts between them they don't have internal conflicts they do not have any issues in how to organize and how to keep the peace between them none of that's a problem for AIS by assumption they're just these other thing that has no such problems and then of course that leads to scenarios like then they kill us all welcome to bankless where we explore the frontier of Internet money and internet

finance and also AI this is how to get started how to get better how to front run the opportunity this is Ryan Sean Adams I'm here with David Hoffman and we're here to help you become more bankless guys we promised another AI episode after an episode with Ellie easier well here it is here's the sequel the last episode of Ellie zuridkowski we titled correctly we're all gonna die because that's basically what he said I left that episode with um a lot of miscuits essential dread yeah existential dread like uh it was not good news in that

episode and I was having a difficulty processing it but Dave and I talked and we knew we had to have some follow-up episodes to tell the full story bankless style and go on the Journey of AI its intersection with our lives with the world and with crypto so here it is this is the answer to that this is Robin Hansen on the podcast today let me go over a few takeaways number one we talk about why Robin thinks Eliezer is is wrong we're not all gonna die from artificial intelligence but we might become their pets number two why we're more likely to have a civil war with AI

rather than being eaten by one single artificial intelligence number three right why robin is more worried about regulation of AI than actual AI very interesting number four why alien civilization spread like cancer this is also related to Ai and super interesting number five finally we get to what in the world does Robin Hansen think about crypto David why was this episode significant for you Robin Hansen is such a great thinker he's uh absolutely a polymath and really like Eliezer

progresses in his thoughts in the very linear logical fashion so he's easy to follow along with and so the first half of this episode maybe the 45 minutes 50 minutes is all about just the AI alignment debate and Eleazar versus Hanson which is a debate that has actually been going on for for many many years now this is not the uh over a decade yeah you're right this is not the first time that Eliezer has heard about Robin Hansen or Robin Hansen has debated Eleazar this is this is an ongoing saga

uh and so this is uh just course material for for Robin Hansen and so we really focus on this AI alignment problem and how these thinkers think that AI will develop and progress here on planet Earth and how they will in Friendly or unfriendly ways ultimately collide with Humanity so that's the first half of this episode the second half of this episode I think is is when this gets really really interesting if you just listen to the first habit this episode you would just think like oh this is the other half of the conversation to the AI debate which it

is the second half connects this to so many more rabbit holes and so many more topics of conversation that are actually I would say deeply ingrained to bankless content themes uh the themes of competition versus coercion the themes of exploring Frontiers uh the thing of of moloch and the prisoner's dilemma and how or things coordinate across across species and so uh the he we connect AI alignment to Robin hans's famous idea

that he calls grabby aliens if you haven't heard about grabby aliens you're in for a treat so this goes from what is a simple counter argument to a debate that we've had to a multi-faceted exploration uh that is just so cursory of many very many many deep subjects that I hope to explore further on bankless yeah and honestly David I'm dying to record the debrief with you because I want to get your take on this episode that was in con you can see how giddy I was in the second half of the episode I know and I want to contrast it

with with our Ellie user episode and how these two thinkers think and who do you think has the stronger case the debrief episode is the episode Dave and I record after the episode where we just talk about what just happened give our raw unfiltered thoughts so we're about to record that now if you are a bankless citizen then you have access to that right now if you'd like to become a citizen click the link in the show notes and you'll get access to our premium RSS feed where you'll have access to that also this episode will become a collectible next Monday collecting this

episode so hard me too I've got that laser episode my uh collections I'm also collecting this we release episode collections for our key episode of the week every Monday the mint time is 3 P.M Eastern and whatever time zone you're in you have to convert that uh that's it we're gonna get right to the episode with Robin Hansen but before we do we want to thank the sponsors that made this possible including our favorite crypto Exchange Bank list Nation we are excited to introduce you to Robin Hanson and he is a professor of Economics at

George Mason University and a research associate at the future of humanity Institute at Oxford this is uh takes an interdisciplinary Research Center approach that investigates big picture questions about humanity and its prospects and I think explaining exactly who Robin is and what he's doing is not a trivial task because he's a he's a polymath certainly spans many things he's provided many different mental models across various um like I don't know various disciplines but I would not call him conventional by

any means and I'm sure bankless listener you you will see what we mean here today Robin welcome to bankless glad to be here I think I can try to explain the kind of weird that I am yeah go ahead oh please that's because I can't explain the kind of weird ideas so I think I'm conventional on methods and weird on topics hmm so I I tend to look for a neglected important topics

and where I can find some sort of angle but I'm usually looking for a pretty conventional angle that is some sort of usual tools that just haven't been applied to an interesting important topic so I'm not a radical about theories or methods so use things like science and math and statistics and all of those normal non-radical things right I've spent a lifetime collecting all these usual tools all these systems really and I'm more of a polymath in that I'm trying to combine them on neglected important

topics so if you go to a talk where everybody's arguing and you pick a side I mean they got chances you're right are kind of small in the sense that there's all these other positions and you know maybe you'll be right but probably you'll be wrong because you're picking one of these many positions right if you go pick a topic where nobody's talking about it you just say anything sensible you can probably be right and we we've I think recently ran into somebody uh who follows that that path of sorts uh somebody who thinks very

logically and rationally but is applying it to uh unique more unique frontiers of the place that humanity is uh and that is uh our recent episode with Eliezer who followed a decently logical path that was relatively easy to follow that unfortunately led us into a dead end for like Humanity uh and so it was uh something that uh bankless that me and Ryan as co-host of this podcast but then also many of the listeners uh felt

trouble with because we Eliezer was able to guide us in a very simple and logical path on over onto the brink and so we're hoping to continue that conversation with you Robin uh as well as be able to explore blur explore some New Frontiers yeah Robin I I'm just wondering if we could just Wade right into the deep end of the pool here because what happened is basically user came on our podcast we thought we were going to talk about Ai and safety and Alignment all of these things you know talks about that a lot and we thought we were going to tie that to crypto what ended up happening Midway

through that podcast Robin is I got an existential crisis so did David the rest of the agenda seemed meaningless and unimportant because here's Eliezer telling us basically that the AI was imminent he didn't know whether it would happen in two years and five years and 10 years in 20 years but he knew the Final Destination which is that AIS would kill all of humanity and that we didn't have a chance and basically and I'm not being hyperbolic here Robin I know you haven't had a chance to go through that episode but he basically says you know kissed your spend time

with your loved ones because you do not know how much time you actually have and so this left like me and I think many bankless listeners on kind of a cliffhanger of like oh my God are we all gonna die and David tried to talk to me after that episode he's like Ryan it's okay like you know but but we knew we also had to like find someone who could give us another interpretation of what is going with on with AI and Robin we have chat gbt for it looks incredibly sophisticated it looks like it's

advancing at Breakneck speed and we're worried about the scenario so when Ellie easier calci says we're all going to die what do you what do you make of that do you think we're all going to die so AI inspires a lot of creativity regarding fear um and I think honestly most people as they live their lives they aren't really thinking about the long-term trajectory of civilization and where it might go and if you just make them think about

that I think just many people are able to see scenarios they think are pretty scary just based on you know projection of historical Trends toward the future and things changing a lot so I want to acknowledge there are some scary scenarios if you just think about things that way and I want to be clear what those are but I want to distinguish that from the particular extra sphere you might have about AI killing us all soon and I want to describe the particular

scenario Elie iser has in mind as I understand it as a very particular scenario where you have to pile on a whole bunch of assumptions together to get to a particular bad end and I want to say those assumptions seem somewhat unlikely and piling them all together makes the whole thing seem quite unlikely but nevertheless you just think about the long-term trajectory of civilization it may well go places that would scare you if you thought about that and so that'll be the

challenge for us to separate those two so which one would you like to go with first I would like to start with understanding what you think his assumptions are and maybe all right maybe starting let's do that Okay so the scenario is you have an an AI system like some coherent system it's got an owner and Builder people who sponsored it who have some application for it who are watching it and using it and testing it and um you know the way we would do for any

ideas system right there's an system and then somewhere along the line the system decides to try to improve itself now this isn't something most AI systems ever do and people have tried that and it usually doesn't work very well so usually when we improve Asia systems we do it another way so we train the more and more data give them more Hardware use a new algorithm but the hypothesis here is we're going to train this system is going to be assign the task figure out how to improve yourself and furthermore it's going to find a

wonderful way to do that and the fact that it found this wonderful way makes it now special compared to all the other AI systems so this is a world with lots of AI systems this is just one it's not the most powerful or the most impressive or interesting except for this one fact that it has found a way to improve itself and this way that it can improve itself is really quite remarkable first of all it's a big lump so most Innovation most improvements in all technology is lots of little things you gradually learn

muscle things and you get better once in a while we have bigger lumps and that scenario here there's a really huge law and this huge lot means the system can all of a sudden be much better at improving yourself then not only it could before but in essence then all the other systems in the world put together it's really quite an achievement this this lump it finds and a way to improve itself and in addition this way to improve itself has two other unusual features about Innovations first it's a remarkably broad Innovation

applies across a very wide range of tasks most Innovations we have and how to improve things are relatively narrow they'll let it improve in a narrow range of things but not over everything this Innovation lets you improve a really wide range of things and in addition most Innovations you have let you improve things and then the improvements run out until you'll find some other way to improve things again but this Innovation doesn't run out it allows this thing to keep improving more for many orders of magnitude you know maybe 10 artists Max or something like it's just really a huge

Innovation that just keeps last just keeps playing out it just keeps improving it doesn't run into errors while it improves itself things that Slow Down slow it down and get stuck for a long time it just it just keeps working right okay and whatever it does to pursue these Innovations these you know self-modifications will change it they probably will change its software configuration maybe that's relative use of resources the kinds of things it asks

for how it spends its time and and money that it has doing things the kind of communication it has you know it's changing itself and its owners Builders the ones who are you know sponsored it and made it and have have uses for it they don't notice this at all it is vastly improving itself and its owners is just oblivious now initially it's just some random obliviousness now at some point the system will get so capable maybe you could figure out how

to hide its new status and its new trajectory and then it might be more plausible that it succeeds at that if it's now very capable of hiding things but before that it was just doing stuff approving itself and its owner manages were just oblivious either they saw some changes they didn't care uh they they misinterpreted the changes they had some optimistic interpretation of where that could go but basically they're oblivious so if they knew it was actually improving enormously they could

be worried they would like step it maybe pause it try variations try to study it and make sure they understand it but they're not they're not doing that they are just oblivious and then the system reaches the point where it can either hide what it's doing or just rest control of itself from these owners builders and in addition like if one that rests to control itself presumably they would notice that but then and they might try to retaliate against it or recruit other powers to to you know lock it down but by assumption

it's at this point able to resist that it is powerful enough to either hide what it's doing or just rest control and resist attempts to control it at which point then it continues to improve becoming so powerful that it's more powerful than all the other everything in the world including all the other AIS and then soon afterwards um its goals have changed so during this whole process I mean two two things have to have

happened here one is it had to become an agent that is most AI systems aren't agents they don't think of themselves as I'm this person in the world who has this history and these goals and this is my plan for my future you know they are tools that do particular things somewhere along the line this one became an agent so this one says this is what I want and this is what who I am and this is how I'm going to do it and in order to be an agent that needs to have some goals and during this process by which it improved at some point it became an

agent and then at some point its goals changed a lot not just a little in effect now so any system we can think in terms of its goals if it takes actions among a set of options we can interpret those actions as achieving some goals versus others and for any system we can assign it some goals although the range of those goals might be narrow if we only see a range of narrow actions uh so we might not be able to interpret goals more generally so if we have an AI system that you know is a taxi driver

will be able to interpret the various routes it takes people on and how carefully it drives in terms of some overall goals respect to how fast it gets people there and how safely it does but maybe we can't interpret those goals much more widely as what would it do if it were a mountain climber or something because it's not climbing mountains right but still with respect to a certain range of activities it had some goals and then by assumption basically in this period process of growing its goals just become ineffect radically different

uh and then by assumption radically different goals through this random process are just arbitrarily different and then the final claim is arbitrarily different goals when they look at you as a human you're mostly good for your atoms you're not actually useful for much anything else at some point and then you are recruited for your atoms I.E destroyed and that's the end of the scenario here where we're all we all die so to recall the set of assumptions we've piled on

together we have an AI system that starts out with some sort of owner and Builder it is assign the task to improve itself it finds this fantastic ability to improve itself very lumpy very broad or works over many areas of magnitude it applies this ability its owners do not notice this for many orders of magnitude of improvement presumably or when it happens really really quickly potentially well that that would be

presumably the most likely way you can imagine the owner's not noticing perhaps but the fundamental thing is the owners don't notice if it was slow and they are just didn't want us though the scenario still plays out uh you know so the key reason we might postulate fast is is just to create the plausibility that the owners don't notice um because otherwise why would they notice um so you know but that's also part of like the the size of this Innovation right um that is or we're already approving AI

systems at some rate and so uh if we're gonna if this new method of improvement was only going to improve AI systems at the rate they're already improving then this AI system won't actually Stand Out compared to the others right in order for this to stand out it'll have to have a much faster rate of improvement to be distinguished from the others and this will then have to be substantially faster right because that would set the time scale there for what it would be to to be in the scenario so it both needs to be faster than the rate of growth of

other AI systems at the time substantially and fast enough that the owner Builders don't notice this radical change in its agenda priorities activities they're just not noticing that and then they don't notice it on to the point where this thing acquires the ability to become an agent have goals hide itself or you know free itself and defend itself

and then the last assumption and its goals radically change even if it was friendly and Cooperative with humans initially which presumably it was later on it's nothing like that it's just random set of goals at which point then by assumption now it kills assault so the question is how plausible are all those assumptions and I so we could walk through analogies and prior Technologies and histories in the last few centuries and I think film Advocates like Ellie

Eiser will say yeah this is unusual compared to recent history but they're gonna say recent history is irrelevant for this this is nothing like recent history the only things that are really relevant comparisons here say that you're right you know the rise of the human brain and maybe the rise of life itself and everything else is irrelevant so then they will you know reject other recent few centuries technology trajectories as not relevant analogies what did you just call Eliezer Robin a a

what Advocate a fuma Advocate what is foom form is just another name for this explosion that we've been talking about yeah intelligence explosion yeah I got to describe it gotcha uh kurzweil's stuff like that kind of thing um Singularity that sort of thing okay so well so Singularity is a different concept than Foo okay different content in some sense a foom is a kind of Singularity but not all singularities Robin thank you for guiding us because we're still learning in this right like uh bankless we had never done an AI podcast previously we covered a lot with

crypto and coordination economics and now we're doing this AI podcast and I feel like we got just got punched in the face so um you re-articulated while we're here yeah we're walking slower you re-articulating eliezer's assumptions is uh I think very helpful to me and so we want to get to like why you think those assumptions are unlikely uh to be true but but I do think you are right in in the episode with him um he basically sort of painted this uh Fantastical story of these assumptions and he he

basically said yeah those assumptions the things that you're describing I think and I don't want to put words in his mouth so maybe this is what I was hearing him say is you're just describing intelligence Robin that's what intelligence does and I'll give you exhibit a it's called human beings and I'll give you the algorithm it's called Evolution gradient descent over uh millions of years and hundreds of millions of years and we end up with a super like an intelligence but relative to maybe the animal kingdom but super intelligence that exerts its dominance and its will will has changed from just

procreating and spreading its genes genes and memetic material to something that Evolution would have never The evolutionary algorithm would have never envisioned it actually doing and so I think maybe what I was hearing the criticism would be like we already have an example of this Robin it's called intelligence and it's called humans what do you think about this so as I said um if we just think about the long run future we're in we can generate some

scenarios of concern independent of this particular set of assumptions Ellie azer had set up so um you know the scenario where humans arise and then humans change the world I guess you could imagine as the scary to Evolution If evolution could be scared but evolution doesn't really think that way but certainly you can see that in the long run you should expect to see a lot of change and a lot

of ways in which your descendants may be quite different from you and have agendas that are different from you and yours that is I think that's just a completely reasonable expectation about the long run uh so we could talk about that in general as your fear I just want to distinguish that from this particular set of assumptions that a word piled on as the film star as a film star it was like something that might happen in the next few years say and it would be a very specific event a

particular computer system suffers this particular event and then a particular thing happens that's a much more specific thing to be worried about than the general trajectory of our descendants into the long-term future so which one would you like to talk about I I'm trying to summarize really just the perspective differences here and I know you've gotten you've had this debate with Eleazar before so this is this is like review for you I think um eliezers conclusions is that while the

the future is Unwritten and the path of our future can be many and multivariate and we can have different possible outcomes Eliezer is like well all all roads lead to the super intelligence taking over and I think just to summarize your position is like that is a possible path um and it is something to consider and it is still less likely than the many many many other possible paths that are also perhaps an aggregate much more

likely is that a fair uh summary of your position so let's talk about this other more General Framing and argument so we could just say in history Humanity has changed a lot not just a little a lot we've not just changed some particular Technologies we've changed our culture in large ways we've changed the sort of basic values and habits that humans have and our ancestors from 10 000 or a hundred thousand or a million years ago

if they looked at us and saw what we're doing it's not at all here they would Embrace us as uh you know they are proud descendants they are proud of and happy to have replaced them but that's not at all obvious uh you know even just in the last thousand years or even shorter we have changed in ways in which we have repudiated many of our ancestors most deeply held values we are we rejected their religions we've rejected their patriotism rejected their sort of family

Allegiance and family Clan sort of allegiances we we have just rejected a lot of what our ancestors held most dear and that's happened over and over again through a long-term history that is each generation we have tried to chain our train our children to share our culture that's just a common thing humans do but our children have drifted away from our cultures and continue to just be different

and you know over a million years humans fundamentally ourselves changed and one of the things that happened is we became very culturally plastic and so culture Now is really able to change us a lot because we are we have become so able to be molded by our culture and even if our genes haven't changed that much well they've changed substantially say in the last 10 000 years our culture has enormously changed us and if you project the same Trend into the future you should expect that this will happen again and again

our descendants will change with respect to cultural Evolution and their technology and the structure of their society and their priorities and then of course at some point not to distant future we will be able to re-engineer what we are or even what our descendants are and that will allow even more change that is once we can make artificial Minds for example there's a vast space of artificial Minds we can choose from and we will explore a lot of that space

and that allows even more big possibilities for our descendants could be different from us so this story says our descendants will become yes super intelligence and yes they will be different from us in a great many ways which presumably also include values and if what you meant by alignment was how can I guarantee that my distant descendants do exactly what I say and believe exactly what I believe in will never disappoint me in what they

do because they are fully under my control I gotta go gee that looks kind of hard compared to what's happened in history so now if that's the fear you have I got to endorse that that's not based on any particular scenario of a particular computer system soon and what trajectory of an events it'll go through that's just projecting past Trends into the future in a very straightforward way so then I have to ask like is is that what you're worried about no and that's

that is not what I'm worried about that is my base case that like we're going to get more intelligent technology is going to change us culturally it's going to change the trajectory what if change speeds up a lot so that this thing you thought was going to happen in a million years happens in a hundred well I mean for me personally I'm a more of a techno optimist so I would be more on the side of like within reason uh of course more embracing of these types of change I know others

aren't quite as embracing and and also this was not the scenario at all that LEDs are presented he presented the scenario of not rapid change that you might not like in the future and it could come within your lifetime but actual obliteration of humanity like literally rearranging our atoms for some other artificial intelligence purpose and while you agree with like there will be lots of change as there has been in the past perhaps that change will even accelerate as we we um delve deeper into the kind of the

technology that that is in our future you do not think that an AI will simply the super intelligent artificial intelligence will simply obliterate humanity and kind of wipe us from uh from creation entire it'll be it won't be quite as drastic as that look let's be careful about noticing exactly what's the difference between the scenario I presented and the scenario he presented because they're not as different as you might think in both scenarios there's a descendants

in both scenarios The Descendants have values that are different from ours and in both scenarios they're certainly the possibility of some sort of violence or you know disrespective property rights such that The Descendants take things instead of asking for them or trading for them um because that's always been possible in history and it can remain possible in the future you know today most changes peaceful