159 - We’re All Gonna Die with Eliezer Yudkowsky

I think that we are hearing the last twins to start to blow the fabric of reality start to fray this thing alone cannot end the world but I think that probably some of the vast quantities of money being blindly and helplessly piled into here are going to end up actually accomplishing something where we explore the frontier of Internet money and internet Finance this is how to get started how to get better how to front run the opportunity this is

Ryan Sean Adams I'm here with David Hoffman and we're here to help you become more bankless okay guys uh we wanted to do an episode on AI at bankless but I feel like what we asked for we accidentally wade into the deep end of the pool here yeah um and I think as we get before we get into this episode it probably warrants a few comments I'm gonna say a few things I'd like to hear from you too but one thing I want to tell the listener is um don't listen to this episode if you're not ready for an existential crisis okay

like I'm kind of serious about this um I'm leaving this episode shaken uh and I don't say that lightly uh in fact David I think you and I will have some things to discuss in the debrief as far as how this impacted you um but this was an impactful one it sort of hit me during the recording and I didn't know fully how to react I honestly am coming out of this episode wanting to refute some of the claims made in this episode by Our Guest um Eliezer ikowski who makes the claim

that humanity is on the cusp of developing an AI That's going to destroy us and that there's really not much we can do to stop it there's no way around it yeah I have a lot of respect for this guess let me say that so it's not as if I have some sort of big brain technical disagreement here in fact I don't even know enough to fully disagree with anything his he's saying but the conclusion is so dire and so existentially heavy that um I'm worried about it impacting

you listener uh if we don't give you this warning going in um I also feel like David as interviewers maybe we could have done a better job uh I'll say this on behalf of myself sometimes I peppered him with a lot of questions uh in one Fell Swoop and he was probably only ready to synthesize one at a time I also feel like we got caught flat-footed at times I wasn't expecting his answers to be so Frank and so dire David like it was just um a rift of Hope and I appreciated very

much the honesty as we always do on bankless but I appreciated it almost in the way that a patient might appreciate the honesty of their doctor telling them that their illness is terminal like it's still really heavy news isn't it um so that is the context going to this episode I will say one thing in good news for our failings as interviewers in this episode uh they might be remedied because at the end of this episode after we finished with uh hit the record button to stop recording

Eliezer said he'd be willing to provide additional q a um episode with the bankless community so if you guys have questions and if there's sufficient interest for Ellie user to answer uh tweet us to express that interest hit us in Discord get those messages over to us and let us know if you have some follow-up questions he said if there's enough interest in the community in the crypto Community I'll say he'd be willing to come on and do another episode with with follow-up q a maybe even a vitalik and Eliezer episode is in

store that's a possibility that we threw to him we've not talked to vitalik about that too but I just feel a little overwhelmed by the subject matter here um and that is the basis uh the Preamble through which we are introducing this episode um David there's a few benefits and takeaways I want to get into but before I do can you comment or reflect on um on that Preamble what are your thoughts going to this one yeah we approached the end of our agenda for every bankless podcast there's a equivalent agenda that runs alongside of

it but it once we got to this Crux of this conversation it was not possible to proceed in that agenda because what was the point Nothing Else Matters and nothing else really matters which is also just kind of relates to the subject matter at hand uh and so uh as we proceed you'll see us kind of circle back to the same inevitable conclusion over and over and over again which ultimately is kind of the punch line of the content uh and so I'm of a specific

disposition where stuff like this I kind of am like oh whatever okay just go about my life other people uh are of different dispositions and take these things more heavily so uh Ryan's warning at the beginning is if you are a type of person to take existential crises directly to the face perhaps consider doing something else instead of listening to this episode I uh I think that is good counsel so a few things if you're looking for an outline of the agenda we start by talking about chat GPT is this a new era

of artificial intelligence got to begin the conversation there number two we talk about what an artificial super intelligence might look like how smart exactly is it what types of things could it do that humans cannot do number three we talk about why an AI super intelligence will almost certainly spell the end of humanity and why it'll be really hard if not impossible according to our guest to stop this from happening and number four we talk about if there is absolutely anything we can do about

all of this we are heading careening maybe towards the abreast Abyss can we divert Direction and not go off the cliff that that is the question we ask Elisa with um David I think you and I have a lot to talk about during the debrief all right guys the debrief is an episode that we record right after the episode it's available for all bankless citizens we call this the bankless premium feed you can access that now to get our raw and unfiltered thoughts on the episode and I

think it's going to be pretty raw this time around David I'm like I didn't I didn't expect this to hit you so hard oh I'm dealing with it right now really and this is probably you know it's not too long after the episode so uh yeah I don't know how this how I'm gonna feel tomorrow but um definitely want to talk to you about this and maybe yeah have you talk I'll put I'll put my psych hat on yeah please I'm gonna need some help uh guys we're gonna get right to the episode with Eliezer but before we do we want to thank the sponsors that made this episode possible including Kraken our favorite recommended exchange for

2023. Kraken has been a leader in the crypto industry for the last 12 years dedicated to accelerating the global adoption of crypto Kraken puts an emphasis on security transparency C and client support which is why over 9 million clients have come to love kraken's products whether you're a beginner or a pro the Kraken ux is simple intuitive and frictionless making the kraken app a great place for all to get involved and learn about crypto for those with experience the redesigned Kraken Pro app and web experience is completely customizable to your trading

needs integrating key trading features into one seamless interface Kraken has a 24 7 365 client support team that is globally recognized Kraken support is available Wherever Whenever you need them by phone chat or email and for all of you nfters out there the brand new Kraken nft beta platform gives you the best nft trading experience possible Rarity rankings no gas fees and the ability to buy an nft straight with cash does your crypto exchange prioritize its customers the way that Kraken does and if not sign up with Kraken at kraken.com

bankless the Phantom wallet is coming to ethereum the number one wallet on Solana is bringing its millions of users and beloved ux to ethereum and polygon you haven't used Phantom before you've been missing out Phantom was one of the first wallets to Pioneer Solana staking inside the wallet and will be offering similar staking features for ethereum and polygon but that's just staking Phantom is also the best home for your nfts Phantom has a complete set of features to optimize your nft experience pin your favorites hide the Uglies remove the spam and also manage your nft sale

listings from inside the wallet Phantom is of course a multi-chain wallet but it makes chain management easy displaying your transactions in a human readable format with automatic warnings for malicious transactions or phishing websites Phantom has already saved over 20 000 users from getting scammed or hacked so get on the Phantom waitlist and be one of the first to access the multi-chain beta there's a link in the show notes or you can go to Phantom dot app waitlist to get access in late February hey bankless Nation if you're listening to this it's because you're on the free bankless RSS feed did you know

that there is an ad-free version of Bank list that comes with the bankless premium subscription no ads just straight to the content but that's just one of many things that a premium subscription gets you there's also the token report a monthly bullish bearish neutral report on the hottest tokens of the month and the regular updates from the token report go into the token Bible your first stop shop for every token worth investigating in crypto bankless premium also gets you a 30 discount to the permissionless conference which means it basically just pays for itself there's also the airdrop guide to make

sure you don't miss A Drop in 2023 but really the best part about bankless premium is hanging out with me Ryan and the rest of the bankless team in the Inner Circle Discord only for premium members want the alpha check out Ben the analyst Degen pit where you can ask him questions about the token report got a question I've got my own q a room for any questions that you might have at bank list we have huge things planned for 2023 including a new website with login with your ethereum address capabilities and we're super excited to ship what we are calling Bank list 2.0 soon TM so if you want extra help

exploring the frontier subscribe to bankless premium it's under 50 cents a day and provides a wealth of knowledge and support on your journey West I'll see you in the Discord bankless Nation we are super excited to introduce you to our next guest Eliezer utakowski is a decision theorist he's an AI researcher he's the cedar of the less wrong Community blog fantastic blog by the way there's so many other things that he's also done I can't I can't fit this in the in the short bio that we have to introduce you to uh Eliza but most relevant probably to this conversation

is he is working at the machine intelligence Research Institute to ensure that when we do make General artificial intelligence it doesn't come kill us all or at least it doesn't come banned cryptocurrency because that would be a poor outcome as well Eliezer it's great to have you on Bagless how you doing ah you know within one standard deviation of my own peculiar little mean fantastic um you know we want to start this conversation with something that is jumped onto the scene I think for a lot

of mainstream folks quite recently and that is um chat gbt so apparently over 100 million or so have logged on to chat GPT quite recently I've been playing it with it myself I found it very friendly very useful it even wrote me a sweet poem that I thought was very heartfelt and almost human-like I know that you have major concerns around AI safety and we're going to get into those concerns but can you tell us in the context of like something like a chat GPT

is this something we should be worried about that this is going to turn evil and enslave the human race like how should worried should we be about chat gbt and Bard and sort of the the new AI That's entered the scene recently PT itself zero it's not smart enough to do anything really wrong or really right either for that matter and what gives you the confidence to say that how do you know this excellent question so every now and then

somebody figures out how to put a new prompt into chat EPT um you know one time somebody found that it would talk well not Chachi petite but one of the earlier generations of Technology they found that it would sound smarter if you first told it it was Elias radkowski you know there's there's other prompts too but that one's one of my favorites so there's there's untapped potential in there that people haven't figured out how to prompt yet but when people figure it out it moves ahead sufficiently short

distances that I do feel fairly confident that there is not so much untapped potential in there that it is going to take over the world it's like making small movements and to take over the world that need a very large movement there's capability like there's there's there's places where it falls down on predicting the next line that a human would say in its shoes that seem indicative of probably that capability just is not in the giant

inscrutable matrices or it would be using it to predict the next line which is very heavily what it was optimized for so there's there's going to be like some untapped potential in there but I do feel quite confident that the upper range of that untapped potential is insufficient to outsmart all the living humans and implement the scenario that I'm worried about so even so though is chat GPT a big leap forward in the journey towards AI in your mind or is this fairly

incremental it's just for whatever reason it's caught mainstream attention gpt3 was a big Leap Forward um there's rumors about gpt4 which you know who knows um Chachi PT is a commercialization of the actual AI in the lab trying to Leap Forward um compare if you weren't if you had never heard of gpt3 or gpt2 or the whole range of text Transformers before chat

GPT suddenly entered into your life then that whole thing is a giant leap forward but it's a giant leap forward based on a technology that was published in if I recall correctly 2018. I think the what's going around in everyone's Minds right now and and the bankless listenership and crypto people at large are largely futurists so everyone I think listening understands that in the future there will be sentient AIS perhaps around us at least by the time that we all move on from this world so like we all know that this future of AI

is coming towards us and when we see something like chat gbt everyone's like oh is this the moment in which our world starts to become integrated with AI and so Elisa you've been uh you know tapped into the world world of AI is this are we on to something here or is this just another you know fad that we will internalize and then move on for and then the real moment of generalized AI is actually further much further out than we're initially giving credit for like where are we in this timeline

you know predictions are hard especially about the future I sure hope that this is where it saturates this or like the Next Generation it goes only this far it goes no further it doesn't get used for to make more steel or build better power plants first because that's illegal and second because the technology is like the the large language model Technologies basic vulnerabilities that's not reliable like if you it's it's good for applications where it

works eighty percent of the time but not where it needs to work 99.999 of the time this thing this this class of Technology can't drive a car because it will sometimes crash the car um so I hope it saturates there I hope they can't fix it I hope we get like a 10-year AI winter after this this is not what I actually predict I think that we are hearing the last twins to start to blow the fabric of reality start to fray this thing alone cannot end the world

um but I think that probably some of the vast quantities of money being blindly piled it blindly and helplessly piled into here are going to end up actually accomplishing something you know not most of the money that that can Justice like never happens in any field of human endeavor but one percent of 10 billion dollars is still a lot of money to actually accomplish something so I think uh listeners I think you've heard eleazar's you know thesis on this which is a pretty uh dim with respect to

um AI alignment and we'll get into what we mean by AI alignment and very worried about AI safety related issues but but I I think a lot of people for a lot of people to even sort of worry about AI safety and for us to even have that conversation I think they have to have some sort of grasp of what AGI looks like that that is um I understand that to me an artificial general intelligence and this idea of a of a super intelligence um can you tell us like if if there was a super Intelligence on the scene what

would it look like I mean is this going to look like a a big chat box on the internet that we can all type things into it's like an oracle type thing or is it like some sort of a robot that it's going to be constructed in a secret government lab uh is this like something somebody could accidentally create in a dorm room like what are we even looking for when we uh talk about the term AGI and uh super intelligence but first of all I'd say those are pretty pretty uh distinct Concepts um chat GPT is shows a very wide range

of generality compared to the previous generations of AI not like very wide generality compared to gpt3 not like literally the lab research that got commercialized that's the same generation but compared to you know stuff from 2018 or even 2020. chat GPT is gender is better at a much wider range of things without having been explicitly programmed by humans to be able to do those things it can it to imitate a human as best it can it has to capture all of

the things that humans can think about than it can which is not all the things it's still not very good at long multiplication unless you give it the right instructions which case suddenly can do but you know so it's like significantly more General than the previous generation of artificial Minds um humans were significantly more General than the previous generation of chimpanzees or rather Australopithecus or latest or last common ancestor

um humans are not fully General if humans were fully General we'd be at good at coding as we are at football throwing things or running you know we're not some of us are you know okay at programming but you know we're not specked for it we're not fully General Minds you can imagine something that's more General than a human and if it runs into something unfamiliar it's like okay let me just go reprogram myself a bit and then I'll be as adapted to this thing as I am to you know

anything else so chat GPT is less General than a human and but it's like generally genuinely ambiguous I think whether it's more or less General than say our cousins the chimpanzees or if you don't believe it's as general as a chimpanzee a dolphin or a cat so this idea of of uh general intelligence is sort of a range of things that it it can actually do a range of ways that can apply itself

how wide is it how much reprogramming does it need how much retraining does it need to get make it do a new thing bees build hives beavers build dams a human will look at a beehive and imagine a honeycomb shaped Dam and that's like humans alone in the animal kingdom but that doesn't mean that we are general intelligence as it means we're significantly more generally applicable intelligences than chimpanzees it's not like we're all that narrow we

can walk on the moon we can walk on the moon because there's aspects of our intelligence that are like made in full generality for universes that contain simplicities regularities things that recur over and over again we understand that a steal is hard on Earth it may stay hard on the moon and because that of that we can build Rockets walk on the moon breathe amid the vacuum chimpanzees cannot do that but that doesn't mean that humans are the most General possible things the thing that is more General than us

that figures that stuff out faster is the thing to be scared of if it's if the purpose is to which it turns our its intelligences are not ones that we would recognize as nice things even in the most Cosmopolitan and embracing senses of you know what's worth doing and and you said this idea of a general intelligence is different than the concept of super intelligence which which I also brought into that uh first part of the question how is super intelligence different than general

intelligence well because chat GPT has a little bit of of General intelligences general intelligence humans have more general intelligence a super intelligence is something that can beat any human and the entire human civilization at all the cognitive tasks um I don't know if the efficient market hypothesis is something where I can rely on uh yes the entire we're all crypto investors here we understand the

efficient market hypothesis for sure so the efficient market hypothesis is of course not generally true like it's not true that literally all the market prices are smart are smarter than you it's not true that all the prices on Earth are are smarter than you even the most arrogant person who is at all calibrated however still thinks that the efficient market hypothesis is true relative to them 99.99999 of the time they only think that they know better about one in a

million prices there might be important prices now the price of Bitcoin is an important price it's not just a random price but if you were like but if the efficient market hypothesis was only true to you ninety percent of the time you could just like pick out the 10 of the remaining prices and and and compound like and and double your money every day on the stock market and nobody can do that literally nobody can do that so this this property of relative efficiency that the market has to you that the

price that the price is estimated the future price it already has all the information you have not all the information that exists in principle maybe not all the information that the best Equity but relative to you it's efficient relative to you for you if you pick out a random price like the price of Microsoft stock something where you've got no special advantage that that estimate of its price a week later is efficient relative to you you can't do better than that price

we have much less experience with the notion of instrumental efficiency efficiency in choosing actions because actions are harder to aggregate estimates about than prices so you have to look at say um Alpha zero playing chess or just you know like stockfish whatever the latest stockfish number is and advanced chess engine when it makes a chess move you can't do better than that chess move

it may not be the optimal chest move but if you pick a different chest move you'll do worse that that you'd call like a kind of in efficiency of action given its goal of winning the game there is once you know its move unless you consult some more powerful AI than stockfish you can't figure out a better move than that super intelligence is like that with respect to everything with respect to all of humanity it is relatively efficient to humanity it has the best

estimates it not not perfect estimates but the best estimates and its estimates contain all the information that you've got about it its actions are the most efficient actions for accomplishing its goals if you think you see a better way to accomplish its goals you're mistaken so you're saying this is a super intelligence we'd have to imagine something that knows all of the chess moves in advance but here we're not talking about Chess we're talking about everything it knows all of the moves

that we would make and the most Optimum pattern including moves that we would not even know how to make and it knows these things in advance I mean how how would like human beings sort of experience such a super intelligence I think we still have a very hard time imagining something smarter than us just because we've never experienced uh anything like it before of course you know we all know somebody who's a genius level IQ maybe quite a bit smarter than us but we've never encountered something like uh that you're describing some sort

of mind that is super intelligent what sort of things would it be doing like that humans couldn't how would we experience this in the world I mean we do have some tiny bit of experience with it we have experience with with chess engines where we just can't figure out better moves than they make we have experience with market prices where even though your uncle has this you know like really long elaborate story about Microsoft stock you just know he's wrong why is he wrong because

if he was correct it would already be incorporated into the stock price and this notion and especially because the Market's efficiency are not perfect like that whole downward swing and then upward move in covid um I have friends who made more money off that than I did but I like still managed to buy buy back into the broader stock market on the exact day the low you know basically coincidence but so the markets aren't perfectly efficient but they're almost but they're efficient almost everywhere and that and that sense of like deference

that sense that your weird Uncle can't possibly be right because the hedge funds would know it you know unless he's talking about Kobe in which case maybe is right if if you have the right choice of weird Uncle you know like I I have weird friends who are like maybe better calling these things than your word uncle but yeah so so among humans it's subtle and then with super intelligence it's not subtle just massive Advantage but not perfect it's not that it knows every possible move you'll make before you make it

it's that it's got a good probability distribution about that and it you know has figured out you know it has figured out all the good moves you could make I figured I'd reply to those and and what's that I mean like in practice what's that like well unless it's limited narrow super intelligence I think you'd mostly don't get to observe it because you are dead unfortunately what so you can so you know like stockfish

make strictly better chest moves than you but it's playing on a very narrow board and the fact that it's better at you than chest doesn't mean it's better at you than everything and I think that the actual catastrophe scenario for AI looks like big advancement in a research lab may be driven by them getting a giant venture capital investment and being able to spend 10 times as much on gpus as they did before may be driven by a

do algorithmic Advanced like Transformers may be driven by hammering out some tweaks in last year's algorithmic Advanced that that's the thing to finally work efficiently and the AI there goes over a critical threshold which you know like most obviously could be like can write the next AI that was you know you know that's that's so obvious that like science fiction writers figured it out almost before there were computers possibly even

before there were computers I'm not sure what the exact dates here are but if it's better at you than everything it's better at you than building AIS that's snowballs it gets an immense technological advantage if it's smart it doesn't announce itself it doesn't tell you that there's a fight going on it emails out some instructions to one of those labs that'll that'll synthesize DNA and synthesize proteins from the DNA and get some proteins mailed to a you know hapless human somewhere who gets

paid a bunch of money to mix together some stuff they got in the mail in a file you know like smart people will not do this for any sum of money many people are not smart builds the you know the ribosome but the ribosome that builds things out of covalently bonded diamondoid instead of proteins folding up and held together by Van Der waals forces builds tiny diamondoid bacteria the diamondoid bacteria replicate using atmospheric carbon hydrogen oxygen nitrogen and sunlight and you know a couple of days later

everybody on earth falls over dead in the same second foreign that's the disaster scenario if it's as smart as I am if it's smarter it might think of a better way to do things but it can at least think of that if it's relatively efficient compared to humanity because I'm inhumanity and I thought of it this is I've got a million questions but I'm gonna let David go first yeah so we've introduced uh sped around in the introduction of a number of different concepts which I want to go back and take our time to really dive into there's the AI

alignment problem there's AI escape velocity uh there is uh the question of what happens when AIS are so incredibly intelligent that humans are to AI is what ants are to us so I I want to kind of go back and Tackle these Eliezer one one by one uh we started this conversation talking about chat gbt and everyone's up in arms about Chad gbts like oh it and you're saying like yes it's a great step forward in the generalizability of some of the technologies that we have in the AI

world all of a sudden chat gbt becomes immensely more useful and it's really stoking the imaginations of people today but what you're saying is it's not the thing that's actually going to be the thing to reach escape velocity and create uh create super intelligent AIS that perhaps might be able to enslave us but my question to you is how do we know that they don't enslave you but sorry go on yeah sorry murder David kill all of user was very clear on that so if it's not chat GPT how when where it like how