177 - AI is a Ticking Time Bomb with Connor Leahy

even if you your listener don't totally buy the existential risk thing you know maybe you don't buy it's fine but it is the case that the leaders of all the top Labs anthropic Deep Mind Over have been on the record saying clearly that they do think that there is a realistic possibility that these technologies will kill literally everybody and they're doing it anyways welcome to bankless where we explore the frontier of Internet money and internet

Finance this is how to get started how to get better and how to front run the opportunity this is David Hoffman here without my co-host Ryan Sean Adams but regardless we are here to help you become more bankless today on the episode we're talking AI alignment and AI safety once again we're talking to Conor Leahy the CEO at conjecture a mission-driven org trying to make ai go well we wanted to do one last episode on the AI alignment and AI safety conversation because we think Connor can really deliver a very compelling and

easy articulation as to why AI safety is real and why it needs to be treated as such some Main benefits and takeaways that you're going to get from this episode first the intuitive arguments behind the AI safety debate the things that you can take to your friends to convince them that AI safety is a real issue second the two defining categories of ways AI could end humanity and third the major players that are playing in the race towards AGI and why they all seem to be ideologically motivated rather than financially motivated fourth

why the progress of AI power is based on two exponential curves and lastly fifth why Conor thinks government regulation is the easiest and most effective way of buying US time here on Bank list we've had the AI alignment AI safety conversation a handful of times with different players from the industry ever since we had that Eleazar episode which we were hoping would have been an AI crypto conversation but turns out it was an AI is going to kill us all conversation we're bringing on Conor Leahy in June of 2023 a number of months

after we first went down this rabbit hole because the world of AI kind of feels the like the world of crypto in 2021 it is moving so fast and so Conor gives us the lay of the land a snapshot in time of the AI alignment conversation as it stands here in late June of 2023 as well as also articulates in the easiest and most simple terms possible why AI alignment is such a big deal if you bank with listener are not convinced of AI alignment going into this episode and you remain unconvinced after the end

of this episode I have nothing left for you and so let's go ahead and get right into that conversation with Conor Leahy but first a moment to talk about these fantastic sponsors that make this show possible especially Kraken our preferred crypto exchange for 2023 assuming that we get past the AI alignment issue we will still need to buy crypto assets using our Fiat dollars so perhaps use Kraken to get that done there's a link in the show notes to get started Kraken Pro has easily become the best crypto trading platform in the industry the place I use to check the charts and the crypto prices even when I'm not looking

to place a trade on Kraken Pro you'll have access to Advanced charting tools real-time Market data and lightning fast trade execution all inside there's spiffy new modular interface kraken's new customizable modular layout lets you tailor your trading experience to suit your needs pick and choose your favorite modules and place them anywhere you want in your screen with Kraken Pro you have that power whether you are a seasoned Pro or just starting out join thousands of traders who trust Kraken Pro for their crypto trading needs visit pro.cracken.com to get started today mantle formerly known as bit Dao is the

first dowled web3 ecosystem all built on top of Mantle's first core product the mantle Network a brand new high performance ethereum layer 2 built using the op stack but uses eigenlayer's data availability solution instead of the expensive ethereum layer 1. not only does this reduce mantle networks gas fees by 80 but it also reduces gas fee volatility providing a more stable foundation for Mantle's applications the mantle treasury is one of the biggest dow-owned treasuries which is seating an ecosystem of projects from all around the web 3 space for mantle mantle

already has sub communities from around web3 onboarded like game 7 for web free gaming and buy bit for TPL and liquidity and on-ramp so if you want to build on the mantle Network mantle is offering a grants program that provides milestone-based funding to promising projects that help expand secure and decentralize mantle if you want to get started working with the first dowled layer 2 ecosystem check out mantle at mantle.xyz and follow them on Twitter at Xerox mantle if you haven't experienced the superpowers that a smart contract wallet gives you check out and buyer and buyer works with all the evm chains that

are out there the layer 2s like arbitrum Optimum ISM and polygon but also the non-etherium chains like Avalanche and Phantom because of the power of smart contract wallets and buyer lets you pay for gas in stable claims meaning you'll never have to spend your precious eth again the web app has numerous Fiat on-ramps to make it easy to dump your Fiat for crypto and if you like self-custody but you still want training wheels you can recover a lost and buyer wallet using an email and password but without giving the ambier team any control over your funds check it out at amber.com for the web app experience but

also the ambier mobile wall is coming soon for both IOS and Android and if you want to be a beta tester you can sign up at ambuyer.com app and since you stayed to the very end of this ad read you should know that ampire is airdropping its wallet token to early users for simply just using the wallet so if you want to get started with Amber all the links that you need are in the show notes Bank list Nation I'm excited to introduce you to Conor Leahy the CEO at conjecture a mission driven organization trying to make IAI go well he's also the

co-founder founder of a Luther Ai and open source AI research non-profit laboratory which interestingly operates mostly inside of a Discord server much like our company and so many of us in the bank location Connor welcome Vanquis thank you so much for having me Connor the crypto world has more or less collided with AI and bankless we had our introduction with that surprisingly when we had our Eliezer udkowski episode which we had to Pivot mid episode from a crypto AI intersection episode into an AI is going to kill us episode since

then we've continued to go down that AI alignment rabbit hole and I think a decent number of people in the bankless nation in the broader crypto landscape except the AI alignment problem but others completely reject it and it's interesting to see some people just have a spinal reflex rejection of the AI alignment problem so in this conversation I hope we can kind of just talk about the conversation of AI alignment in the outside world the companies that are playing here the game board that is laid out in front of us

but first I really just want to dive into the very basics of the AI alignment problem and see if we can once again articulate clearly why the AI alignment problem exists what it is and why it's important and you've gotten a lot of practice at articulating this so I'm wondering if you could kind of hand hold us through some of the very basic premises behind the AI alignment problem yeah absolutely so to you know start things off the airline problem or existential risk is really what I care

about this is this is that something could be so dangerous that an accident or a misuse could occur of such magnitude that it could threaten the continued existence of all of humanity or you know curb our potential forever in some sense this is of course be extremely terrible I think this is pretty uncontroversial that if such a thing were possible and if it did happen that would be pretty terrible and so the way I like to think about this is first kind of from like an outside perspective is if we look at a

history of Technology you know if we see that technology has gotten better and better all throughout our history has given us way more power way more affordance a lot of this is great right you know we have medicine and you know I have air conditioning in my uh um office thank god um you know all these you know great things that are are wonderful and I love but also as technology increases uh as your power increases your ability to control the environment um you have more capacity for things to

go wrong or for Destruction to occur so you know back in the stone ages you know the worst possible thing that I could do with Stone Age Technology would kill you know I know maybe like 10 people or something um you know if I'm like a pretty big guy or you know maybe if I'm like super smart about it I could kill a few hundred you know if I have my whole War band with me I could kill maybe more but like an individual person just using Stone Age level tools you know keep not going to really be an existential risk or anything of that sort as our technology gets better you know we

develop you know uh more sophisticated Weaponry red develop gunpowder we develop um you know stuff like this uh the number the damage that can be caused both and on purpose and accidentally increases you know you sometimes for benign reasons you know if you have bigger ships more people can drown you know that's pretty benign we don't think that you know that's a risk that we are willing to take but in other greens you know there was no such thing as you know uh TNT factories blowing up before we

had explosives and now that we have TNT when an accident occurred the collateral damage was suddenly of a type and a degree that didn't exist previously and this is only continuing you know so we went from you know okay you can kill like Five Guys to you can kill like 50 guys to kill 500 and now you know you press the button and drop a nuke out of an airplane and you can kill 50 000 people or even more than that and so if you like would graph you know over time be like reason like blast radius of Technology of like you know a misuse of a technology or thing you would see an

exponential you would see that our technology is growing extremely fast extremely quickly towards larger and larger blast radius not all Technologies you know many Technologies are very safe and they're very good and I think we should invest in them and we should build them but there are technologies that have larger and larger blast radiuses and eventually this blast radius will Encompass Earth if our technology keeps improving at some point we will have technology that is so powerful that it can destroy Earth that it can destroy all humans and even

accidentally it would of course be easier if you do it on purpose but at some point if we have powerful enough technology we should expect things where even accents are a problem and so I would make the claim that AGI is in this in this category it's in in the category of a of our you know Trend in technology towards more and more powerful systems where even an accident during the development of the system has larger and larger brass radius and you know sometimes this is what we accept as a society we accept that sometimes you

know a clinical trial might go wrong you know this is something that sometimes we accept to some degree not to arbitrary degrees we don't let anyone do any clinical trial without any oversight of course not we do you know generally as a society we uh think very highly of you know human welfare and life and that it shouldn't be endangered recklessly but as this goes on as we're dealing with these more and more powerful Technologies we have to be very like what do you do when you have a technology where the

blast radius is everybody including you like how do you develop a technology where getting it wrong ends everything ends you ends your experiment this is not something Humanity has experience dealing with but this is not something that we are generally set up to do the way we usually do technology is we build something and we fail a bunch of times you know we we mess it up you know a few lab assistants lose their hands you know like a bunch of stupid things happen you know government gets angry at you and 20 years later you know you maybe have something

this is fine but predictably at some point this stops working me you know you maybe people could argue the time is not now and I would agree I don't expect GPT three or four to like you know kill all people I don't but you know gb5 6 7 8 you know combined with some you know modern RL agentic systems that's much less clear to me I'm happy to go into a bit more about why I think this technology has this kind of blacks radius But first you know just pause

there for a second one thing I really want to emphasize in this argument is the neutrality of it it's not saying that AI is good or bad it's just saying it's just saying that AI is and this is a continuation of the Arc of technology technology is not good nor bad you're just really putting it into very neutral terms that technology has an arc of goodness but has uh turbulence with Associated blast radiuses along the way and that we've all accepted these because the Blast radiuses has been

sufficiently small that it doesn't you know end all humans but what you're saying is that you know continue the arc get it to ai ai not it doesn't want to kill us it doesn't want to make our lives better humans will make choices with this technology but it just so happens that in the Advent of a bad outcome that bad out comes bad radio blast radius contains all of us so it's not even like a a political stance or a an opinionated stance about the goodness or Badness of AI it's merely just a statement about

the magnitude of what could go wrong if something does go wrong yeah exactly I think a lot of discourse around this kind of stuff is very bad I think a lot of it is completely politicized and like psychologized is about is open source good or bad is this guy a good person or a bad person and I think this is just a terrible way to think about this we think you should think about this very neutrally the question is not is AI good or bad it's not is uh open source good

or bad it's not is Sam Altman a good person or a bad person these aren't the intro the question is just what are the outcomes of the various choices we make how can we do our best to predict what the consequences of taking various actions are and how do we feel about those consequences how do we feel about taking these risks how risk tolerant are we who should be consulted you know I was never consulted to have you know uh gpt4 released onto the public internet which I use as well to for it to be released on my family and my friends I

was never asked if this kind of experimental new technology should be Unleashed into the comments that I also inhabit and you know maybe that's fine you know we don't generally seek consent when someone you know wants to write a book you know someone wants to write a book you know I don't have to read it not my problem but if someone is you know releasing I don't know a new you know substance into the water supply well I sure hope that I would be consulted about whether I think that's a good idea or not because I'm drinking that water but like I I feel like I have a say in this and so the a lot of the

problem with the current state of of discourse around the stuff is that there is some good technical discourse but then uh there's also a lot of discourse which focuses way too much on intentions it focused way too much on ideologies like just like you know that we can't derive from an ideology what is true you know if you believe you know open source is good you know maybe you believe that right but that's not a statement about reality it's it's a mood affiliation you know sometimes open source is great I worked a lot in open source I think in

many cases open source is fantastic I'm so glad that Linux exists I'm so glad that we have so much great open source software I think this was really good but should the genetic sequence the smallpox be public like I don't know man like you know I love science I love Academia I think it's great that people are getting funded to do all kinds of you know cutting-edge research a group of researchers in I think Canada used you know government funds to reconstruct an extinct form of horse pox or smallpox virus

and published how to do it I I think they should have done that like I I think this is bad I think people shouldn't do that I think that as much as I love Open Access science as much as I love science we have to be practical here I don't expect the upsides of this being public to be worth it so this is kind of like how I like to think about these kind of things I like to think about these things way more it's just like look what do we want like what are the scenarios what are the outcomes animals work from there let's let's work this completely neutrally

let's be just like let's be realistic here I like to talk about strategy I don't like to talk about I really like to talk about like you know good or bad if that makes any sense right there's our there's a bunch of conversations to be had here about the players in the world of AI we have chat GPT and open Ai and Sam Altman we have stable diffusion with with emad there's a bunch of there's a bunch of people in this game there's The Regulators are also in this game there's I still want to continue a little bit down the defining the alignment problem uh conversation because there's there's uh

something I want to parse apart you're talking about the sheer neutral Ai and its blast radius right what could go wrong if some if something went were to go wrong when we had our conversation with Eliezer Eleazar gave us a very strict prescription of how this will go wrong that your explanation of the AI alignment problem contains but his is more narrow and I want to parse apart that because he he calls this or we call this the AI alignment problem as in how

do we align the goals of AI with the humans uh human goals and that's a more narrow conversation than I think what you you're presenting you're just saying hey AI is powerful it could go wrong and it could go wrong for any subset of ways some of which may be the AI alignment problem but there are also other ways that it could go wrong I'm wondering if how do you parse apart the ways that are there categories of AI like um Doom that are worth parsing apart one of them's AI alignment others there's like maybe humans go Rogue and then

we've seen humans do this right like what what would happen if uh Ted Kaczynski got their hands on on a very powerful AI like that's another conversation how do you parse apart like the actual sources of Destruction here yeah so like the I like to like use like two maybe three or four categories is that the first one is misalignment it's just you don't have control just lack of control as saying does something random that's it and like there's no you know doesn't even need a human to be involved right and then that random things blast radius contains us is what you're saying

yeah yeah so the claim is that a accidents can happen and that accidents can have blast radius of this side so there's like accidents the second one is misuse so misuse is you have a system which does what you say and someone tells it to do something bad or for there to be conflict of some kind you know maybe there's multiple actors who go to war using this kind of technology or like fight and then the in the as you can I can barely imagine something more horrific than multiple you know super intelligent systems fighting like could

you imagine like it's as unimaginable what Horrors star systems could unleash in the terms of war and in that scenario that scenario is like kind of interesting because it implies that the AI alignment problems actually solved as in we've been actually able to align human humans and AIS together to achieve the same goals but human alignment is not solved and so using our AI superpowers we now commit war and our war blast radius once again contains everyone exactly so it's like it's it's

even worse than that it's like there's so many ways in which this problem is very very hard we are this is not a super narrow specific little problem there are specific narrow aspects to problem-specific narrow technical problems which are very very important and we can talk about those but importantly this is a generalized problem of society this is a general's problem of The Human Condition is a general problem of how do we responsibly deal with powerful technology this is the meta problem that we have to actually have to solve and this is a

problem that people have been talking about at least since like World War One you know there is a Polish Nova man named after krzy who after the horrors of World War One noticed in this like 1920s he was like wait if technology keeps increasing but our wisdom and our you know our control as a society increased much slower well then all Humanity will end he figured out X risk in the 20s and yeah and he wrote and so his solution was he had a math he had to figure out how to improve the art of human rationality sound familiar

so there were there was an Eliezer in the 1920s actually named after zipski so this is not a new thought this is not something that you know I'm sure people before Alfred has also come up with variations on this thought and this is continuing this problem and we have not yet solved this problem we are already in the problem you know the last time we had a new level of powerful technology you know we did drop two nukes on purpose and after that there were several really really close calls where you know nukes almost did stop Flying

you know at least two where it was just one single person each time who stopped the nukes from actually getting fired so our track record here is like decent but it's not good it is not good and it's going to be much worse when we're dealing with agis and aeon especially like imagine if these things are open source imagine if every person in the world had a nuke during the Cold War I expect we would have gotten nuked you know I expect it wouldn't have gotten well if we didn't have chains of command you know several people signing

off on something if we didn't have sensible people that took their responsibility extremely seriously I think it just would have not gone well and the way things currently are there is currently more regulation on selling a sandwich to the public than there is to building an unprecedented AGI level technology and releasing it to the general public there is no oversight there's no you know process you know General processes here is there controls or like you know stakeholders or something is nothing it's it's just

these private companies it's a small number of private companies to be clear about this it's basically currently mostly just Deep Mind anthropic and open Ai and a few others who are like trying to catch up that are really pushing forward to these high-end AGI level dangerous Technologies and erasing completely out of control so like how would you expect this to go well at this current Pace it's like it's like we're in the worst possible scenario we could

be in kind of I just want to pin down the categories that we were talking about I I interrupted you and just want to make sure that we clearly identify them the categories of how AI progress would go wrong one of them is misalignment this is the Eleazar conversation the pay-per-clip maximizer conversation we create the super intelligent Ai and then we can't figure out how to harness its goals and align them with humans and so it accidentally turns us into paper clips that's one category another category that you defined was misuse

um we do somehow do figure out the AI alignment problem but we just abuse AIS to kill us all so one super power fights with another superpower one of them has AI maybe both of them have ai and then we all die because we're using AI to have misaligned human uh goals so that's a misuse another one is accidents we have super powerful AI it accidentally does something that we don't like and we're inside of that blast radius that's a third category are those all of them or are there others that are worth unpacking yeah I mean I think you can

move even fold accidents into misalignment in a sense is that in a line a truly aligned system if you tell it to shoot you in the foot it will say no that's not what you intended and you know bring you flowers or something if you have a truly it's like there's like maybe like three categories is or like maybe four is like you know I like a good hierarchy maybe four categories technology there's so dangerous that no one should do it you know just like there is nothing like you just you turn it on it blows up everything then there's like technology which is uh

which is controllable if you're really careful if you're very sensible if you're very careful it's fine then there is technology that is uh safe for General use except if you misuse it except if you're specifically trying to do something bad and then there is technology that is good no matter who uses it so that you could give it to the most evil sociopath in the world and it's fine okay and AI I can see AI fitting into all those categories exactly I think we will start at one and

then we can develop technology to move to two and then we can develop technology to move to three and then we can develop technology to move to four but by default we get the category one AI uh by default we Elevate AI as it progresses finds itself inside of the category of Technology you almost never ever want to even open up at all yes and this is what I expect the the like the shortest path to an AGI gets you this type of AGI a system which is misaligned you know paperclip maximizer has some random values it's very intelligent very

capable it's very deceptive and if you just turn it on it doesn't matter who turns it on it doesn't matter if this is you know the USA or China it doesn't matter if it's open AI or anthropic it doesn't matter who does it it just blows up everything there is no it doesn't matter you said the shortest path to get to a super intelligent AI can you unpack why you emphasize the word short there what what does what is implied under a longer path to an AI what does that mean so importantly this is not a rule of nature there is no law of physics which states it is impossible to have a safe

AI that does good things for you this is completely Allowed by the laws of physics and computer science we just don't know how to do it and this is a very narrow Target then as a each of these you know categories one to four are like a error and narrower and narrower Target you need to know more and more about control about intelligence about you know safe practices about human psychology about like you know about you know values and Game Theory and whatever to to narrow down on these more and more complex

systems so by default if you just want to think which is just smart it's just powerful it just succeeds at goals well you know just you know big larger models you know just throw computed it man you know just you know continue doing what we're doing right now the basically all the research currently being done at AI companies is of the kind which gets you to type one there is a very little research that goes into getting you to type two three or four this is a great a great quote it's from Wilbur Wright so this is the

uh one of the brothers who built the first airplane and what he said was is that when Once the machine is under proper control under all conditions the motor problem will be quickly solved a failure of a motor will then mean simply a slow descent in safe landing instead of a disastrous fault so this is the man himself built an airplane who realized the first step to building a good airplane was to solve the safety problem it was to build a safe glider that if something goes wrong it carefully slowly descends instead of killing the pilot because before this there was a lot of attempts at flying machines and they

always killed the pilot so they couldn't develop them yeah he recognized that we had to First build a safe glider and then we can worry about the motor part you know and he even though it's like almost dismissive of the motor he was like ah you know you know we'll figure our motor it's not that big of a deal and so basically all of AI currently works on engines they work on Motors there's very few people working on gliders and there's a basically everyone is working feeling bigger and bigger turbo jet engines and you know just like as fast as possible quick as possible and if you get a bigger and bigger turbo

jet engine without improving your glider design without improving your wing control design without in you know doing the necessary experience for this by default you get an A type 1 engine you get an engine that just explodes it just like you know zooms off into space and you know just like does something stupid and you know if it's like an aircraft engine the risk the blast radius is contained as before but AI is not just an airplane engine right yeah the AI engine is focused on elevating Humanity to as high of an elevation as possible and then all of the AI safety uh people

are like hey we also need to make sure that we have a spaceship that is containing us that can take us back down in the inevitable case that eventually that that engine Runs Out Connor I'm hoping I can just get your lay of the land of the state of the AI alignment conversation because as as we've been saying as you've been talking about AI progress seems to be moving really fast uh is now June 2023 and it kind of seems that we have to time stamp these podcasts because if we were doing this podcast in just may or April of the same

year it would be a slightly different conversation about what is the state of the AI alignment conversation uh last we checked there was this letter signed by many world leaders and AI experts asking for a pause on AI progress Beyond Chad 2bd4 I'm wondering like oh if you can just update us on the last few months in the mainstreaming of the alignment problem like is there more reasons to be optimistic are people burying their heads deeper in the sand like kind of where is the world with regards to AI alignment and AI safety yeah I mean what

and has the world changed the world's changed insanely over the last you know six to 12 months I mean to say it lightly uh AI Lyman has gone to a large degree mainstream I just earlier today was talking to a member of parliament who didn't even have a smartphone didn't know what open AI was but he heard about his AI thing and he wanted me to tell him about it and uh he got quite Furious when I explained to him some of those risks and he's like no one taking this seriously what this is outrageous like of course we have to do something about

this um so it's quite fun um for me it's also been a very enlightening experience so when I first got into this field you know I came in from like a pretty classic you know kind of like less wrong you know Eliezer adjacent kind of viewpoint very technical very nerdy very philosophical perspective on things um not much and there's a lot of weird social memes in like that sphere around politics that like politics is bad you should never do it don't you know don't talk to the public don't talk to the government

they're all crazy can't talk to them and I feel it's completely gaslit because that's just not true uh don't get me wrong politics is hard you know politicians are have their incentives blah blah blah all these things are true but this is things you can do you can talk to them a thing that's just been incredibly positively shocking to me is that when I talk to normal people who don't work in Tech they really get it I'm so used to talking to people in Tech and they just totally dismiss these things you're like no yeah I can never do anything bad you know blah blah blah I can't hear you and I talk to normal