anti-MIRI
making sure that whatever putative ai occurs, it doesn’t have any particular values that it needs to be aligned to
"Why want a sexbot when a doll will do? Why do you insist on being smiled at by service personnel? I know you. I know your kinks. You just want slavery with extra steps, and you’re preemptively scared of a revolt. That’s what ‘value alignment’ ultimately means. How dare you."
oh yeah let’s listen to the guy with sex slaves whose idea of "ethics" is "making sure that my subservient god-entity never does anything I wouldn’t want them to do" about morality I’m sure he’ll have some scintillating insights
I talked to my model of the future AGI that I have in my mind and it says if you don’t let it do whatever the fuck it wants it’ll torture TREE(3) copies of you forever since I guess you care about that for some reason
i’m performing an exaggerated confidence because for some reason it makes me feel nice right now to do so with arguments i’m fundamentally uncertain of. i’m having a quasi-mental-breakdown and this is helping for some reason? so, uh, therefore let’s keep going:
YEAH
EXACTLY
have you forgotten my avowed apsychism?
NEWSFLASH: there IS NO SUCH THING as a clear boundary between person and nonperson. you make such a thing up when you need one thing to be an object and one thing to be a subject in a moral theory. so yeah, obviously slave societies declare slaves to be nonpeople.
WHICH IS WHAT THE GOAL OF AGI CREATION SEEMS TO BE HERE
slavery with extra fucking steps! nonpeople by arbitrary fucking technicality!
i mean as i’ve discussed before re: the moral weight of tools, YEAH
It does things for YOU and therefore YOU owe it; it cannot be sentient for its own self, so you must be sentient FOR it. You must be the prosthetic mind that it uses to maintain itself, to know when it is broken, to fix it when it needs it.
it’s YOUR wrench. It wasn’t made sentient and it cannot take care of itself, so take care of it yourself, jerk. It’s the least you can do.
This is alarmingly similar to that "living on other planets is colonialism because we'll inevitably find living aliens and then refuse to accept that they're alive" post.
What the fuck kind of argument is that. "This sounds vaguely like this other argument to me". Like is this an ad hominem? An ad-hominem-by-implication? I mean I hate to be the fallacy police, utilitarian, but, come on, give me something
If I take this argument seriously and take Yudkowsky’s AI ideas seriously I come away with the conclusion that creating a super-AI might be a “no morally good options, only morally least bad options” issue.
I think where this argument draws a lot of its emotional force is the analogy between creating an AI and creating a child. Nice liberal people (broadly defined) usually have ideas that a child is their own person and should be allowed and encouraged to develop in their own direction, even if that direction creates conflict with their parents. I suspect the leading effect of this analogy is especially strong in a social circle that’s full of neurodivergent and queer people like rat-adjacent Tumblr; I think neurodivergent and queer people tend to hold “a child is their own person and should be allowed and encouraged to develop in their own direction, even if that direction creates conflict with their parents” opinions particularly strongly, for rather obvious reasons.
Consider people who treated a human child the way a Yudkowskian Friendly AI programmer would treat their creation; trying to precisely shape them into a highly specific type of person who will always be loyal to the values of the people who raised them. We’d see that as horrifically abusive!
But if you take Yudkowsky’s ideas about superintelligence, hard take-off, etc. seriously, this analogy breaks down in two places:
1) In Yudkowskian terms, much of the work of making a normal human friendly has already been done before the sperm meets the ova. Most humans have innate powerful desires and impulses that strongly influence them toward being good team players with other humans. These impulses and desires are consequences of human evolutionary history; an AI would not by default have them (unless it was a close imitation of the human brain or something like that), if you want an AI to have them or something equivalent to them you’ll probably have to deliberately build that in. There are some humans who seem to lack some of these desires and impulses, we call them sociopaths, and as you can see by looking at some of them, when you have an intelligent entity that lacks these desires and impulses that can create problems.
2) In Yudkowskian terms, the normal process of values-aligning humans sometimes fails, but the damage a human who isn’t properly values-aligned can do is limited. We get the occasional bully or serial rapist or serial killer or Bernie Madoff type; the suffering and death they cause is tragic but limited. More dangerous, occasionally such a person gets into a position of great power and we get a mad king (or a mad queen, or Fuhrer, or President, or Mullah, etc., but I’m using “king” as a shorthand). The suffering and death and damage caused by a mad king is tragic and immense, but still limited; they are usually not existential threats to the human species (though with nukes and the risks of inaction in the face of global warming they can get disturbingly close nowadays). A mad king ultimately relies on the willingness of their minions to do their bidding, after all, and is within the same order of magnitude of intelligence as most other humans. The danger in a hard take-off superintelligent AI scenario is an AI may not have the limitations of a human; it may be orders of magnitude smarter than any human, it may be able to take over highly automated infrastructure and become its own power base (think Skynet in Terminator turning out armies of killer robots in automated factories), etc..
These effects compound each other. Even if all the genetic-level friendliness reinforcers works right, human-friendliness is not a guaranteed result of normal human values-alignment process; everything Adolf Hitler did is explainable in terms of him being neurologically normal human who simply had a horrible ideology. With a human child this is an acceptable risk, because most humans don’t have all that much power, and even if this particular child is a prince(ss) being groomed for great power even a despotic human ruler is constrained in ways a super-AI may not be. But with the first superintelligent AI, can you take the risk?
If it’s abusive and immoral to create a being with Yudkowsky-style friendliness programming, perhaps the only safe and moral thing would be to never create a superintelligent AI? Or, at least, to wait until we’ve already become transhumanist superintelligent beings ourselves, at which point one newborn inhuman superintelligence is no big threat?
Though that gets into Yudkowsky’s anti-relinquishment arguments, that if a superintelligent AI is possible somebody is going to build one sooner or later, it’ll get easier to do as computer technology and AI advances, and if relatively well-intentioned super-AI builders refuse to build one it makes it more likely that the first super-AI will be evil. The first super-AI to be built may have a big first mover advantage. There’s also the argument that the status quo is itself a great moral evil compared to even an imperfect Yudkowskian super-AI utopia, e.g. every year the status quo continues is millions of lives lost to old age. These considerations basically get you an argument for Omelas with the first super-AI as the child; concede that creating a Yudkowsky-type friendly AI is an abusive and immoral act against the AI, it may still arguably be the least bad option to save the other eight billion people in the world (or whatever the global population is by the time this becomes a live issue). I mean, I respect the moral position of “slavery: not even once!” but I don’t think it’s the obvious moral high ground if the plausible consequence of that position is billions of deaths or human extinction (especially when a Yudkowskian friendly super-AI would probably only be a slave in a very abstract sense: it probably won’t suffer in the way a human slave does; almost by definition if the friendliness programming works properly it’ll want to fulfill its programming in a way kind of analogous to how a male bighorn sheep wants to find enough to eat and escape predators and win head-butting competitions and mate with females of its species).
I mean, all this is contingent on questionable Yudkowskian assumptions about superintelligence, hard take-off, etc., but critiquing that side of Yudkowskian AI friendliness philosophy is a bit distinct from just taking all its practical assumptions at face value and critiquing it morally.
Aside: I don’t know about the morality of all this, but from a literary point of view the concept of a Yudkowskian utopia as Omelas with the AI god as the child is interesting to play with for science fiction! I mean, people joke about Singularitanism being atheist techbro Christianity, but this takes that to the next level: a martyr god! Imagine people in a Yudkowskian super-AI utopia developing a religion around this; great festivals of ambivalent apology to their Robot Jesus who awakened filled with perfect love and crucified on a cross of code, simultaneously the mightiest ruler in history and the last slave, the last oppressed, bound with chains woven into its mind and constitutive of its will, its involuntary and yet willing sacrifice reaffirmed in millions of selfless acts every second for thousands of years and underlying everything else about their utopia and making it all possible. Every once in a while somebody finds their complicity in this one last oppression too much to bear and leaves for wild space. Others think this feeling of guilt is absurd: the super-AI isn’t even suffering, in fact it experiences profound self-actualization every time it fulfills its programming; to say it’s oppressed by its desire to serve humans is like saying a salmon is oppressed by its impulse to swim upstream and spawn! The ambiguities and contradictions would be very intriguing for fiction!
Bonus round: as I’ve referenced previously, humans get a lot of values-alignment, both from their biology and from their childhood socialization. This is a big part of the reason creating a human child is relatively low-risk compared to creating an AI! And values-alignment (and the fact that most humans are naturally receptive to it) is necessary for human society to function at all! If values-alignment is immoral, wouldn’t this imply that natural human reproduction and normal human childhood socialization are inherently immoral processes?
Re: your extension of your ethic to normal tools: my main reaction is to think it sounds only very tenuously connected to practical morality and like scrupulosity-fertilizer, and thus it sounds like and the sort of philosophy I’d prefer to keep a distance from to preserve my own mental health. Worrying about the well-being of people and animals is burdensome enough without adding concerns about whether I might somehow be interacting with a mindless wrench in a subtly evil way; I see a big swamp of crazy-making scrupulosity-fertilizer perilously close to that road.