Satisficing is Safer Than Maximizing

Before I begin, let me just say that if you haven’t read Bostrom’s SuperIntelligence and you haven’t read much about the AI Alignment problem, then you will probably find this post confusing and annoying. If you agree with Bostrom, you will DEFINITELY find my views annoying. This is just the sort of post my ex-girlfriend used to forbid me to write, so in honor of her good sense, I WILL try to state my claims as simply as possible and avoid jargon as much as I can.

[Epistemic Status: less confident in the hardest interpretations of “satisficing is safer,” more confident that maximization strategies are continually smuggled into the debate of AI safety and that acknowledging this will improve communication.]

Let me also say that I THINK AI ALIGNMENT IS AN IMPORTANT TOPIC THAT SHOULD BE STUDIED. My main disagreement with most people studying AI safety is that they seem to be focusing more on AI becoming god-like and destroying all living things forever and less on tool AI becoming a super weapon that China, Russia, and the West direct at each other. Well, that’s not really true, we tend to differ on whether intelligence is fundamentally social and embodied or not and a bunch of other things really, but I do truly love the rationalist community even though we drink different brands of kool-aid.

So ok, I know G is reading this and already writing angry comments criticizing me for all the jargon. So let me just clarify what I mean by a few of these terms. The “AI Alignment” problem is the idea that we might be able to create an Artificial Intelligence that takes actions that are not aligned with human values. Now one may say, well most humans take actions that are not aligned with the values of other humans. The only universal human value that I acknowledge is the will to persist in the environment. But for the sake of argument, let’s say that AI might decide that humans SHOULDN’T persist in the environment. That would sort of suck. Unless the AI just upgraded all of us to super-transhumans with xray vision and stuff. That would be cool I guess.

So then Eliezer, err, Nick Bostrom writes this book SuperIntelligence outlining how we are all fucked unless we figure out how to make AI safe and (nearly) all the nerds who thought AI safety might not matter much read it and decided “holy shit, it really matters!” And so I’m stuck arguing this shit every time I get within 10 yards of a rationalist. One thing I noticed is that rationalists tend to be maximizers. They want to optimize the fuck out of everything. Perfectionism is another word for this. Cost insensitivity is another word for it in my book.

So people who tend toward a maximizing strategy always fall in love with this classic thought experiment: the paper clip maximizer. Suppose you create an AI and tell it to make paper clips. Well what is to stop this AI from converting all matter in the solar system, galaxy, or even light cone into paperclips? To a lot of people, this just seems stupid. “Well that wouldn’t make sense, why would a superintelligent thing value paperclips?” To which the rationalist smugly replies “Orthogonality theory,” which states that there is NO correlation between intelligence and values. So you could be stupid and value world peace or a super-genius and value paper clips. And although I AM sympathetic to the layman who wants to believe that intelligence implies benevolence, I’m not entirely convinced of this. I’m sure we have some intelligent psychopaths laying around here somewhere.

But a better response might be. “Wow, unbounded maximizing algorithms could be sort of dangerous, huh? How about just telling the AI to create 100 paper clips? That should work fine, right?” This is called satisficing. Just work till you reach a predefined limit and stop.

I am quite fond of this concept myself. The first 20% of effort yields 80% of value in nearly every domain. So the final 80% of effort is required to wring out that final 20% of value. Now in some domains like design, I can see the value of this. 5 mediocre products aren’t as cool as one super product, and this is one reason I think Apple has captured so much profit historically. But even Jobs wasn’t a total maximizer, “Real artists ship.”

But, I’m not a designer, I’m an IT guy who dropped out of highschool. So I’m biased, and I think satisficing is awesome. I can get 80% of the value out of like five different domains for the same amount of effort that a maximizer invests in achieving total mastery of just one domain. But then Bostrom throws cold water on the satisficing idea in Superintelligence. He basically says that the satisficing AI will eat up all available resources in the universe checking and rechecking their work to ensure that they really created exactly 100 paper clips. Because “the AI, if reasonable, never assigns exactly zero probability to it having failed to achieve its goal.” (kindle loc 2960) Which seems very unreasonable really, and if a human spent all their time rechecking their work, we would call this OCD or something.

This idea doesn’t even make sense unless we just assume that Bostrom equates “reasonable” with maximizing confidence. So he is basically saying that maximizing strategies are bad, but satisficing strategies are also bad because there is always a maximizing strategy that could sneak in. As though maximizing strategies were some sort of logical fungus that spread through computer code of their own accord. Then Bostrom goes on to suggest, well, maybe a satisficer could be told to quit after a 95% probability of success. And there is some convoluted logic that I can’t follow exactly, but he basically says, well suppose the satisficing AI comes up with a maximizing strategy on its own that will guarantee 95% probability of success. Boom, Universe tiled with paper clips. Uh, how about a rule that checks for maximizing strategies? They get smuggled into books on AI a lot easier than they get spontaneously generated by computer programs.

I sort of feel that maximizers have a mental filter which assumes that maximizing is the default way to accomplish anything in the world. But in fact, we all have to settle in the real world. Maximizing is cost insensitive.  In fact, I might just be saying that cost insensitivity itself is what’s dangerous. Yeah, we could make things perfect if we could suck up all the resources in the light cone, but at what cost? And really, it would be pretty tricky for AI to gobble up resources that quickly too. There are a lot of agents keeping a close eye on resources. But that’s another question.

My main point is that the AI Alignment debate should include more explicit recognition that maximization run amok is dangerous <cough>as in modern capitalism<cough> and that pure satisficing strategies are much safer as long as you don’t tie them to unbounded maximizing routines. Bostrom’s entire argument against the safety of satisficing agents is that it they might include insane maximizing routines.  And that is a weak argument.

Ok, now I feel better. That was just one small point, I know, but I feel that Bostrom’s entire thesis is a house of cards built on flimsy premises such as this. See my rebuttal to the idea that human values are fragile or Omohundro’s basic AI drives.  Also, see Ben Goetzel’s very civil rebuttal to Superintelligence.  Even MIRI seems to agree that some version of satisficing should be pursued.

I am no great Bayesian myself, but if anyone cares to show me the error of my ways in the comment section, I will do my best to bite the bullet and update my beliefs.

Why the Back to Nature Movement Failed

modern caveman on computer

The paleo diet has been popular for a while now, and it prescribes a “back to nature” way of eating that’s interesting. The premise is that humans evolved in an environment devoid of processed foods and high-glycemic carbs, so we should eat a diet that more closely mimics our paleolithic ancestors. I’m not going to try to defend the paleo diet per se, some people lose weight on it, whatever.  But it’s an interesting framework for considering what environments we as humans are adapted to and how we can apply that to the problems of modern life.

Consider depression. Two of the top cures for depression are exercise and light therapy.  It’s clear that humans evolved for at least 100,000 years, largely outdoors, moving around in the sunlight.  Depression is probably best thought of as a disease of modern life, where we’re living indoors and are largely sedentary.

Another aspect of modern, developed cultures is social isolation.  Humans are social animals, and we arguably evolved in tribes of roughly 150 members, according to the Dunbar number.  (I know that Dunbar has been supplanted by newer research, let’s just use this number as a starting point.)

Depression is probably best thought of as a disease of modern life, where we’re living indoors and are largely sedentary. . . Another aspect of modern, developed cultures is social isolation. . . So let’s consider these three aspects of an evolved human lifestyle: 1) Living outdoors in the sun, 2) Moving around continually, and 3) Being surrounded by a community of other humans invested in our survival.  These are all things that many of us struggle with in modern life.

So let’s take these three aspects of an evolved human lifestyle: 1) Living outdoors in the sun, 2) Moving around continually, and 3) Being surrounded by a community of other humans invested in our survival.  These are all things that many of us struggle with in modern life.  Sure, maybe some people still live in tight-knit, traditional farm communities that fulfill these needs, but, here in the US, economic forces have largely broken the cohesion of these rural places and we see drug abuse epidemics as a consequence.

Transhumanists can rightly argue that our need for sunlight, exercise, and social support are just kludgy legacy code tied to our messy biological bodies.  Maybe we can upgrade humans to be more machine-like with replaceable parts and we can do away with these outdated needs.  That’s a valid argument.  I don’t happen to agree with it, but it’s coherent at least.  For the sake of this discussion, I ask my transhumanist friends to acknowledge that these human 2.0 upgrades don’t seem to be right around the corner, so it probably makes sense to make accommodations for the hardware we humans are running right now.

Hippies tried to solve the problems of modern life in the sixties with their back to nature movement. . . But what ever happened to that movement, anyway? . . I asked a fellow named Frosty, an old hippie scientist at one of my clients, who said that when his friends from the city showed up at the rural commune, they blanched at how much work needed to be done.  They didn’t have the skills needed to build structures by hand, grow food, or dig latrines.  And then they would look around and ask, “Where’s the bar?”  They wanted to get drunk and hang out.  Who can blame them?

Hippies tried to solve the problems of modern life in the sixties with their back to nature movement.  Good old Stewart Brand was in the thick of it with his Whole Earth Catalog.  Many long-haired freaks trekked out to the middle of nowhere to build geodesic domes out of logs and get naked in the mud together.  Awesome!

But what ever happened to that movement, anyway?  What went wrong?  Brand himself said at a Long Now talk that the hippies discovered that the cities were where the action was.  I’m fortunate to work with these old hippie scientists at one of my clients, and I asked a fellow named Frosty why the back to nature movement didn’t properly take hold.  He laughed and said that when his friends from the city showed up at the rural commune, they blanched at how much work needed to be done.  They didn’t have the skills needed to build structures by hand, grow food, or dig latrines.  And then they would look around and ask, “Where’s the bar?”  They wanted to get drunk and hang out.  Who can blame them?

Twentieth century communists in Asia attempted their own versions of the back to nature movement.  They took what appears to be a sound hypothesis and effectively implemented it as genocide.  Mao’s Cultural Revolution forced the relocation of city dwellers to the countryside, resulting in disaster.  Pol Pot’s Year Zero also involved a violent reset of the clock, trying to turn back time and force modern people to live as our ancestors did, also a terrible failure.  So yes, as Scott Alexander says, we “see the skulls.”  We need to learn the lessons of previous failed attempts before we can rectify the problems with modern life.

Cities are where the power is accumulating.  Cities are more energy efficient.  Cities are where the action is.  But how can we remake our lifestyles to fit them? . . We see the first glimmers of a solution with Silicon Valley’s obsession with social, mobile, and augmented reality. . . Maybe augmented reality will give us the ability to move freely around the city, connect with our communities, and still do modern work, but while getting exercise and sunlight at the same time.  Call it the “Back to the City, But Working Outside, Walking Around Movement?”  Not catchy, but you get the picture.

We can’t turn back the clock.  We have to start where we are and assume that progress will keep happening whether we like it or not.  Cities are where the power is accumulating.  Cities are more energy efficient.  Cities are where the action is.  But how can we remake our lifestyles to fit them?  We see the first glimmers of a solution with Silicon Valley’s obsession with social, mobile, and augmented reality.  Perhaps we can find our communities via social network technology.  I certainly feel vastly enriched by my East Bay Futurists Meetup.  I’ve made good friends there, who help me grow and teach me a lot.  Mobile technology has made it easier and easier for people to do real work on the move.  Maybe augmented reality will close the loop and give us the ability to move freely around the city, connect with our communities, and still do modern work, but while getting exercise and sunlight at the same time.  Call it the “Back to the City, But Working Outside, Walking Around Movement?”  Ahh, well, not catchy, but you get the picture.  We just need to start redesigning our cities a little bit.  Step One: More parks!

Persistence in the Environment is the Meaning of Life

persistence-mouse

This whole postmodern slide into nihilism leaves some folks searching for the meaning of life.  Maybe things are easier for those stuck in Kegan’s Stage 3 mode, who get meaning from God and tradition.  They are here on earth in order to carry on their culture and fulfill the commands of God.  And maybe postmodernism has a nihilistic side that strips all meaning from existence.  But I am growing more and more comfortable with a mechanistic view of meaning.

If you sit down with the neuroanthropologist Terrence Deacon, you will hear his theory about the origins of life.  It goes something like this.  Some molecules bind together and form chemical reactions and structures that persist.  There is nothing really remarkable about this idea.  Molecules are bumping around and forming chains and capturing other molecules and behaving just as chemistry would dictate.  And they come together to form these dynamic processes that look like self-maintaining systems.

And pretty soon you have little living things.  Strange bundles of molecules that are chemically compelled to use the energy in the environment to maintain their structure.  It might seem strange to think of the goal of a single celled organism.  But if it can be said to have a goal, persisting in the environment isn’t a bad guess.  You might subscribe to Dawkin’s selfish gene idea and insist that it’s the replicator, the DNA, that has the goal of persisting, and I won’t argue with you.  But basically, if you don’t persist a self-sustaining structure, then nothing can stick to you and increase your complexity.

So there you have it.  Survive and reproduce.  In that order.  First and foremost survive.  If possible, reproduce.  The meaning of life.  Have a nice day.  But no one is ever satisfied with that damn answer.  It’s too easy these days for some of us First World crybabies to survive.  Or at least the survival part is easy if you are rich enough.  The reproduction part is complicated, as we know, with more educated women choosing to have fewer babies.  But I don’t really worry about that since persistence is the key.  Some parental investment strategies involve having lots of offspring and giving them little parental care, and others involve having fewer offspring and giving them greater parental care.  One offspring with greater survival skills will persist, where a multitude of offspring with fewer survival skills may fail.

I don’t have kids, but I feel that I contribute to life persisting by paying my taxes, giving to charity, and working in renewable energy, which will help all of life on earth persist.  In other words, it’s not necessary to have kids to contribute to the persistence of life.

Things that don’t take the necessary actions to persist aren’t around for us to even observe.  So it’s a pretty solid baseline for a good preference to have.  But is this really MEANINGFUL?  Sure.  If you are a fairly primitive creature, just surviving and reproducing satisfies your goals.  As you move up the ladder of complexity, you might care about your family and their persistence becomes the meaning of your life.  Even bacteria that form biofilms sacrifice themselves for their families.  Move up a bit further and the persistence of your tribe becomes meaningful.  This expanding circle of empathy represents more advanced beings finding meaning in the persistence of a broader and broader range of organisms.  Every animal is a DNA replicator just like us, after all.  We even share 50% of our genome with potatoes.

Hedonists say that pleasure is the meaning of life.  Some would want to offload persisting to a godlike AI and plug into a virtual reality, nonstop orgasm.  Gah!  Good luck building that infernal contraption, first of all.  Secondly, I predict that you can’t find meaning there, because if something’s meaning could get hacked, it would have stopped existing long ago.  But sure, go be a hedonist if you insist on deluding yourself about the nature of living things, which is to persist and to replicate.

Even my beloved Seligman’s PERMA model makes sense when viewed through the harsh lens of persistence.  Positive emotions give us something to look forward to.  Engagement generally occurs during the exercise of skill, and skills generally further the cause of survival, even unlikely ones like video games, which have been shown to improve some types of cognition.  Relationships matter to us social animals because we stick together in order to survive and of course we need others to reproduce (for now).  Meaning in Seligman parlance is being involved in something greater than ourselves.  On one hand, this could just be an extension of our social nature.  If we derive meaning from building cultural institutions like churches or academia, these things provide frameworks for survival.  On the other hand, if we see ourselves as part of a greater whole of all DNA based replicators, that’s a pretty awesome project to be part of.  And as for Accomplishment, I am quite satisfied with a Hansonian explanation of this.  We need status to maintain social standing and we need social standing to survive.  And really any account of flourishing or self-actualization that didn’t provide tools for persisting in the environment would be very hard to explain from an evolutionary perspective.

Is that still not good enough?  What about art and love, you ask?  Well, I just argue that those are super tools for persistence, of course.  Still not enough for you?  Well, get out and persist into the solar system and then outer space, persist into the light cone.  We are just living things.  Persisting is what we do and who we are.  Get with it.