I think it’s clear up front that, as humans, we are kept alive by a complex web of needs. Various four-siren alarms are going off all the time to stop us from killing ourselves, from a balance alarm to stop you when you’re going down the stairs too fast, to a relationship alarm when things are not going well between you and a significant other. These needs, and their alarms or what have you, are not something we are always consciously aware of, or are able to even consciously express. You may get a strong dislike for the colour turquoise around the time you’re 18, only to discover later that you just really wanted to move out of the house (sorry Mom). Whatever the needs, and their respective alarms are, they fit nicely into a framework that keeps us in some form of a state that helps us do whatever it is that we do. A bit of a tautology there. There are no explicit goals set out for us, and even obvious ones, like reproduction or spreading forth or happiness are ill-defined. Happiness can be produced from much sadness, in one individual or in groups of people. People who are unable to have children can devote their time to creating works of art, which is higher risk, but may yield greater reward in terms of spreading your seed, if you want to get out the economic rulers. One of my favourite guidelines, courtesy of The Jeff and Casey Show, is creativity above all else: “the proliferation of new combinations of stuff”. That should be the most important thing. But whether you agree with me about that is immaterial for what I want to talk about.

As we grow and change as individuals and groups, we develop greater means to make ourselves happy and serve our needs. We are also better able to articulate those needs (accurate or not). I am able to construct a machination that provides me with unlimited amounts of beer. But that’s a bodily thing, and has been frowned on as “gluttony” for a very long time, so let’s skip past that to mental needs.

And desires. Or whatever. I’m terrified of how much I am able to endlessly mentally stimulate myself these days. And many people do. But that’s not JUST it. That’s not just what I want to talk about. Surely, anything in excess is bad. The most obvious, pop-culturific examples I could give are from MMORPG’s such as World of Warcraft, which are tuned to be the exact type of addictive mental stimulant that you cannot leave. Many modern video games are designed like this. I am still surprised that a game can get away with proudly describing itself as “addicting”. But. I didn’t really want to talk about games, because that is an over-simplification of what I am really trying to express.

We are able to shape the world around us. Physically, yes, and we become more capable we can do so more and more. Physically. And mentally, we are able to construct things that are soothing, titillating or exciting. Exactly things so designed, thanks to The Scientific Method. We can, in principal, invoke a certain response or emotion in a certain amount of the population. It is good that we are able to do this – this should not be scary stuff, and it really is not surprising. However, I am concerned how this pertains to The Structure of the World. Why do we choose to exist, why do we go about our day-to-day lives, why do we put up with the shit parts of our lives, and how can we be satisfied (or whatever) in the end? We put up walls between us and the great outdoors that change our perception of the world around us so that is shapes itself in a way that is more expected, more attainable, more easy to work with, in some capacity, but there is still us and there is still the world and when we change ourselves with the raw ability that we can change ourselves I’m worried something is going to get all Ouroboros and eat itself in one giant loop.

This may have happened already.

I guess what I am afraid of is a singularity of values. I hate to use the s-word because it is almost a cliche in modern philosophical circles, but what I am curious about is what happens when we are able to modify our own values, specifically and exactly. What then? I wish I had better willpower when it comes to Getting Work Done. What if I was able to make that happen? Who am I then? Was that actually what I really, truly wanted and needed to be a better me, or do I have to much trouble Getting Work Done because my work is shit and I need a change of scenery?

I mean, we change ourselves overtime for the better. Certainly. If you can imagine it this way, your values at time step n determine who you attempt to become at time step n+1. But what is the difference between step n and step n+m? Can n+m look back at n and recognize them? Is n+m who n wanted to b?. Does n, in the past, have a right to care whether or not their wishes were fulfilled? This sounds a great deal like the metaphor of Schild’s Ladder (Greg Egan wrote a book about this). While we change greatly over time, we can be sure that there is some connection of identity because, while over a large time period there are large jumps, each change from version to version of a person is a tiny step.

Dustin’s career aspirations, age 12 to 23:

I want to be an astronaut
I decide to study engineering
I realize that isn’t enough and I decide to study physics
I notice the beauty and elegance of certain structures and decide to study math
I wish I could express myself better so I start learning how to program computers
I am frustrated with the lack of human values in mathematical concepts and things we make and discover Human-Computer Interaction
I start to think about how I have this thing I’ve been doing on the side for a while (improv) and how to mix it with what I do seriously, and end up with Gestural Interaction.

So, from the big astronaut to gestural interaction. That’s the series of small steps for me, from age 12-ish until the present. Would age 12 Dustin be happy with where he is now? Probably if I tried to explain it to him, he would not get it, and might be kind of disappointed. Or he might be asking all sorts of annoying kid questions about what it is like to grow up.

Whoah that was an intense tangent. So, despite my fears about the Values Singularity, it kind of all turned out okay for me, I guess. But then again, I didn’t use any crazy technology to modify my values. However, it is a fool’s game to call any set of things we do “technology” and another part “naturalness”. Nothing is really that new. It is just how fast things move.

Another aspect of Greg Egan’s book was the notion that certain values are sinkholes. Once you end up in a certain region of values, you stay there and swirl around. So, if you value sitting around all day, every time you get to adjust your values, you adjust them so you sit around all day more. What’s to complain about – that sounds awesome! Your goal is simple and you have achieved it. The same goes for hard work. If you idealize hard work, you are stuck with thinking that hard work is good.

Calvin wants a space shuttle. Hobbes wanted a sandwich. Hobbes got his wish.
What if you decided to wish for a dream that was attainable.

It all sounds like a runaway optimization to me. And that is what I am terrified about. This is the thing that makes me curl up in a dark corner of my room and shiver myself to sleep in existential horror. Just someone please tell me that this is the right thing to care about, because I am not sure if it is.