A Dream-Locked Vision

A had a strange experience last night.

Background: I had had some back pain and insomnia for a couple days. It seemed better yesterday morning, but got worse over the day. I met up with friends midday near Prospect Park to grab drinks on a patio. The back pain was getting worse, and I gleefully informed them of my plans to get an edible and not leave my bed for the rest of Saturday. I ate the edible around 4 pm. I cozied into bed, surrounded by takeout as I enjoyed the sensation, and over the rest of the day watched an impulsive mix of The Boys, Primal, Mystery Science Theatre 3000, and Love, Death and Robots. I drifted in and out of sleep a few times. It got darker in my bedroom as the sun went down.

Some time in the night I woke up, looking at the Christmas lights I had left going over my bed. As it had all day, it was hard to move but eventually I rolled my head to the side and…my view did not change. I shifted my neck left and right and moved my eyes around but my view was, consistently, the line of sight from when I first awoke. Here is a re-enactment:

(There is some sun coming in from the other room when I recorded this re-enactment, so when I experienced this vision, it was a bit darker)

This was starting to get unsettling. Thoughts raced through my head: had I somehow…become blind, and this was the memory of my last moments of vision, cycling forever? Was I experiencing a form of sleep paralysis, awake but unable to move? This predicament was quite unlike any previous experience on edibles.

I moved my arms slightly and could feel the texture of the sheets. I managed to work my body up to a sitting position, and could reach up and flip the light switch above my bed. This switch controlled the wall fixture that my lights are hanging on, in the above video. The light did not turn on when I flipped the switch, but, I thought I could see my vision get slightly brighter. My point of view didn’t change, the Christmas lights kept flickering, but I thought by turning my light switch on and off I could see a small amount of illumination change at the corners of my vision, like reality was creeping in around the fixed vision I was having. This lent credence to the theory I had gone blind. Scarier and scarier.

Since my laser eye surgery in late 2020, my eyes have been drier than normal. My consistent ritual after waking has since been to use some hydrating eye drops, only after which it is more comfortable to keep them open. Maybe I needed to hydrate my eyes to cure my current brand of blindness?

I’m going to describe what I did with my body for the next several minutes. This is a bit hard to understand, but during that entire time my point of view, what I saw with my eyes, never changed; just the flickering Christmas lights against my wall, the telescope to the right and the two paintings on the left. I could feel with my skin and hear with my ears and navigate around my apartment as if I was totally in the dark, but what my eyes showed me never went away, except during blinking.

I rolled to my left and fumbled through my drawer with my right hand, rooting around for eyedrops with my left hand, but couldn’t find them. I rolled again to my right and ended up on the floor on all fours. I crawled forward to the stairs, and reached up with my left hand to grab the railing, carefully descending. I turned the corner – my hands extending to help navigate and feet sliding across the floor. I felt the fridge and nearly went for the kitchen sink, but was afraid of hitting my head against any dishes in the sink if I put it low. I turned into the bathroom, finding the door and then the sink. I managed to turn on the tap, and make a bowl of water with my fingers and splashed my face, rubbing my eyes repeatedly. My vision didn’t change at all. I reached backwards and turned the bathroom light on and off, and thought I saw the same slight change in illumination as I had before, but still my original point of view remained primary. I now remember the feeling of getting on my knees and resting my cheek on the counter. I splashed water onto my face as the tap kept running, and blinked my eyes multiple times but my vision would not reset to where the rest of my body’s senses thought I was.

That’s it, I was having some sort of medical emergency! I felt too unsure of myself to call 911 right away, and needed to talk to someone else first. Several years ago, I had a very high fever that I had to go to emergency for, and I talked to a friend first who convinced me that, yes Dustin, you should go to emergency. I was reminded of that. But of course I couldn’t see to operate the phone – still all my eyes saw was the same point of view (please refer to video above).

I stumbled over to where I kept my phone charging on top of my fridge, pulled out the cord, and held down the power button on the iPhone that triggers Siri. I heard it beep, and said “Call Dad”. It worked – which I inferred because I held it to my ear and heard it ringing. It rang out several times, but he didn’t pick up. Of course, it was probably late. From my point of view, I could see there was no sun anymore. There was a big clock in front of me where my body was in the kitchen, but my eyes were unhelpfully not looking at anything that gave me the exact time. I think the last time I looked at a clock it was 9pm-ish. By now, it was probably the middle of the night. I got Siri to call my brother on the west coast – he was probably still up! The phone started ringing, and he picked up, “Hey?” in his perfectly unique tone of supportive befuddlement. I started “I…uh…think I’m…”

And then I woke up. I could tell I was back in a body where the senses were in concert again. I was staring right at the point of view I’d been locked at during the entire experience, but when I rolled my head left and right, the view changed to my relief. Slowly, I got up to a sitting position. I retraced the steps I had taken: out of the bed, down the stairs, to the bathroom and to my phone. My clock on the kitchen wall said 1 am.

I think the easiest explanation is that I had a strange dream. Some would say I had an “out of body” experience in dream form, but for some reason one of the senses (vision) stayed behind.
There was no trace that I had actually done the things I imagined I had done – the sink was not wet, and there were no calls recorded on my phone. I sleepwalked a few times when I was younger, but the last known time happened when I was 9; it was possible I had a semi-conscious sleepwalking experience but since what I dreamt of led to no physical effect in the world, I doubt it.
My preferred theory is I entered a dream state just as I was falling asleep, and either started the dream while my eyes were still open, locking my vision to a single point of view, or somehow dreamt with my eyes open. Hypnogogia is a very special, and I have had enough strange experiences falling asleep in lectures to not consider this far outside the normal. Add in the lingering effects of the edible and everything was explained.

Rationally reassured, but still unsettled, I futzed around a bit to reassure myself I was in real reality, and went to sleep without incident.

An Alarming Epilogue

I woke this morning more refreshed than I’d felt in over a month, with almost all of my back pain gone. I started my day with stovetop espresso, got on gym clothes and headed out. All was normal and good, the last night simply a stranger than normal dream I already planned to blog about.

When I got back from the gym in the door at home I was struck with an intense smell – like someone had dumped a liter of paint thinner in my kitchen. The fumes were overwhelming, almost like I was about pass out. I frantically checked my gas stove, turned on every fan in my place and opened my windows. I texted my neighbours, and looked through my cupboards in case some cleaning solution had broken. I could find no satisfying explanation.

One of my neighbours did get back to me – the monk next door who incidentally helped install my AC units, since they face right onto his porch. His porch, which is just a tar-topped roof he puts plants on, had been resealed mid-week. My AC units, running with great effort, had been pumping these fresh sealant fumes right into my apartment for days. It was the time of day now where the sun faced directly onto his porch, increasing the pungentness of the fumes. Perhaps I just noticed the fumes for the first time because they were stronger, and leaving and coming back to my apartment hit me with the sensory shock instead of it slowly building up.

My apartment is now airing out, and I write up this unsettling experience from a patio in Brooklyn.

Posted in moments | Leave a comment

1px VR

I have always had slightly dry eyes. This has made wearing contacts difficult. During the pandemic, I gradually became inexcusably annoyed at having to choose foggy glasses or blurry vision when wearing a mask in public. Independently, it occurred to me in The Year of Isolation that this would be an ideal time for some long at-home recovery surgery. Sadly, there are no elective procedures I really needed or wanted, not even cosmetic. I even researched getting an elective tonsil or appendix removal, but the risk-reward payoff was not there. In 10-20 years there will probably be cool cyber body mods available, but I’m just not at the right time in history yet. Anyway, the fogginess and spare time led me to get laser eye surgery, PRK as opposed to the more popular LASIK, as this was slightly better for the shape of my corneas. A known side effect is that one’s eyes may get slightly drier, but it was worth it.

My PRK surgery was in December 2020. My vision is 20:20 now, and I love it. In addition to dry eyes, I’ve historically had “shy eyes”; my eyes overreact when anything is near them. This is yet another reason why contacts sucked. Any time I’d try out eye shadow or eyeliner, it was tediously time-consuming to get right. For the PRK surgery, they put me on Valium. This was interesting metaphysical experience because I just…did not care about my body any more. They put a laser tip right next to my eye ball and told me to look in a specific spot and not to look elsewhere. I don’t know what safety mechanisms there are in case I did not obey. I smelled my cornea burning during the operation, just like burning hair, yet still held fast. No mustering of willpower needed on my part; I just didn’t care about what was happening to my body.

These days, I put in 1-2 eye drops daily, and they’re the first thing I must reach for whenever I wake up. The dryness is particularly annoying in winter months. One surprising feature about eye dryness is that there’s a lag where they are too dry a few hours before you interpret the symptoms as “dryness”. Your vision worsens, and your eyes will feel tired long before you realize they are dry. I put up a vision chart in my kitchen as a sanity check that would help me know when to get eye drops. At a regular optometrist checkup I complained again about my eye dryness and they checked out my tear glands. They are atrophied (cause unknown, irrelevant) and operating at 50% capacity. This is called Meibomian Gland Dysfunction (MGD).

One treatment for this is Intense Pulsed Light Therapy, which is the actual preamble to the real subject of this post. My glands being atrophied means my oil content is low. There’s another pharmaceutical solution to dry eyes due to low water, but this wouldn’t help me much.

IPL is a new thing…initially for cosmetic skin care treatment. As far as I can tell, its function to improve eye drop gland function was discovered entirely by accident (link to paper by Craig, Chen and Turnbull) thanks to improvement in eye glands in patients getting cosmetic treatments.

Here’s a quote on the purported mechanism of action from this paper by Toyos, McGill and Briscoe:

Potentially, IPL near the lid should cause closing of the abnormal blood vessels secreting inflammatory mediators and decrease bacterial overgrowth; an eventuality we began to observe early in our practice when some of our patients treated with IPL showed improvement in their MGD and DED.

So, now I’m in a once/month regimen where I sit in an optometrist’s office and have intense pulsed light directed around my eyes. This is through custom hardware, that looks like this:

Intense Pulsed Light Therapy HMD

I wear this HMD for 15 full minutes. The word “pulsed” is in the treatment, but the pulsing isn’t really noticeable. I have difficulty describing the effect of experiencing a constant red, more red than you can handle or conceive, unending, as your only stimulus. My eyes are closed the entire time, and yet the brightness I perceive is more than when I look directly at the sun with my eyes closed. My entire existence and the universe is red. I go on journeys in the world of red and converse with the red. Red and I plan trips, picnics and projects together, and we make after-dinner arrangements. All I’ve ever known is red and we are for each other. The only thing I need is red. This is as involved as any hallucinogen.

Have we, perhaps, considered that the problem with VR is that we’ve been pursuing many pixels, rendered very quickly and accurately, and yet this is the wrong approach? What if we simply made 1 pixel, but made it very, very good?

Posted in commentary, me-news, moments | 3 Comments

Books Read 2020

This year was a year of very long books, often on walks, or drives. In a first, the best piece of literature I “read” was the narrative game Disco Elysium. In extreme contrast, I loved the long, ancient tomes Herodotus’ Histories, and Giovanni Boccaccio’s The Decameron; both great reminders of the universality of humans barely coping through interesting times.

Sword of Destiny – Andrzej Sapkowski
How To Write Science Fiction & Fantasy – Orson Scott Card
To Your Scattered Bodies Go, Riverworld Volume 1 – Phillip José Farmer
Loonshots- Safi Bahcall
We Are Legion (We Are Bob) – Dennis E. Taylor
For We Are Many – Dennis E. Taylor
All These Worlds – Dennis E. Taylor
Invisible Planets – Edited by Ken Liu
Disco Elysium – Studio ZAUM
Mind of my Mind – Octavia Butler
Safe Sex Issue 1-6 – Tina Horn
The Masque of the Red Death – Edgar Allan Poe
Gideon the Ninth – Tamsyn Muir
World War Z – Max Brooks
The Shambler from the Stars – Robert Bloch
Haunter in the Dark – HP Lovecraft
Wuthering Heights – Emily Brontë
Empire Star – Samuel R. Delany
Babel-17 – Samuel R. Delany
Histories – Herodotus
Attached – Amir Levine and Rachel S.F. Heller
The Rime of the Ancient Mariner – Samuel Taylor Coleridge
Exhalation – Ted Chiang
Second Variety – Philip K. Dick
Deadhouse Gates – Steven Erikson
Travesties – Tom Stoppard
Symbiosis Theory – Choyeop Kim
Memories of Ice – Steven Erikson
A Canticle for Leibowitz – Walter M. Miller Jr.
Constantinople: City on the Golden Horn – David Jacobs
Octavia Butler – Kindred
The Cartography of Sudden Death – Charlie Jane Anders
Ancestry & Culture: An Alternative to Race in 5e – Eugene Marshall
The Decameron – Giovanni Boccaccio
The Rise of Kyoshi – FC Yee
Money Shot Vol. 1 – Tim Seeley & Sarah Beattie
Hench – Natalie Zina Walschots
Peter Watts Is An Angry Sentient Tumor – Peter Watts
How To Have a Good Day – Caroline Webb
Internal Family Systems Therapy, 2nd Ed. – Richard C. Schwartz and Martha Sweezy
Open Earth – Sarah Mirk, Eva Cabrera, Claudia Aguirre
Divergent Mind – Jenara Nerenberg
Ten Days in a Mad-House – Nelly Bly
The Canterbury Tales – Geoffrey Chaucer
Fray – Joss Whedon, Karl Moline, Andy Owens
Carmilla – Sheridan Le Fanu
Parable of the Sower – Octavia Butler
Interior Chinatown – Charles Yu
The Open Society and Its Enemies – Karl Popper
Recursion – Blake Crouch
The Body Keeps The Score – Bessel Van Der Kolk
Dealers of Lightning – Michael Hiltzik

Posted in Uncategorized | Leave a comment

Species-Wide Imposter Syndrome

Have you ever had that moment where you’re walking from one meeting to another and something clumsy happens – like you trip over a chair corner, you realize your shirt is undone, or you sneeze inelegantly.

And then the entire facade of civilization comes crashing down. We’re barely different from the apes messing around hanging out in trees and throwing their own poop at each other as a form of culture. We just happened to decide to live closer together, organize on an industrial scale to enable that density, wear clothes that represent our personality, have ‘work hours’ and in the morning meet for ‘stand ups’ to present our ‘project updates’. The standups are important to push our distinct bipedalism features.

All these systems and rituals we’ve built to make up our civilization are just things we made up!? And we keep them around because they kind of work? The audacity! Why haven’t we bothered to stroll down to the local Intergalactic Library to pick up a book on being a civilization? We expect new parents to read a couple books about parenthood. Yet here we are just fucking around, eating fruit that’s gone bad and deciding we liked it so much we’ll build an entire industrial complex controlling it.

If you’ve ever experienced imposter syndrome as an individual, it eventually goes away, or it becomes less frequent as you get better at managing it. If you’re lucky to have received external validation from other beings of your personal value, you can look at these lil bits to keep your imposter syndrome at bay, as a shrine of treats.

Humanity isn’t going to get any meaningful external validation. Maybe never, maybe not for a while. In fact, any other civilization we run into might turn out to be manipulative dicks about it. On Earth, the history of first contact between societies has not been super-smooth. If either member of first contact felt validated after the experience, it wasn’t for good reasons.

So we’re just stuck with this permanent species-wide imposter syndrome. And the terrifying responsibility that nobody is going to step in and rescue us from our shitty choices, so we actually need to think them through? That sucks.

Anyway – sorry I have to go just now, I need to prepare a Gantt chart for something.

Posted in commentary, moments | Leave a comment

Books Read 2019

My favorite book this year is either Voyager in Night by C.J. Cherryh, or Teckla by Steven Brust. The former is about an interaction with a computer consciousness as it’s falling apart, the latter is about a socialist uprising in a fantasy world, in the context of negotiating differences in romantic relationships. Both are wonderful.

The Rewind Files – Claire Willet

The Business of Broadway: An Insider’s Guide to Working, Producing, and Investing in the World’s Greatest Theatre Community – Mitch Weiss & Perri Gaffney

Beneath the Keep: Fifth Edition Fantasy #14 – Chris Doyle

XCrawl: New Year’s Evil: Dungeon Crawl Classics – Brendan Lasalle

Rosewater – Tade Thompson

The Fermi Paradox is Our Business Model – Charlie Jane Anders

Invisible Cities – Italo Calvino

The Healthy Brain – Aileen Burford-Mason

Port Eternity – C.J. Cherryh

Voyager in Night – C.J. Cherryh

Use of Weapons – Iain M. Banks

Gate of Ivrel – C. J. Cherryh

Mongols, Huns and Vikings: Nomads at War – Hugh Kennedy

Why Do Birds – Damon Knight

Stanislavski: An Introduction – Jean Benedetti

Mindset: The New Psychology of Success – Carol S. Dweck

Lord of Thunder – Andre Norton

Cracking the Coding Interview; 6th Edition – Gayle Laakmann McDowell

The Silmarillion – J.R.R. Tolkien

Wave Without A Shore – C. J. Cherryh

The Midas Flesh Volume 1 – Ryan North

The Einstein Intersection – Samuel R. Delany

The Hug – Lesley Simpson

The 48 Laws of Power – Robert Greene

Fall; or, Dodge in Hell – Neal Stephenson

The Name of the Wind – Patrick Rothfuss

The State of the Art – Iain M. Banks

Jhereg – Steven Brust

Yendi – Steven Brust

Teckla – Steven Brust

The Westing Game – Ellen Raskin

Blindsight – Peter Watts (re-read)

Long Distance Relationships: The Complete Guide – Dr. Gregory T. Guldner

Eastern Standard Tribe – Cory Doctorow

Alien 3: The Unproduced Screenplay – William Gibson, Johnnie Christmas, Tamra Bonvillain

Trail of Cthulhu – Kenneth Hite

The Snow Queen – Joan D. Vinge

The Laundry Files Volume 1: The Atrocity Archives – Charlie Stross

Die Volume 1: Fantasy Heartbreaker – Gillen, Hans, Cowles

Odyssey – Keith Laumer

The Artist’s Way – Julia Cameron

Binti – Nnedi Okorafor

Posted in Uncategorized | Leave a comment

GPT-2 and Culture Ship Names

Brief Personal Update: I accepted a new job and moved to Manhattan – more on that later. Critically, I signed a lease on an apartment after 3 years of cybernomadism. I intend to live here for a while, but I want it to feel like a spaceship on the move. Thus, it needs a proper spaceship name patterned after the names in The Culture Series, a post-scarcity intragalactic space opera where each ship has its own advanced AI personality.

GPT-2 is the newest hot shit in text generation, courtesy of OpenAI. I seeded it with the following incredients:
* the entire list of Culture Ship names appearing in literature
* relevant poetry from a witchy collaborator
* my worldbuilding docs

The final name of the ship I chose I’ll hold back for now. Here are the other names GPT-2 and I generated that I didn’t end up using. Please enjoy.

Fittingly, This One Is Still Working
Outlandish Situation (But Not Too Outlandish)
Another Nonsense Factory Item
Painted By A Genius
To Be Continued
This One Looks A Little More Like An Artifact
From The Other Side Of The Universe
About A Very Mysterious Entity
A New Lover From Another Side
Of the Universe This One Will Help Me
The Wizard’s Folly (A Very Serious Thing)
This One Looks Really Fancy
Maybe It Was Something From Another World
In The Woods Where I’m From I Found An Empty Coffin
(Note To Self: Don’t Tell The Kids I Did This.)
End Is Nigh
The Lying Kind Of Love
For You I’d Not Want To Let Go
For In An Attempt So Grand That It Exceeds Comprehension
Perfect Match for My Mood
Determination Of The Dormant Soul
(When You Are Not Looking That Is.)
Aspects About It I Found Out Later
Of Course I Also Liked To Eat
One of the Most Fun Kinds of Travel
You Could Say I Enjoy My Own Company
Outlandishness Aside, What Could I Not Like?
I Love It, It Just Does What It Says
Painted By Its Colors and Form
A Very Serious Case Of “Positivism”
Folly is Fortunate And Irreversible
Aspects Of The Folly Problem Are: The Idea That There Exists an Ultimate Truth
Uncontrollable, but Very Focused On The Present Moment

Some of my favourite space denizens:
(When You Are Not Looking That Is.) sounds like a spy ship,
Determination Of The Dormant Soul is a storage ship in deep sleep,
Painted By Its Colors and Form is an art party ship,
Aspects About It I Found Out Later is an investigator.

Posted in creations, me-news, technical, wordplay | 2 Comments

Upcoming Topics in Computing (that I’m interested in)

I have spent a lot of time in academia, and the last four years at commercial startups, including running a few myself. This gives me a special vantage point on upcoming computing areas – I want to jump over what’s interesting now to what will be interesting in 5 years. My current job hunt has been a really great excuse to do an industry overview and talk to everyone; I honestly wish I could do a two-month sabbatical each year and go spend a day talking to every single interesting company or research organization.

In the next decade, the most interesting computing trends to me are:
– Synchronicity and Liveness
– Cyborgized Cognition
– Telepresence Embodiment

I want to work on these problems whether in research, or commercially. If you work in this area and want to just chat, or hire me, give me an email, or tweet.

I’m going to talk about the next decade shortly, but first here are a few trends I participated in that changed significantly over the last decade:
Tabletop Computing used to be a big deal, with the Microsoft Surface, DiamondTouch and interactive wall displays. My Master’s Degree was all about tabletop computing. However we don’t really have anything exciting happening in this area now – nobody does day-to-day computing on or around horizontal surfaces. The only interesting game in town is maybe TiltFive.
Spatial Computing as a term has managed to get out of tech and academic circles and into general discussion. This is great – it’s a useful summary of all the interesting work happening in Computer Vision, AR, VR and Volumetric Capture. Previous attempts to summarize the field fell into useless 360 vs Stereo or AR vs VR vs XR vs “Cinematic Reality” hair-splitting.
Gestural Interaction was how I thought (and many people still think) we’d interact with computers in the future. The idea of implicit input, or that maybe people will learn American Sign Language to talk to computers, is such an enticing idea that it’s drawn too many people in. But gestural interaction has failed to get real traction. There’s just too much noise in real-life gestural environments. Most gestures actively used in hand-tracking UIs are indexical instead of symbolic, and keyboards and voice interaction still being most important.
Deep Learning unfortunately means too many things right now. Anything that involves some statistical analysis is now called Deep Learning. Marketers that used to use SPSS to analyze Likert scores instead use differently-branded software packages to do deep learning, with an unclear change in actionability. Doing an analysis of an image based on edge detection could be construed as using a small CNN, so it is now called Deep Learning, too. I’m not trying to gatekeep, but I do wish that in general people were more specific, and didn’t use an obfuscated deep learning technique when a much clearer tool like linear regression works totally fine and is easy to understand.

Let me tell you what I’m excited about for the next decade of computing

Synchronicity and Liveness
We are so used to the convenience of asynchronous communication that it’s now common to text or schedule in advance of calling, and that calling someone without warning is considered either rude and requires an apology, or a potential emergency. In timezone-distributed tech circles, almost every meeting is a Google Calendar invite. Messages are sent slightly before or after the scheduled time, with either an apology that someone is only slightly late, or seeking confirmation that the meeting is still on. There’s just too much overhead.

To have quality synchronous time, people still fly in to meetings. This has been a bit bizarre during my current job hunt as companies have spent probably mid-five-figures now flying and hotelling me various places, only to have many of my “onsite” meetings happen in a single-person conference room with a remote employee over Zoom.

Slack usage is pervasive at tech companies of all sizes, since instant messaging is better than emails for quick, time sensitive communication. However, if an employee gets overwhelmed with notifications, they tend to turn them off. Incentives for communicating honestly about your availability are misaligned, and in-person cues about availability aren’t properly translated.

In real-person meetings at someone’s desk, you can get a sense of how busy they are, or if they’re talking to someone else that you can interrupt if you need to. Currently our interaction states are binary: I’m in a call, or I’m not. I’m “online” in Slack, or I’m “offline”, and I may have set the “offline” status artificially because I need to concentrate on something and can’t be bugged at the moment.

I hope new research or businesses explore making synchronicity at a distance more friendly. Calendly has been incredibly useful for me to schedule time to speak with recruiters, as it reduces all the back-and-forth scheduling iterations to a single click. I wish when I called someone, and they’re busy, that we could set up a system to automatically initiate a call again when their current call is done, or we’re both free again. Dialup – sort of a synchronous, audio-based social network, is doing very interesting things with calls that aren’t invoked by either callee. There’s interesting research or a product to be done in communicating and requesting availability – this is handled well in-person with body language, open/closed doors, quick, ephemeral interruptions without disrupting ongoing meetings. Computing doesn’t handle this well at all. I almost want an always-on video conference setup that people can request to join, and then a queue if the join isn’t immediate.

When I was running ticketing for The Aluminum Cat, one of the surprises was there was no online ticketing service that handled time zones well. Turns out, most tickets purchased are for in-person events, so this use case isn’t handled well at all. I’d love a company to build an equivalent of Doodle + eBay + calendly for bidding on group activities, where a booking transaction occurs automatically once quorum (financial, headcount) is met.

Cyborgized Cognition
The average person still refers to artificial intelligences, algorithms and machine learning as “the other”. These often feel a lot like “the other” because when we interact with them, they’re run by large, impersonal, companies whose interests aren’t the same as ours. An AI may be branded as a decision engine, but if that’s done by a company that also serves ads, it doesn’t generate great consumer trust that my behaviour isn’t being influenced outside my best interests.

Once we fix this trust problem (which might require fixing capitalism, tbh), we can start by forming better more useful, actionable relationships with AI – a cyborgization.

The AIs I actively engage with most on a regular basis are my Gmail spam/important filters, and my Facebook and Twitter feeds. Training my Gmail spam/important filters feels like gardening, in that I know it takes extra work to mark messages as important or not instead of deleting or archiving them, but I think it will make my inbox more manageable in the future. When I tell Facebook, Twitter or Instagram that an ad or a post isn’t relevant for me, I don’t have the same relationship; because I know these social networks prioritize engagement (ad views) over what I may want. “We’ll show you less of these in the future” is the best possible phrase to describe the change, but it still feels passive aggressive, since we know the feed algorithms aren’t ultimately trying to serve you.

I have enough presence of mind to spend time gardening algorithms, but from when I talk to other less tech-savvy people, they don’t understand (or believe) that algorithms can change. There is definitely interesting ethnographic research to be done on how the average user can be educated about interacting with algorithms – currently they’re treated like omniscient oracles, or malicious agents able to read your mind. I’d also love to have a browser plugin that lets me know when a news feed is changing on me, sort of like sousveillance for A/B testing.

For cyborgized cognition, I think eventually our day-to-day lives will more resemble a continually negotiated tradeoff between tasks your AI handles, and you handle. Cyborg chess is a good model for this. There are startups that use the term “personal assistant” or “concierge” which I feel a bit grossed out by, since I grew up middle-class and those are things that only useless rich people use. I think there will eventually be lots of interesting research and design work to be done in the area of users and an AI negotiating implicit permission and agency, but none of that can be effectively be used in the wild yet until the trust relationship is better handled.

Maybe Apple will figure it out first. Historically, engaging with an algorithm you don’t own with large compute resources has meant that you give up a significant chunk of your privacy, however there’s a recent Apple Machine Learning Journal publication that has demonstrated remote model training that’s privacy-conscious.

Telepresence Embodiment
From spatial computing, and remote presence via drones such as Double, or wall displays such as Tonari, or live volumetric capture such as Mimesys (acquired by Magic Leap), people are going to spend more time interacting with other people and things at a distance with varying degrees of embodiment.

There’s interesting problems to be solved around scheduling, which I already covered in my section on Synchronicity, but embodiment and expression is an interesting area as well.

Embodiment could be for the purposes of self-expression, such as has been common in online games for a long-time. Wearing a different avatar is like simply wearing a mask. To appear older or of a different gender in voice chat, people have often used audio filters. In social spatial computing environments with live motion capture such as VRChat or Rec Room, it could be foreseeable that users may want motion filters – to appear more masculine, or less clumsy, or mobile at all when they may be quadraplegic.

For human-negotiated attention in remote presence, eye contact is important. Pluto is currently exploring using depth sensor data to correct remote eye contact during video calls so that face-to-face eye contact is preserved.

Update: Attention Correction is included in the upcoming iOS 13:

This works pretty immediately in a one-on-one video conference setup, if you want to preserve literal eye contact. But what if you’re in a multi-way video chat, like you’ve just formed Voltron:

From Perry Bible Fellowship

In this case, it becomes clear that just faithfully reproducing eye contact is overly literal. If I’m person A, I may be able to see when person B is looking at me, but when person B looks at person C, I may not be able to tell. It may be better to adjust person B’s video, as it appears to person A, so that person B’s eyes are oriented towards person C.

The interesting conclusion here is that with remote presence, there’s many reasons to transmit more higher-fidelity expressions of embodiment, but not to reproduce them literally at the remote end.

Posted in research | Leave a comment

The Aluminum Cat Documentary Released Now!

When you make an interactive show, it’s hard for the audience to tell just *how* interactive it is. This is part of the rare magic of any participatory theatre; even Keith Johnstone said don’t bother trying to convince your audience that a show is improvised, because they’ll never believe you. So we at Escape Character decided to show you.

For the Aluminum Cat’s run, we had 35 shows, with a total of 131 audience members. We trained up 3 actors to run the script (Original Cast Stephanie Malek, who took the first pass on all the characters, then later me and Ted Charette). Our script (by Natalie Zina Walschots) had a few possible endings, but these are more like directions that you can depart from a forest, not set paths like choosing between a few roads. We watched all the shows, made *spreadsheets* of player choices, and assembled it all in this 23 minute documentary. Enjoy!

If you played the show and were surprised what other people did, let us know! Subscribe to Escape Character’s mailing list.

Posted in art, creations, games, improv, theatre | Leave a comment

Taekwondo does not support Double-Jumping

I started Taekwondo in November 2018. I’ve wanted to try a martial art for years, but it’s been one of those hobbies in the backlog that required a chance encounter to nucleate.

I used to do more cardio and cognitive intensive physical activities: gymnastics, figure skating and parkour (amateurishly), but the last few years have only been biking, weightlifting and rock climbing.

Being a programmer, my body goes through long daily phases of neglect where it’s kept catatonic while my mind is in Infinite Fun Space. This takes a toll over time, and weightlifting has been great to work my body through a range of motion. Whenever I let my weightlifting regime slack, I can tell that it has been keeping my posture in check.

Unfortunately, I get bored easily with exercise. With both rock climbing and weightlifting I…somehow drift away since my mind feels unengaged, and I stop pushing myself as hard as I could. I listen to podcasts, but they put me in a contemplative state, not an engaged state.

I love biking at high speeds through urban environments. If I could somehow program while also doing this, I think this would be peak activation of all my pleasure centres at once (Sidenote: I should prototype this, and maybe I can make a more visceral version of Mecha Trigger).
Sadly, biking is not really possible through the winter, so I’ve been antsy for proper exercise for months.

But back to our main topic, which is Taekwondo. My first major observation is that punching and kicking consistently is surprisingly difficult. It reminds me of the time I was learning archery. It was only two or three months into Taekwondo that I got to properly spar with someone, where we’re both actively trying to kick and punch each other, in a “fire at will”.

To my surprise, jumping is way less effective than video games and dreams have been teaching me my entire life.

You see, I’m energetic, and flighty, so jumping excitedly out of the way when I’m in danger is a natural response. A lifetime of playing videogames where this is rewarded and encouraged has not helped, but rather has reinforced this instinct. In most of my dreams, I fly. My dream flight takes two different forms: a) the muddy, drifting hovering a metre above the ground b) soaring on air currents high in the sky. Over my life, I’ve become highly familiar with how to fly in these two configurations. I’ve even had Inception-style recursive dreams, where I’m flying, then wake up and discover I can still fly, and celebrate that all this practice has paid off and I can fly in real life. And then I wake up one more time, back in the real world, and find myself staring at the ceiling, asking why I would do this to myself.

To my surprise, when you jump out of the way of a punch or kick in real life, you, midair, are bound by Newton’s First Law, and have negligible effect over your momentum. There’s no double-jumping, mid-air steering, or glide mechanics at all.

I’ve discovered this thanks to every time I jump in the air, I get kicked or punched in the fucking ribs, and land, on my side, on the ground.

I have had heated conversations with my instructor where he’s informed me that there is no way to “train enough to charge up my chi so I can fly Dragonball Z style”. Harrumph.

So my current dodge strategy has had to change to sliding abruptly across the ground. This is way less cool, but has meant I don’t get knocked out of the air as much, which is nice. When you’re in the middle of a floaty jump, you’ve effectively removed yourself from combat, so now that I’m spending more time in combat, I can opportunistically make crazier moves. My current favourite is disruptive axe kicks that force my opponent to take a step back and blink a couple times.

My feet have had trouble dealing with all the sliding, and so I’ve had to get special foot moisturizer for them. My strength and reflexes are apparently fine, so currently the three major things holding me back in Taekwondo are:
– flexibility
– feet aren’t moist enough
my instinct to point my fingers dramatically

Posted in commentary, me-news | Leave a comment

Player Character Bios in Participatory Media

Originally published in Escape Character’s Newsletter.

Question: What’s the best way to hack someone who’s never LARP’d[1] before to get into character?

Our goal here is to have the player buy into the stakes of the show before they cross the threshold [2] into the space of the show. In Escape Character’s projects where the players talk directly to the actor, our initial moments have been charming, but the player engagement with their characters hasn’t persisted.

Stuff we’ve tried:
– Give them a profession: “You’re an inspector, your job is to investigate a dead body”
– Give them a unique characteristic: “You’re a nautical prodigy”

A new approach we’re trying is “You’re going to be a spy, and need to come up with a cover story”. The player does this in collaboration with the performer, and the performer pulls an appropriate prop out of the invisible Prop Box. (Suggested by performer Anders Yates)

Answer: Using collaborative “Cover Stories” seems to be a little better, but we’re still iterating.

The video above is from Sparasso [3], our in-development telepresence immersive theatre toolset for XR environments.

[1] LARP = Live Action Roleplaying
[2] From the Hero’s Journey
[3] Dionysus’ rebirth via disembodiment

Subscribe to our newsletter for more Escape Character updates.

Posted in commentary, creations, games | Leave a comment