Or, How I Fixed A Real-Time Image Transmission Protocol For A Live Event By Making A Numbers Station

The evening before the GDC round table panel on Location-Based Stories, I was with most of them in a pub. I ended up here because I’m one-half of Playlines with Rob Morgan, who was on the panel.

As we got deeper in beers, we started telling war stories of the craziest things that went well or poorly with past projects in the wild. Here is one thing I did that went exceptionally well, but shouldn’t have.

~~~

It was autumn of 2011. Myself and a team of professors and students at the University of Toronto and OCAD were doing a project for the all-night art event Nuit Blanche. Tweetris was a multiplayer game where two players raced to match a shape from the game Tetris, as judged by a Kinect:

The first person to match the shape, and hold it stably for a period of time, had their picture taken and tweeted to our live feed, @TweetrisTO, which you can still see! Then, anyone during the event could go to a now-defunct URL and actually play Tetris with the bodies of players. You can see what this looks like in the middle of this video.

Getting down to the wire, we had the web app pulling the twitter images correctly. However, over the course of any game of Tetris, blocks will have parts removed from them as parts of them are destroyed. The dev building the web app was also the producing the event, and we couldn’t find a javascript image crop function that did exactly what we wanted (it was also 2011, so any advice you give now may be out of date). The images captured of the game winners were meant to be split into 3×4 blocks, and we were hoping that could be done on the web app side. However, the web app was not doing what we wanted. The image transfer “backend” was already twitter, and we had no time to build something else.

So, instead of an image crop web-app-side, we built a numbers station with a secret twitter account. A Numbers Station is a way to describe a series of odd shortwave radio stations that have transmitted numbers, spoken aloud, for decades, presumably used in transmitting intelligence information to spies. You should really read the Wikipedia article.

The Tweetris team had already built the capability to download images from Twitter into a webapp, so we changed our backend to have a secondary, secret twitter account, to which the Kinect app tweeted pre-cropped square images at, including a formatted CSV of which tetris piece it corresponded to, including sub-piece coordinate and rotation. Here is what that looked like:

Unfortunately, all of the twitpic links for this account appear to have been cleared out. But, if you can imagine, these were all random portions of grainy images of human bodies; half of a face, a hand and a sleeve, a shoe, etc. All somewhat off-centre and with small amounts of motion blur. Miraculously, even though this account received 4x the image tweets of the main TweetrisTO account, it did not get tagged as spam during the live 12-hour event. We obviously didn’t announce or promote anyone following this secret account.

So, there you go. Your backend not working? No worries – just make a number station in plain sight!

Tweetris was really successful, leading to a talk I did at Gamercamp, and several subsequent projects and papers, including a Kinect multiplayer game I worked on for a while.