Hacking for science and creating synesthesia

16 11 2011

By Marissa Fessenden

This past weekend I willingly deprived myself of sleep in order to participate in Science Hack Day in San Francisco.

What is a Hack Day?

Well, a hack is a quick solution to a problem. Not necessarily pretty, but probably clever.

A Hack Day is usually a 48 hour event where people with ideas get together and make something cool! Wikipedia lists a bunch of hack days around the world. It seems to be a fairly new phenomenon. Science Hack Day was first held in London last year. Since then people have gotten excited and made things with science! (exclamation points are very much part of the culture) in Mexico City, Cincinnati and last weekend in San Francisco for the second time. Upcoming events include Nairobi and Cape Town.

The idea behind science hack day is that there is so much data out there, and so many things to be found, that people can do science or use science in innovative ways through hacking.

Last year’s winningest hack was the particle wind chime. Matt Bellis, a physicist and post-doc at Stanford, led the team. He writes:

The idea was to allow users to take properties of the particles that we observe in our detector (energy, distance from the interaction region, type of detector it is interacting with, etc.) and map that onto sonic characteristics (volume, timbre (instrument), pitch, duration, etc.). In this way, the user can explore the data themselves and find mappings which either make sense to him or her, or are simply more aesthetically pleasing.

I admit that I was not expecting to be of much use at a hack day. I have no experience in software development or coding and very little ability to use tools and build stuff. But the website said “If you’re a coder, designer, scientist, hacker or just an enthusiastic person with good ideas, Science Hack Day is for you.” I wasn’t sure that I had good ideas, but I could do enthusiasm.

Emails from Ariel Waldmen, the founder of Science Hack Day SF, repeatedly emphasized that I didn’t even need to have an idea before coming. And that eavesdropping on other people was completely acceptable. So I went.

I wandered around the morning of the first day, introduced myself to people and tried to think about how I could be useful. Everyone was very friendly, very enthusiastic and full of ideas.

After inspirational talks I found a couple of young women who won me over with their NCBI ROFL shirts. We were going to create a walk-in, interactive model of a cell, but didn’t think of a satisfactory way to accomplish that over lunch. Instead we turned to the group behind us and decided to help with their idea: create a synesthesia machine to override our everyday senses.

Synesthesia is a mixing of the senses. Synesthetes might report seeing green when they read the number 4, feeling a strong impression of a particular shape when they eat ice cream  or tasting raspberries every time they listen to the third movement of Beethoven’s Moonlight Sonata. Scientists used to believe that this was just a strong memory or that people were making up the sensations. But recent studies show a neurological basis. This study found additional activity in the brain of synesthetes corresponding to their sensations.

The project was based on an idea by this guy:

Liam in the foreground, Fen inspecting code in the background. Photo by Marissa

I asked our fearless leader to tell me about his idea to create a synesthesia machine. He responded via email:

There’s a strong evolutionary pressure to optimize our perception of the world to emphasize information that makes a difference to our survival. This means that we necessarily under-appreciate a large amount of the information in the world, that is, we ignore much of reality. The goal was to short-circuit the pre-processing systems in our visual cortex by sending visual input into an inappropriate sense, the sense of touch. The hope is that the resultant synesthesia would allow us to appreciate a different slice of reality to the quotidian.

Hacking this idea did take us most of the time. I stayed up until 3:30 a.m. and I loved it.

Lil modeling the placement of speakers.

We used a webcam to capture an image and translate that to a 12 pixel square. Each pixel in the square would register the image as either white or black. That would translate to an “on” or “off” code transmitted to an Arduino –“an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software.” The Arduino was wired to small, cheap speakers sewn on the inside of a full head mask. So the visual input was translated to vibration (and sound) on your face!

Initially, we thought we could design a vest lined with the speakers. I was ready to try and modify a thrift-store purchase into a tight fitting garment. That probably would have been beyond my sewing ability. We decided that a mask would be a more visually striking design.

Lil found a pattern online of a bondage mask. Yes, I know … but we needed something that would keep the speakers in contact with your face. I didn’t find a good pattern for spiderman masks, so bondage mask pattern it was.

Other members of the team worked on coding. I have very little idea what goes into software programming, so I don’t know what they did, but they did it well.

We also needed to wire the Arduino to the speakers, something that required soldering and some electronic-building knowledge. I remember building a tiny robot using a kit when I was younger, so I did get to try my hand at soldering. Though I am not an expert, my skills in that area improved under the tutelage of another member of our group (Parker, who was also my old friend from college).

I think was was most remarkable was the way each of us in the group was instrumental in getting to the finished product.

The final product was…amazingly creepy. We call it Syneseizure. It may possibly be the next superhero or supervillain.

Lil, Parker in mask and Bala. Photo by Marissa

Check out our project’s blog for more photos, a link to the code we used, video of the first public test and video of our presentation at the end of the hack day. Our hack won the People’s Choice award!

The other hacks were incredible. From a globe with a laser inside that tracked the movement of the International Space Station, to a drinkable DNA extraction (with Bacardi 151 and strawberries!), there was some seriously cool things going on.

I had a great time. Thanks so much to the organizers who worked to keep us hacking and fed! They did so much. Here is a time lapse video of the 8 hours of set up at the space. And some more photos of the day.

More reading:

An article from Scientific American about synesthesia.

website about synesthesia for kids, from the University of Washington.

The American Synesthesia Association


Undead Science

28 10 2011

Beth Marie Mole

by Beth Marie Mole


Hordes of people—numbed by the Great Recession, perhaps—have once again staggered toward fresh zombie fare. But unlike other zombie fever outbreaks, splatters from the pop-culture feast are landing in the far corners of science—and they’re leaving quite the mark.

Zombie Science Invasion

From left to right, zombified Dustin Adams, Julia Kelly, and Stephanie Lukin invade UCSC's science hill.

Since George Romero’s 1968, genre-defining film, Night of the Living Dead, zombie zeal has lurched in and out of popularity.  Some link their recurring resurrections to the economy—zombie apocalypse as blue-collar strife?—others to their flexibility.

I side more with the latter.

A few years ago, a friend and I hosted a seven-week long zombie movie series, presenting a doubleheader each week. We started with Romero’s Living Dead series then ventured to the work of successors, including Sam Raimi and Dan O’Bannon, and humorous homage, such as Slither and Shaun of the Dead.

Our audience was mostly our science comrades from biology, ecology, spatial epidemiology, and sociology, and there was something for everyone.

The zombie skeleton supports a wide variety of meaty topics—as the plumpness of the genre demonstrates—allowing writers and directors to dissect complex social issues and our fear of unknowns.  But in the latest zombie outbreak, scientists are the ones biting. In fact, they’ve clamped on and welcomed shuffling zombies into narrow allies of science, including zombie parasitology, apocalypse preparation, disease ecology and zombie infection modeling, and zombified anesthesiology.

Then there’s Dr. Bradley Voytek, a cognitive neuroscientist at the University of California at San Francisco, and the leading zombie brain expert with the Zombie Research Society.

Using current understandings of the brain, advances in imaging, and a careful “examination” of zombie behavior, Voytek and his colleague, Dr. Timothy Verstynen at the University of Pittsburgh, diagnosed the undead with ‘Consciousness Deficit Hypoactivity Disorder’ and modeled how a zombie brain looks.

Crunching the numbers on zombie apocalypse modeling (Munz et al., 2009)

He defines CDHD by “ the loss of rational, voluntary, and conscious behavior replaced by delusional/impulsive aggression, stimulus-driven attention, and the inability to coordinate linguistic behavior.”

After Voytek presented their findings at this year’s Comic-Con and ZomBcon, I tracked him down to see if the undead are bringing life to scientific discussion (all the while denying my nerdy impulse to ask him what it’s like to sit next to Romero and his take on the heated question of fast zombies). It turns out zombies are spreading science.

At Comic-Con, Voytek sat on a panel with Max Brooks, author of The Zombie Survival Guide and World War Z, which is being made into a movie starring Brad Pitt.

“After our panel and during the Q&A session things got a little awkward; the majority of the questions weren’t directed at him, but rather were questions for me about the zombie brain. It was amazing,” Voytek says. “I had questions from students studying psychology and neuroscience, from a man who had suffered a stroke, a deaf gentleman, and so on. A group of us hung out and chatted about zombie movies and neuroscience for about 30 minutes after the panel.”

In his presentations, Voytek went through the seven classic symptoms of being undead and presented the neural explanation of each one.  For example, the most obvious symptom, reactive-impulsive aggression—that unrelenting desire to devour the living—is likely caused by abnormalities in the orbital frontal cortex, the lower part of the frontal lobe.  The orbital frontal cortex usually shuts down rage signals pumped out of by amygdala, a little almond shaped region in front of the orbital cortex.

In a case study of a man who had a steel rod go through his orbital frontal cortex, scientists found that without it you might switch from relaxing on the couch with some tea to savagely attacking someone in the garden.

“At its heart, the zombie genre is horrific. There are a lot of scary things in the world, and entertainment lets us encounter and confront our fears in a very safe way.”

Voytek and Verstynen developed explanations for the other six symptoms (loss of impulse control, motor deficits, language difficulties, attention problems, and memory loss), which Voytek chronicled this week in blogs. The final product is a comprehensive view of the zombie brain.

A normal brain compared to the brain of someone suffering with CDHD

While the zombie brain has the public salivating, Voytek has gotten cheers from other scientists for his zombie breakthroughs.  At zomBcon, members of the audience asked Voytek about the ‘herding’ behaviors of zombies in this season’s premier of The Walking Dead.  A professor of wildlife ecology approached him afterward to say he enjoyed seeing the public engaged in a scientific discussion.

“It’s funny to me that the best method I’ve found to engage the public in science is with something so fantastical and made up! I think it’s similar to how pulp sci-fi novels from the 1950s and 60s got so many people interested in the space sciences in the 60s and 70s,” he says.

Whether it’s the horror, exploring the unknown, or the economy, the zombification of science is drawing people in.

“The wonder and imagination is what draws people to science,” Voytek says. “The joy of discovery is what keeps them there.

Join us…