[epic_dropcap style=”dark_ball”]T[/epic_dropcap]he Brain is the biggest untapped resource in video gaming. Sure, everything you play, see and hear is being processed by it, but your grey matter doesn’t play an active role in controlling the on-screen action. It receives input through your eyes and ears (and hands if you are holding a feedback controller) and tells your body what to do. But what if we cut out the middle man, so to speak, and let the brain control the game directly? No more use for your hands holding complicated controllers or getting cramps from mouse and keyboard. Just you sitting comfy in a chair thinking your way through hordes of baddies.
[epic_quote style=”quotation_1″]
300 milliseconds. That’s how long it takes for the computer to react to your thoughts.
[/epic_quote]
Is that even possible nowadays?
According to extensive studies in the Netherlands brain-controlled navigation is feasible. Yeah! Wait… what happened to killing bad guys left and right by the mere thought of it? Apparently THAT is a loooong way off.
Jan van Erp proudly shows the ‘TNO tactile Brain-Computer Interface’. At first glance it looks like a cap divers may wear when going underwater. The string of wires exiting a hole in the back suggest otherwise. This is serious laboratory gear. Van Erp has spent the last couple of years developing ways to let the brain control on screen action at the Netherlands Organization for Applied Scientific Research (TNO). He and his team didn’t invent the method of on-screen navigation using the brain, but they have made it much more responsive, shaving off milliseconds of response time, bringing the tech a little bit closer to gaming applications.
So how does it work?
Well, we cannot ‘think’ an object on-screen left or right. Our brain has to be tricked into action. The user wears a so-called vibration belt that sends little jolts to different locations around the waist. Each location corresponds naturally with a navigation direction. When the player wants to go left, his brain lights up when the corresponding location is vibrating. The computer sees this and thus realizes which way the player wants to go.
Human body
It works. But it’s nowhere near fast enough for real-time interaction. Any zombie, even the really slow ones, would have had more than enough time to crawl towards you and start eating your eyeballs before you had a chance to move your character out of the way. 300 milliseconds. That’s how long it takes for the computer to react to your thoughts. That’s too long. Now there’s two reasons for that. Computer hardware and the human body. The first one can be overcome when technology advances. Van Erp and his team developed a way to help biology and make the brain react faster. With even more trickery. They added additional stimuli like light flashes on-screen and clicking sounds left and right, in addition to the vibration belt. This way they took the reaction time down to 70 milliseconds.
The time-locked brain responses used for navigation have an interesting side effect. They reflect mental states. For example, the size of the brainwave that indicates direction will be lower if the user has to divide his attention between other tasks. This is a relatively easy way of checking one’s mental load, or how intuitive the design of the user interface is. This knowledge is valuable to optimize brain controlled devices and any type of human-computer interaction.
So we are getting there. Slowly but surely. Now the next step is to make that diver cap look sexy. •
[epic_box style=”” color=”dark” customcolor=”” align=”” size=”box_full” margin=”none” customsize=”” shadow=”on” corners=”on”]
Almost Jedis: Dreams of Danu
The brand names of brain computer interfaces (BCIs) may sound a little intimidating: the Epoc from Emotive Industries, the NeuroSky MindWave and the Neural Impulse Actuator by BCInet. The devices themselves look perhaps even more impressive, especially the Epoc with its many tentacle-like sensors. For the guys of start-up game studio Dreams of Danu however, these gadgets are bread and butter. Designer and Artist Floris Versendaal is quick to lower our high expectations: “Players are used to getting immediate feedback based on their input, but current-gen BCIs don’t allow you to perform Jedi-like instant thought commands”. So much for science fiction. As it turns out, BCIs don’t respond like a controller would to commands like “shoot”, “run” or “go right”. They merely measure brainwaves. But being able to create games that respond to a player’s moods and emotions, attention and concentration, that is what they do make possible. Versendaal explains: “I created a Breakout-like game that’s controlled with mouse and keyboard, but gameplay and difficulty level change depending on feedback sent by the BCI. It became the starting point for our first commercial title MindOut.” The team further designed Ker of Eden, a game offering a meditative experience. “I wanted to research whether it’s possible to steer thoughts of users by utilizing sound effects.” (The answer was yes, by the way) These small steps are far removed from what people usually think of when imagining brain controllers; i.e. instant thought control. But turning impossible expectations to realistic ones has become a common ritual that Dreams of Danu performs for users and clients.
[/epic_box]
