November 2004

Thinking back on last week’s ISWC & ISMAR, I think there are three especially ripe areas of wearables research in the next few years:

• Fusion of Wearables and Ubicomp: This is an area I’ve thought was ripe for a while, but apart from location-beacons and markers for AR (Augmented Reality) there’s surprisingly little research that combines Ubiquitous Computing and Wearables. There are exceptions, like Georgia Tech’s work with the Aware Home and some work in adaptive “universal remote controls” for the disabled, but it feels like there should be some good work to be done combining the localization of Ubicomp with the personalization of Wearables. It also nicely fits with Buxton’s argument that the key design work to be done is in the seamless and transparent transitions between different context-specific interfaces.

• Social Network Computation, Visualization & Augmentation: This research has been going on for awhile, especially at the University of Oregon and more recently at the MIT Media Lab, but it seems to be getting traction lately. This sort of research looks at what can be done with multiple networked wearables users in a community. Typical applications include automatic match-making (along the lines of the Love Getty that was the craze in Japan several years ago), keeping a log of chance business meetings at conferences and trade shows, understanding social dynamics of a group like whether one person dominates the conversations, and real-time visualization of those social dynamics.

• AugCog / Wearable Brain-Scanning: As I mentioned in a previous post, this is potentially a big breakthrough. I don’t mean in the sense that it solves problem the wearable field has been struggling with, but rather that this could open a whole new branch of research. Neuroscience has taken off in the past 10 years with advances in brain-imaging technology like functional MRI. The downside is that you can only see what the brain is doing when performing tasks inside a lab setting — it’s studying the brain in captivity. Wearable sensors give us the ability to study the brain in the wild, and to correlate that brain activity with other wearable sensors. That plus the lower price should enable all sorts of new research into understanding how we use our brains in our everyday lives. That, in turn, will hopefully lead to new ways to augment our thinking processes, whether by modifying our interfaces to match our cognitive load, providing bio-feedback to help treat conditions like ADHD or perhaps addiction, or even physically stimulating the brain to treat conditions like Parkinson’s.

  That’s not to say there aren’t broad and potentially frightening aspects to this technology, but the issue that concerns me most applies generally to our recent understanding of the brain: I don’t think our society is prepared yet to deal with the coming neuroscience revolution. Our justice system, religion and even our system of government is based on the worn-out Cartesian idea that our minds are somehow distinct from the wetware of our brains and bodies. It’s been clear for decades that that assumption is false, but so far we’ve tried to ignore that fact in spite of warnings from science fiction and emerging policy debates about mental illness, psychoactive medication, addiction as illness and the occasional the-twinkies-made-me-do-it defense. The applications envisioned by AugCog are going to force the issue further, and societies doesn’t make a shift like that without serious growing pains.

One of the most exciting talks for me was the joint ISWC/ISMAR keynote by Dr. Dylan Schmorrow, one of the program managers for DARPA. The program managers are the guys who decide what research projects DARPA should fund — the best-known PM was probably JCR Licklider, who funded the Intelligence Augmentation research that led to the invention of the Internet, the mouse, the first(?) hypertext system, etc. The current program Dylan talked about was Augmented Cognition, which I’m now convinced could become the biggest breakthrough in wearable computing yet.

Intelligence Augmentation tried to support human mental tasks, especially engineering tasks, by interacting with a computer through models of the data you’re working with — that was really the start of the shift from the mainframe batch-processing model to the interactive computer model. AugCog is about supporting cognitive-level tasks like attention, memory, learning, comprehension, visualization abilities and basic decision making by directly measuring a person’s mental state. The latest technology to come out of this effort is a sensor about the size of your hand with several near-infrared LEDs on it in the shape of a daisy, with a light sensor in the center. The human skull is transparent to near-IR (that’s how you get rid of all the heat your brain produces), so when it’s placed on the scalp you can detect back-scatter from the surface of the brain. By doing signal processing on the returned light you can detect blood-flow and thus brain activity, up to about 5cm deep (basically the cortex). They’ve already got some promising data on detecting understanding — one of the things DARPA is especially interested in is being able to tell a soldier “Do this, then that, then the other thing… got that?” And even if he says “Yup” his helmet can say “no, he didn’t really get it….” Outside of military apps (and getting a little pie-in-the-sky), sometime down the road I can imagine using this kind of data to build interfaces that adapt to your cognitive load in near real-time, adjusting information displayed and output modalities to suit. In the more near-term, these devices are starting to be sold commercially and cost on the order of thousands of dollars, not tens or hundreds of thousands. That means a lot more brain-imaging science can be performed by a lot more diverse groups.

For more info check out www.augmentedcognition.org, or go to the Augmented Cognition conference being held as a part of HCI-International in Las Vegas July 22-27, 2005.

[I’ve been trip-blogging this past week but haven’t had convenient net access, so I’m afraid the real-time aspects of blogging are lacking… now that I’m hooked into the wireless at DEAF04 here’s some of my backlog.]

Bill Buxton’s ISWC keynote made a lot of points, but the one that struck me most was derived from three basic laws:

1. Moore’s Law: the number of transistors that can fit in a given area will double approximately every 18 months.
2. God’s Law (aka the complexity barrier): the number of brain cells we have to work with remains constant.
3. Buxton’s Law: technology designers will continue to promise functionality proportional to Moore’s Law.

The problem then is how to deliver more functionality without making the interface so unwieldy as to be completely unusable. Buxton went on to talk about the trade-off between generality and ease-of-use: the more specifically-designed an interface the easier it is to use but the more limited its scope.

The key, he argues, is to make lots of specific applications with interfaces well-suited for their particular niche. Then you don’t need a single general interface, but instead can concentrate on the seamlessness and transparency of transitions between interfaces.

It’s a nice way of thinking about things, especially when thinking about the combination of wearables and ubicomp (see next post).

Like in previous years, the big theme here at ISMAR (the International Symposium of Augmented and Mediated Reality) seems to be registration and tracking — how to detect where objects and people are in the physical world so you can overlay graphics as accurately as possible. AR isn’t my main field, but I’ve had a couple of conversations so far about how we’re really reaching a point of diminishing returns. It’s great that we’re seeing minor incremental improvements in this area, but what we’re really lacking are new, innovative uses of AR to push the field further. Unfortunately, it sounds like at least in part a lot of these new innovations didn’t make the cut for the conference because they lacked in strong evaluation or quantifiable contribution to the field — it’s much easier to judge the quality of a new camera-based image-registration method than it is to judge the usefulness of a brand new application.

The Software Agents field was a response to a similar stagnation in Artificial Intelligence. AI researchers had a lot of good but imperfect tools that had been developed over the years, but kept trying to solve the really hard general problems. Software Agents grew out of the idea that it was OK if your algorithm wasn’t perfect in every condition so long as you cleverly constrained your application domain and designed your user interface to cover for those imperfections. It was a struggle to get acceptance of the idea at first, and in the end a few of the big players in the new domain went and founded their own conference rather than try to fit their own work to the evaluation metrics used for more traditional AI papers. Hopefully it won’t take such a dramatic move on the part of AR researchers to breath new life into this field.