DocBug

Very fly

I picked up a Fly Pentop the other to play with (one of the advantages of being a user interface researcher is all the toys :). Here're a few thoughts.

There're at least four challenges with using a real ink pen as a computer interface:

1. There's no way to constrain where the user draws: strokes that should be invalid in your application leave marks just the same as valid strokes.
2. There's no way to tap or "drag" without leaving a mark, and no way to erase marks after you've left them.
3. There's no display to give feedback to your actions or computation. (The Fly uses audio for feedback, which gets around this limitation to some extent.)
4. You need a pre-printed sheet of paper for every pre-designed interface — the Fly comes with about 15 pre-printed games sheets, everything from word-search pages to maps you can tap on to test your geography knowledge. That doesn't scale well when dealing with applications with many different pages or when maintaining large numbers of applications, especially given that pages get "used up" when you write on them. On the other hand, the fact that paper is consumable makes for market opportunities that normal software doesn't have... I expect Leapfrog isn't too upset about that fact.
5. If you don't use a pre-designed interface (that is, you start with a blank sheet of paper) then the user is forced to draw the entire interface. The Fly has one game where you draw a piano keyboard that you can then play by tapping on the keys, but first it gives you explicit instructions like "draw 9 short vertical lines, going from left to right" to make sure you draw a keyboard it can understand.

In spite of these limitations, it's extremely engaging to be able to draw your own functional user interface — as anyone who read Harold and the Purple Crayon or watched Simon in the Land of Chalk Drawings as a kid knows. The effect really hit me when I was making a calculator. First I wrote the letter "C" and circled it to enter calculator mode. Then, as the pen spoke instructions to me, I drew a big rectangle and started to fill it with numbers and arithmetic symbols. I realized about three numbers in that I didn't have to stick to the usual layout and placed the rest of the numbers going up, down and sideways. Then I tapped on the numbers with the pen to type out 22 + 44..." only to discover I'd forgotten to draw an equals sign. I quickly drew one in, then tapped it to hear the pen speak "22 + 44 equals 66". It was as if I were running from something in the land of chalk drawings and someone suggested we draw a door so we could escape!

The interface also feels more magical than it would if it were implemented on a tablet PC. This could be a novelty effect — I'm used to paper being static and non-functional and computer screens being reactive — but I think it's also because it feels like the pen is reacting to my physical environment, rather than simply reacting to the way I interact with it. When I interact with a tablet PC, I think of the computer as being the screen (even if the actual CPU is somewhere else). With the Fly, I think of the pen and speaker as being the device, but not the paper. That means even though a tablet PC and the pentop computer might implement the exact same interface, I feel more of an emotional attachment with the pen because it appears to be observing and sharing my external environment and not just the actions I perform directly on the device.