We have the Hardware…

I have been thinking about the Apple iPad, specifically how it relates
to other devices.
I then recalled my undergraduate course in human computer interaction,
and the work that was done on ubiquitous computing.
I looked about me, and realised that I was surrounded by devices that
were restricted to a few research labs fifteen years ago.
The hardware is all there, with a few differences, but we are a long
way from the vision of computing that came out of the last decade of
the 20th century.

In the 1970s the Xerox Palo Alto Research Center (PARC) gave birth to
the computing interface that would be used for the
next 30 years (at least):
the personal computer, modern graphical user interfaces, Ethernet
networking, laser printers, and object-oriented programming.
Famously, Apple
stole most of these ideas to create the unsuccessful Lisa and
highly successful Macintosh computers.
As a follow-up, under the leadership of Mark Weiser, Xerox PARC started
work on the next generation of devices.

What they came up with was the concept of ubiquitous computing
— where computers will be vastly better at getting out of the
way, allowing people to just go about their lives

(Weiser, 1993).
To achieve this Xerox PARC decided that we needed three devices to
supplement, or largely take over, from the personal computer: tabs,
pads and boards.



A cell phone is a common form of a tab
interface.
It is about the same size as a Post-It Note™



A pad interface is about the same size as the
screen on a netbook.
The Apple iPad is a very similar size.



The original board interface was used as a
collaborative drawing and presentation space.
The shared space created by game consoles are more common.

Tabs

Tabs were the smallest interface; about the same size as a
Post-It Note.
Originally they had very limited capabilities, with 128KB of store and
the ability to play video at four frames a second.
Tabs did have wireless networking, using a bespoke system that Xerox
PARC developed, so they could work like a two-way pager (a device that
the original tabs superficially resembled).
Input was provided by a pen, but because of the limited processing
power a simplified alphabet called Unistrokes was used
(Goldberg and
Richardson, 1993).

Within an office building a tab could report its position, so your
workmates will know where to find you, a bit like an advanced RFID
security card.

Cell phones (alias mobile phones) are the same size as tabs.
The intervening few years have seen store increase into hundreds of
gigabytes,
video increase to full-motion,
three wireless protocols (Bluetooth, 802.11, and GSM) are typically
used for communication,
and GPS is used to track the location of the cell phone anywhere in
the world (except in an office building, where satellite coverage can
be a bit spotty).

Pads

Pads, or tablets, have had a longer history than tabs, as far as I can
figure out.
Commercial pen-based interfaces had been available for a couple of
years by the time Xerox PARC started working on the MPad.
Unlike the commercial offerings of the time, the MPad had wireless
networking,
multiprocessing (unlike the not-yet-shipped Apple iPad), and multimedia
capabilities.
However, it was the software that made the MPad dramatically different
from the current crops of pads, but more on that later.

The pad has had a hard time commercially.
Microsoft has been pushing tablets for over twenty years without much
success (I can recall booting a Windows 3.1 for Pen Computing machine).
The Apple Newton was problematic enough to get
a mention on
The Simpsons.

Far more successful has been the netbook, which has a user
interface similar to a desktop but it is as portable as a pad, for a
lot less money.
Interestingly, some netbooks are acquiring touch-screens, so they
can behave even more
like
tablets.
(The photo above is of a netbook, rather than an actual pad.)

Boards

Of the three interfaces I recall learning about as a student, the
board is the first one I can recall ever seeing, sort of.
The idea was to create a shared space where many people could interact

much like a whiteboard.
The Xerox LiveBoard
(Elrod et al, 1992)
was the result of the PARC work on boards, which actually made
it to market as a Xerox product.
It could be used to record drawings, and make presentations, blazing
a trail that Microsoft PowerPoint and Apple Keynote follow.

I first came across the LiveBoard concepts with the electronic
whiteboards, which would record and play back the drawings that
were made on it.
However, game consoles are most common board-sized interface that I
come across.
They drive million-pixel displays, providing a shared space where
people can interact using wireless controls.

The Hardware Difference: Keyboards

The tab developed by Xerox PARC did not have a keyboard as they have a
very small interaction area — too small for a keyboard

(Weiser, 1993).
The Palm Pilot and Apple Newton had pen-based input like the Xerox
tab.
However, most tab-sized devices are not pen-based: even the Apple
iPhone uses a keyboard when text needs to be entered.
I do not know why this is the case.
Maybe it was
a
long running patent suit
(1997–2006) brought by Xerox
against Palm (over the Xerox Unistrokes system, on which Grafitti was
based) that made others wary.
Maybe keyboards are easier to implement and learn.
Maybe the tricks used by tab-like devices to allow text input
(like multi-tap and T9) are good enough:
the speed of input using T9 is
14.63wpm
(? 1.09)
(Wobbrock et al, 2007)
compared to
15.8wpm (? 4.02) for Unistrokes
(Castellucci et al, 2008).
The Unistrokes patent must be close to expiring, so we will see
soon if someone picks up the pen-input ball and runs with it.

Keyboards were optional on the MPad developed by Xerox,
all netbooks have a physical keyboard, while the Apple iPad relies on
an on-screen keyboard.
As for boards, the
Nintendo
Wii,

Sony
Playstation 3,

Microsoft
XBox 360
consoles all have an optional keyboard.
In addition, Microsoft PowerPoint and Apple Keynote presentations
are usually presented on workstations acting as boards and
the keyboards are vital to the creation of many presentations (sadly).

…But Where is the Software?

Reading back across the ubiquitous computing papers, old and new,
I am amazed at how far the hardware has come, and how little the
software has changed.
The original concept was to move beyond the intimate nature
of personal computing, and to create devices that were better suited to
communication and collaboration.
The software would allow tabs, pads, and boards to interact with each
other; not just your own devices, but those used by others.
A presentation could be displayed on a board, appear simultaneously
on pads (for annotation), and be controlled by a tab
(Myers, 2001).
A drawing made on one persons’s tab would appear on the shared board.
A running program would follow you (well, your tab) from your
workstation to another person’s office, where it will display itself so
you could discuss it, a bit like cut and paste writ large
(Rekimoto (1997)
dubbed it pick-and-drop).
There was excited (albeit awkward) talk of shared situations and
collaboration.
However,
the seamless interplay between the multiple device surfaces
that Weiser imagined is still far from reality

(Klokmose et al,
2009).

Recent Apple products also lack the joy of ubiquitous computing.
The language that
Steve Jobs uses

to talk about the iPad is still the language of the personal computer.
Take the list of tasks that the iPad has to support:

  • Browsing (which did not exist at the time of the Xerox MPad),
  • Reading email,
  • Viewing your photos,
  • Viewing video,
  • Listening to music,
  • Playing games,
  • Reading eBooks

There is no shared space, beyond email (which is older that I am)
and Web browsing (which is fifteen years old by my count).
Instead he claims that unless a pad is better than those tasks listed
above it has no reason for being.
He then goes on to discuss how the iPad can be personalised and
discusses the many single-user tasks for the iPad.
When he briefly discusses sharing, in the context of photos, the talk
is centred around a single device.

I am not a ubiquitous computing researcher (I studied undo) so I cannot
offer any insight into why the software is not there, while the
hardware is more than capable.
Instead I take heart that there are still people working away at the
problem
— publishing papers about
frameworks
and
software
architectures,

as user-interface researchers do when they need something to present and the coding is harder than expected.
Software is also what free and open-source development is good at.
With the free and open software present in
tabs of all shapes and
sizes,
pads
and
boards
I hope that some hackers may be inspired to take up the ubiquitous
computing ideas, dodge the remaining patents, and create software that
brings people together, allows them to share situations (not matter
how awkward) and make the world a better place.

Me? I am still trying to sort out email…

One Response to “We have the Hardware…”

Leave a Reply

You must be logged in to post a comment.