Inkwell: Authors and Artists

    

Boy, have you ever hit an important nail on the head, Dan.  This is a huge
area where our most commonly used computer interfaces could be greatly
improved.  Like many designers, I'm a Macintosh user and have been since
early 1984.  But even with the many amazing improvements in capability and
overall user experience in OSX, and especially in the latest "Tiger"
version, Apple continues to leave a lot of low-hanging fruit on their tree.

My biggest gripe is in the most common of system tasks I do again and again.
 Dozens and dozens of times every day, I'm using the Finder dialogs from
within open applications to find and open some other file.  And yet *every
single time I open a file finder,* I'm forced to repeat the following steps:

1)  Enlarge the file finder.  Despite the fact that I have a huge screen,
capable of showing me a long list, the file finder continues to appear in a
small, inefficient size.

2)  Click on the "Date Modified" column in the file listing (after
navigating to the directory I want to find the file within).  I produce
hundreds and hundreds of files, and organize them by daily iterative
groupings, so I'm always looking for the most recently modified files.  Yet
OSX cannot seem to remember how I prefer to sort and view my searches.  I
don't know if this is Apple's problem, or how application developers are
perhaps not availing themselves of Apple's capabilities. Like many
designers, I use Adobe Creative Suite programs intensively, and I know that
the file finder dialogs that appear when I'm using these products don't seem
to learn a thing about how I continually configure them.

These two simple interactions may not seem to be to taxing, but in being
forced to do them over and over again thousands and thousands of times,
Apple is literally stealing hours and hours of my attention and time.  If
you look at how these small and unintelligent burdens add up for individuals
over product lifespans, it's an incredible human cost.

While I'm among the biggest Apple fans on the planet, I'm continually amazed
at how they're missing many opportunities to do exactly what you're
suggesting - developing ways to better utilize users' behaviors to shape
their experience and efficiency.
  

    
IMO part of that repetition you're engaging in is, as you say, a theft of
your time and energy.

But part of it is not; it's what is required for any human to achieve deep
learning (mastery) of any physical task. Between intention and action are
your hands and an object to be manipulated. Small but frequent repetitions
of customary activity decrease cognitive load as muscle memory takes over.
(Indeed, it takes effort for an expert user to actually describe the
particular steps taken, just like a musician's fingers move effortlessly
without conscious direction.)

Further, many software-driven artifacts contain the ability to automate and
personalise the experience to make a more efficient interface for a
particular user. (The obvious example is a *nix command line; I can change
the conversation by adding my own tools, or those created by someone far
more competent than I. The other example that comes to mind is the personal
Amazon page, discussed in the book.)

> But might it not be better to subtly and over time have the users'
> observed behaviors create personal preferences?

I'm going to have to re-read that part of your book and offer a detailed
answer, since that's one of the most intriguing parts of the book IMO. My
simple answer is NO - there are costs exceeding the benefits most of the
time. You risk disrupting flow, essentially, and that's the biggest buzzkill
possible short of ending the conversation. It might work for low-bandwidth
interfaces such as text messaging with a numeric keypad.
  

    
A few things...

Csikszentmihalyi's ideas of flow involve maintaining stress or anxiety
levels in an area that is somewhat short of being overwhelmed. This
occurs by balancing the perceived challenge against the skills an
individual already has to tackle those challenges. Usually, flow
experiences involve a challenge that is slightly greater than the
skills an individual possesses. This explains why video games, for
example, often create flow experiences. The challenge increases as the
player completes levels and gains more skills and experience. With
task-based applications, the challenge should come from completing the
activity, as opposed to learning to use the system. If I'm authoring a
novel, I want to focus my attention on the quality of my writing, not
on learning the skills to use the Word Processor. 

In terms of our devices adapting to our usage patterns, the impression
I get is that some of the perceived failures of so-called "adaptive"
systems (e.g. Word adaptive menus), have made this an area few are
willing to explore. There seem to be inherent dangers in having systems
adapt over time to user behaviors. For one, such systems violate
certain core heuristics of usability like predictability and
consistency, and force users to repetitively scan lists that they would
already have memorized, had it not been for the continual
"adaptation".


That being said, I agree with Jim that certain "low-hanging fruit",
like search prefs, would appear to be ideally suited for adaptation. As
with anything in design, the details matter. Given the human tendency
to anthropomorphize, if the software, product or website in question
refuses to acknowledge changes in the behavior of its users, this could
easily be seen as "stubbornness", and trigger feelings of frustration,
affecting the user's relationship over time with the product.
  

    
A few things...

Csikszentmihalyi's ideas of flow involve maintaining stress or anxiety
levels in an area that is somewhat short of being overwhelmed. This
occurs by balancing the perceived challenge against the skills an
individual already has to tackle those challenges. Usually, flow
experiences involve a challenge that is slightly greater than the
skills an individual possesses. This explains why video games, for
example, often create flow experiences. The challenge increases as the
player completes levels and gains more skills and experience. With
task-based applications, the challenge should come from completing the
activity, as opposed to learning to use the system. If I'm authoring a
novel, I want to focus my attention on the quality of my writing, not
on learning the skills to use the Word Processor. 

In terms of our devices adapting to our usage patterns, the impression
I get is that some of the perceived failures of so-called "adaptive"
systems (e.g. Word adaptive menus), have made this an area few are
willing to explore. There seem to be inherent dangers in having systems
adapt over time to user behaviors. For one, such systems violate
certain core heuristics of usability like predictability and
consistency, and force users to repetitively scan lists that they would
already have memorized, had it not been for the continual
"adaptation".


That being said, I agree with Jim that certain "low-hanging fruit",
like search prefs, would appear to be ideally suited for adaptation. As
with anything in design, the details matter. Given the human tendency
to anthropomorphize, if the software, product or website in question
refuses to acknowledge changes in the behavior of its users, this could
easily be seen as "stubbornness", and trigger feelings of frustration,
affecting the user's relationship over time with the product.
  

    
The low-hanging fruit stuff should be a no-brainer. There's no reason
for the repetition of mindless tasks that interfere with getting work
done.

The trickier stuff is what you all are alluding to: how do you make it
so that high-value, high-skill tasks are adapted to your style of
working? It's relatively easy to make a point-and-shoot camera, much
harder to make a professional-grade camera that over time becomes
customized to a pro photographer.

You can do this poorly pretty easily too: "I see you're writing a
letter. Would you like me to help format it?" Ugh.
  

    
One more thing, Jim, since you are talking about Apple. Why oh why
hasn't someone--a large online company like Google, the Linux open
source community, Sun, a start-up, anyone--jumped ahead of the desktop
metaphor for an OS? I'd love to see more experimentation in this area.
I thought OSX was going to be a radical jump, but really it was more
evolutionary than revolutionary. And Windows Vista, it appears, will be
extremely compromised (when it ever comes out).

Any thoughts on this, folks?
  

    
Start by just remembering the most recent settings of everything.
  

    
Dan, did you ever use a system than ran MagicCap?  It started to get
out of the desktop metaphor by including other rooms of your house and
a street outside that connected to the world.
  

    
Have you seen The Brain, by TheBrain Technologies? It's a visual/spatial
system for organizing information. I'm ambivalent about it, but some people
swear by it.
  

    

> #81 of 84: Dan Saffer (dansaffer) Mon 16 Oct 2006
> (08:25 AM)
>
> One more thing, Jim, since you are talking about Apple. Why oh
> why hasn't someone--a large online company like Google, the
> Linux open source community, Sun, a start-up, anyone--jumped
> ahead of the desktop metaphor for an OS?


Heh.  Well funny that you should ask, because among the very first things I
began thinking about after getting a Macintosh in 1984 was what would truly
constitute a fundamental leap forward in user interfaces at the general
computing level.

By 1988, when I was working with Norm Cox and Alan Mandler on Sun
Microsystem's Open Look GUI, we were beginning to see a number of desktop
windowing GUIs, including Motif by Hewlett Packard (designed by Barry Mathis
and his design group.  Barry was an industrial designer, by the way),
NeXTStep for NeXT (which I joked at the time should be called 'NoTHER, since
even though it was a powerful and breaktrhrough operation system, was still
not moving beyond the then already established GUI paradigm), and IBM's OS2
Presentation Manager (which was worked on by Norm Cox, Edward Tufte, and Dan
Boyarski).  We also were seeing the emergence of a number of gestural pad
computers in 1989-90, including Go Corporation's slate pad.  Microsoft
succeeded in killing that by bringing out PenWindows, and then after all the
competition was dead, discontinuing that.  I still believe that Microsoft's
presence and dominance of the market in the 1990s was largely responsible
for the stifling of a great deal of innovation in the greater field.

It was in 1988 that I began to think seriously about what would truly be a
leap forward.  Through a variety of sources and research, I'd become very
interested in simple but powerful forms of information visualization. An
early database visualizer for the Macintosh, MacSpin,  projected pixels into
a rotatable 3D space, representing three axis of criteria.  As you moved
your mouse over the pixels in the resulting swarm, you could click and get
information about that object.  I was impressed and inspired by how powerful
it was, and yet how simple.

So I immediately began my InfoSpace project:

<http://www.well.com/user/jleft/orbit/infospace/index.html>

InfoSpace was a conceptual design and user interface framework that
established and modeled a number of then novel ideas.  First off, it
recognized the importance of moving beyond the limitations of a small
display (even our largest screens today are small, compared to the amount of
visual information we take in from the real world around us), and moving it
to a virtual environment.

Secondly, it introduced the idea of a networked information and media
browser, so that every computer would not be an island, but rather connected
to the net and able to search and retrieve information beyond its own local
domain.

Thirdly, it represented data in a variety of ways, including visualized and
interactive swarms of symbolic objects.  Instead of retrieving thousands of
items and displaying just the first few on a list, it displayed them all in
a swarm visualization, and allowed for metadata (what I called
"metainformation") representing many forms of comparative attributes to
determine each object's shape, size, spatial location, behavior, etc..  And
by being able to interact with each of these attribute criteria,
relationships and values of individual objects could be seen and noticed out
among the dense clusters, landscapes, or other visualized formats.

And fourth, the OS and the browser were seen as integrated and one and the
same.  The *flat* desktop would give way to a pragmatic and useful
visualized space, allowing our highly evolved visual cortexes to perceive
and become aware of much greater amounts of information than we were being
afforded by current computer interface models.  I contend that our computers
today are still, essentially, apeing the cave wall and two-dimensional
media, with some hyperlinking thrown in.  This is an incredible limitation
to our *awareness*.

I believe that one of the numerous revolutions in computing and information
science in the coming years will be the interactive visualization
revolution.

To speak to <cascio>'s comment about The Brain, which is a relationship
visualizer, I purposely went in an alternative direction with the InfoSpace
model.  I don't draw lines between objects to show relationships, but rather
rely on the manipulation of simple controls mapped to the many
visualization-supporting metadata criteria to produce noticeable reactions
amidst dense visualizations.

Some subsequent illustrations of these ideas are shown here:

<http://www.well.com/user/jleft/orbit/vizrev/slides/5.html>

<http://www.well.com/user/jleft/orbit/vizrev/slides/7.html>

<http://www.well.com/user/jleft/orbit/vizrev/slides/8.html>

When I arrived in Palo Alto in late 1989, I immediately arranged a meeting
with Apple's Advanced Technolgy Group (ATG) to present my browser ideas and
thoughts on the importance of interactive visualization.  I was told, "These
are interesting ideas, but we don't feel that the internet is the future of
the Macintosh."

I also had no luck in gaining interest at Xerox PARC, nor from any other
people in the computer science or information research communities.

I did, however, get interest from, of all places, the architecture
community.  But there's a huge bias against "practitioners" who are not part
of the Ph.D. researching community.  I never had any success at all, despite
multiple attempts, to collaborate or work with anyone at Interval Research,
and so over the nineties began to give up hope that the research community
was truly interested in any but a few specific, and often repeated ideas for
what the future will hold. These are all fine ideas and hold promise, such
as ubiquitous computing, and artificial intelligence programs.

But I still maintain that the most powerful augmentative technology in the
future will be interactive visualization methods that allow computers to
present data in a way that our incredibly powerful visual cortexes will
process in pre-cognitive ways.

For instance, when I step outside into my yard, I don't see a sign that
says, "I'm a yard.  Your search has retrieved 12,289,321,461 blades of
grass.  Here are the first twenty in list form."  Instead, I see the entire
yard, and can see dandelions and stray basketballs and mole hills.  I see it
all.  Imagine visualizing data that way, but also being able to
interactively explore it, not by zooming around in it (a common
misconception, I've found), but rather seeing it as an overview and getting
data points to show themselves when various criteria and attributes are
invoked and manipulated.

This, in my opinion, is the only way in which humans will be able to use our
powerful biological means to leverage an awareness of the vast amounts of
information and interrelationships that are out there.

But hey, I'm not a Ph.D.  I'm just a practitioner.
  

    
>>> I still believe that Microsoft's presence and dominance of the
market in the 1990s was largely responsible for the stifling of a great
deal of innovation in the greater field.

Those fukkers set back the desktop by at least a decade. Gopod knows
where we would have been without the guys who got the Web going. 

>>> I did, however, get interest from, of all places, the architecture
community.

That's because they're interested in usable design. When I was at
AutoDesk and briefly working on UI stuff we had some of our best
conversations with architects. 
  

    

Dan also brought up the Linux Open Source community, which I've always held
out great hope for.  Though I think everyone realizes that it faces some
significant challenges, when it comes to developing systems that are large
in scale and really deserve to have very integrated user experience
architectures.  Much of what we've seen from the Open Source community in
the way of desktop interfaces are essentially equivalencies of existing
GUIs.

I don't blame the lack of major innovation and continued large leaps forward
in interface technology solely on Microsoft, but I do believe they were
responsible in large part for the "grey panel" world that descended on the
otherwise promising decade of the 1990s.  But our fields inability to learn
to accelerate the rate at which we innovate is evident from the history.
Engelbart first showed window and mouse GUIs in the 1960s.  It took a decade
for that to emerge in the products of Xerox PARC.  And it was yet another
decade for Apple to properly productize it.  And another decade for
Microsoft to spread a dull grey version of it to the larger world, and so
on.  We can't afford to innovate our user experiences at this slow rate.  We
need to work harder, and seek more ways, and engage more types of people to
innovate (and actually implement, I hasten to add) faster.

If you go back to the period of the late 1980s / early 1990s, there were a
number of exciting and alternative interfaces that were emerging. One
promising overlap, which may eventually still appear, was the potential
convergence of gaming interfaces and desktop software.

Some of you may recall the incredible film from 1991, Wim Wender's "Until
The End Of The World."  I still consider it to be the greatest user
interface film of all time.  Set in the future of the turn of the millenium,
the film featured many different types of well thought out interfaces,
ranging from in-car customized navigation systems to desktop software that
showed animated agents carrying out tasks and demonstrating key transitions
in processes.

I remember my friends in the field of interface design at the time all felt
this was a major and realistic look at how user interfaces could begin to
evolve.

Also, my subsequent exposure to the game software industry, which is as
different and separate from the corporate desktop software world as day is
from night, showed me some real differences in how development was done in
those two domains.

All I can say is thank our lucky stars that Steve Jobs came back to Apple,
turned the place upside down, and began innovating and pushing forward
again.  But there are so many *different directions* we can, and need to go
in.
  

    
I've never used MagicCap, but I have used The Brain and, while
interesting, found it overly time-consuming and frustrating to use. But
The Brain is more of a new filing system than it is a complete OS. (I
made an addition to the current hierarchical filing system--piles--in
my Master's thesis: FilePiles.)

That's an interesting story, Jim. The desktop metaphor has been so
successful and wide-spread that it's hampered development and use of
other metaphors. I'd say this is especially true in mobile devices. A
desktop on a small screen is pretty ridiculous.
  

    
Michael, I'd say that SOME architects care about usable and useful
designs. Others, like many of the starchitects, don't give a rat's arse
about that, only imposing their vision on the building's occupants and
the cityscape. Rem Koolhaus springs to mind, especially. Aside from
some (Frank Gehry for example), I get more inspiration from "outsider"
architects and urban planners like Christopher Alexander, Jane Jacobs,
and the Learning from Las Vegas authors. They have a lot to teach us
about creating spaces for interaction in a humane way, I think.
  

    
I've never seen "Until The End of the World." I guess it gets added to
my Netflix queue now. :)

The Open Source Community is going to need some designers in it if
we're going to push the boundaries of the OS. I think the gaming
industry is a likely source of new OSes and new interaction paradigms.
I mean, look at all the attention the Wii has gotten lately for
gestural interfaces! And hell, the PSP, Xbox, Playstation, Nintendo,
etc ARE OS systems already, we just put them into the category of
"gaming consols" and leave them there, when, sitting in millions of
living rooms, is a giant hard drive with new input devices and a large
screen. Probably several of these new OSes, including Tivo.
  

    
>They have a lot to teach us about creating spaces for interaction in
a humane way, I think.<

Isn't one of the obstacles the previous conditioning and conventional
aesthetics that most of us are subject to, conscious and/or
unconscious?

This is sort of an "installed base" that represents a considerable
investment.  We may all have some areas of our lives where we find that
"improvement" isn't worth the effort to change.
  

    
>>> If you go back to the period of the late 1980s / early 1990s,
there were a number of exciting and alternative interfaces that were
emerging

It's not just interfaces. Think about all the different Unix vendors
there were in the mid 1980's. We had a chance for all kinds of
innovation on the OS level, and M$ squelched it.

>>> Michael, I'd say that SOME architects care about usable and useful
designs.

True, but all the ones I worked with cared deeply (I studied
Environmental Design at Cal, or tried to since I was actually an
Engineering major, and started my career doing computer analysis for
same).
  

    
(Well, minus one guy who was only interested in the bucks).
  

    
I agree strongly with #91.

I think interface design change is like any other technological evolution:
what's extant is Good Enough, so incremental improvements will be ignored.
Substantial changes will be required, and they need to have low pain of
adoption as well as order-of-mangitude improvement over what already exists.

It's faintly ridiculous that we're still abusing the desktop metaphor 20+
years after Macintosh made it commonly-known. It's more ridiculous that I'm
supplying these words via a keyboard whose design has injured me. Ehh.

Mobile devices are where change has happened, imo - simple chat and email
have changed quite a bit as people struggle to find new approaches to using
the devices to communicate.
  

    
I suppose the question is one of form. Have we found the correct,
stable form for digital devices, in the same way we found it for cars,
say, where there are only incremental changes, not anything
revolutionary for decades? We might have, but I'm skeptical. The medium
is so fluid and can be so ubiquitous and unbound that I doubt we're
anywhere near a stable form for it yet. But perhaps I'm wrong. Perhaps
we (and by we I mean mainly the good people at Xerox Parc) guessed
correctly all those years ago. The web, however, seems to point
otherwise, and elsewhere...
  

    
I don't have anything to contribute to this discussion at the
moment, but I just wanted to let you guys know that I am 
reading along, and enjoying it.  Thanks.
  

    
The "installed base" of what is familiar and known is a significant
obstacle to overcome. Which is why--and rightfully so--most changes are
incremental. it's simply easier, both to design and develop and for
users to understand. Raymond Lowey's brilliant term sums it up so well:
MAYA. Most Advanced Yet Acceptable.

How do you create that breakthrough product that changes the category
is the real question. Because if it is demonstratively better like the
iPod, Tivo, and the Palm Pilot, users will get it. Maybe it will take a
while (the iPod took a while to become a huge hit, Tivo is still
struggling), but the user base can and will change.
  

    
Isn't this really a question of efficiency? Sure, many of these new OS
paradigms are very interesting ideas that look "cool". But none of the
ones I have seen can compete with the highly efficient action of
moving a mouse a couple of inches on a table to make a cursor cross a
20" monitor.

The WII controller is great for games and entertainment focused
applications. But, for basic task work, I want something that will
improve my movement efficiency by taking small motions and scaling them
up to the tool I'm using. The interfaces in, for example, Minority
Report, looked great on screen, with users manipulating files through
broad, sweeping gestural movements. But an interface like this would
also require high levels of upper body strength to use all day, and be
physically taxing. Can anyone say repetitive strain injury?

I see the innovations (albeit incremental) occuring more in the
information visualization realm as Jim had suggested.

Does anyone know of an interface input that is as efficient as a mouse
and cursor setup, in terms of scaling up small movements, but improves
on the inherent weaknesses of this input method? (aside from
Command-line) ;)

  
  

    
The click wheel did if for the ipod, but it's well suited to that task
(of scrolling through thousands of songs).

What about more general computer input?
  

    <scribbled by gail Tue 17 Oct 06 14:30>