Inkwell: Authors and Artists

    
We're pleased to welcome David Weinberger back to the Inkwell.

David Weinberger got a Ph.D. in philosophy from the U of Toronto in 1978. He
then taught philosophy for six years. Because there was no tenure track
where he taught, he left academics and became a marketing writer for
Interleaf in the mid-1980s; Interleaf made early electronic document
software, with special capabilities with structured documents and SGML. He
left in 1994 as VP of Strategic Marketing and became a marketing consultant.
In the mid-1990s, he was VP of Strategic Marketing for Open Text, a search
engine company becoming a pioneer in intranet collaborative software. Since
then he has been a consultant, writer and speaker.

Throughout his career, he has been a writer, published in a wide range of
journals, from Wired to Harvard Business Review to TV Guide. For almost ten
years, he's been a commentator on "All Things Considered." He is a co-author
of The Cluetrain Manifesto, the author of Small Pieces Loosely joined, and
an early and avid blogger. His new book, Everything Is Miscellaneous, was
published on May 1.

For the past three years, he has been a fellow at Harvard Law's Berkman
Center for Internet & Society. Next year he'll be co-teaching a course at
Harvard Law on whether the Web is different or just more of the same.

Leading the conversation with David is Jon Lebkowsky. Jon is an authority on
social media, web community, user experience design, and effective web
strategy. He has worked as a consultant, CEO, technology director, project
manager, systems analyst, and online community developer. He is also
knowledgeable of Internet policy and trends.

Great to have you guys here!
  

    
Hello to all, and a big welcome to David Weinberger. David, my first
question is not imaginative, but I'm always fascinated by the diverse
and interesting ways authors are drawn to book projects, and how those
projects evolve. What motivated you to write _Everything is
Miscellaneous_, and how close is the final book to the original
concept?
  

    
Hah! Because irony is the strongest force in the universe, "Everything
Is Miscellaneous" (can we call it EiM from now on, before our carpals
tunnel their way to freedom?) went through 18 months of full time
slicing, dicing, and rearranging before I started writing. I knew what
phenomenon I wanted to examine, and I had a sense of the set of topics
and examples I'd be dealing with, but the overall theme kept shifting.
As did the title.

I knew I was interested in the importance of metadata. That's what's
enabled us to flourish rather than drown in the age of information
overload. But, I had been through two stretches that convinced me that
making metadata explicit often leads to paltry, inadequate results.
Also, it hurts.

First, in the philosophical portion of my "career," I reacted
viscerally against the dominant schools of thought that assumed the aim
of philosophy is to be clear and distinct. Analysis of course has its
place, but it has always seemed obvious to me that most of what matters
can't be said. So, when a Web form asks you to list your interests,
you freeze. I do, anyway. Or if you ask me to describe my children, at
the end I'll always be left with the feeling that I left out the most
important parts. The power of language - as many have noted - is in
what it doesn't say. (Ring one up on the Cliche-O-Meter!)

Second, in the late 1980's and early 1990's, I survived the SGML wars.
SGML would enable complex document sets to be created, maintained,
retrieved, and reused far more easily. But, industries couldn't agree
on the details of which metadata to capture, and lots  writers saw the
creation of metadata as red tape. Explicit metadata sucks. Usually.
(Cory Doctorow made these points in his enlightening and entertaining
way in his MetaCrap article.)

So, I knew I wanted to write about metadata because it's crucial and
maddening and elusive.  On the other hand, who cares about metadata?
What counts is the effect it's having on our institutions and their
authority. So, at various times, the rubric of the book was the promise
of the implicit, messiness as a virtue, social knowledge, and even
(for about four minutes) the problem with Aristotle. 

Only eventually did I realize the book was about the emerging
principles of organization. And then late in the process the
"miscellaneous" - a term I'm sure I'll regret using - became the
official topic of the book.
  

    
To me it seems like a comprehensive examination of the state of knowledge or 
knowledge architecture in a world where pretty much all information that's 
stored is stored digitally and enhanced with metadata. It's a world where 
everyone should be assigned a librarian at birth... and I notice that you've 
dedicated the book "to the librarians." Are librarians the new masters of the 
universe?
  

    
The key is the "enhanced with metadata." Otherwise the miscellaneous
would remain miscellaneous...unlike things next to unlike things. But
the digital miscellaneous is the _potential_ for finding and arranging
ideas, information and works along whatever lines of likeness we want. 

We always and ever keep the miscellaneous from staying miscellaneous:
We do a search that clusters items one way, and then we read a playlist
that clusters them another. We can do this because the organization of
the miscellaneous not only uses metadata to make the connections
("Here are all the photos tagged 'iraq'"), the organization is itself
metadata: A playlist pulls together pointers to files without
rearranging the files themselves.

So, in this world, where are the librarians? They have been the
masters of the metadata we can anticipate people will find useful. We
can predict people will want to search by author, title, topic, year.
The challenge now is also to help us pull together what we need based
on metadata no one can anticipate.
  

    
Can you point to examples where that's happening?
  

    
The Web is just about all examples of us pulling together information
in unanticipated ways, although the range of metadata we use for this
goes from carved in stone (granted, stone-based forms are a bear to
fill in) to deducing metadata from implicit traces.

At one end, there are the many ecommerce sites that carefully
categorize their offerings based on a taxonomy they built or glommed.
But even a site that thinks that each product has to go in a single
category -- and why would you want that? -- still will give us multiple
ways to browse and search. Maybe there's a sloppy eater who wants to
see only size 16 shirts, in no-iron cotton, with short sleeves,
available now, in any pattern that will hide food stains. The more
metadata, the happier that user's experience will be.

I'm finding encouraging the rise of sites that use faceted
classification, which lets users browse a taxonomic tree via
pre-established metadata categories, but browse that tree in whatever
order suits them at the moment. The root in one session can be a fourth
level branch in the next. (<a
href="http://flamenco.berkeley.edu/";>Flamenco</a> is an Open Source
system. <a href="http://www.endeca.com";>Endeca</a> and <a
href=http://siderean.com/";>Siderean</a> provide commercial systems. <a
href="http://www.newegg.com";>NewEgg.com</a>'s "guided navigation"
system is an example of facets at work.)

At the other end, there are the social tagging sites (<a
href="http://www.flickr.com";>Flickr.com</a> and <a
href="http://www.delicious.com";>del.icio.us</a> are the canonical
examples, of course) where each user can make up her own categories. If
Flickr tried to build a taxonomy for photos -- essentially, a taxonomy
of visible things, plus all the meanings associated with human
experience -- it still couldn't anticipate that this photo of a sand
castle is actually a memento of my anniversary. (<a
href="http://www.corbis.com";>Corbis</a>, by the way, does have a
taxonomy of photos, including a controlled vocabulary for evocative
images. It's useful and it helps, but allowing user tagging would help,
too.)

Then, of course, there are the ways services such as Google and Amazon
use the metadata we've left behind for other purposes (e.g., links we
create in pages, links we've clicked on) to try to induce what will be
relevant to us. That can be used against us, for the simple reason that
there's always something more to worry about. 

Lot's of what's most exciting on the Web is, in fact, about helping us
put together pieces without first requiring that those pieces be
carefully categorized by experts. Rather than filtering and organizing
on the way in, we're enabling users to filter on the way out. That
enables them to find, re-find and understand in ways that reflect their
particular project, interests and context.
  

    
No questions yet, but this is fascinating reading!
  

    
It occurs to me that we need a glossary moment here... it would be helpful if
you define some of the terms we're going to be using in talking about the
book.  I'm thinking about ontology, taxonomy, folksonomy, and metadata, first
off. I think ontology/taxonomy are especially critical, and hard for some, to
understand. Could you say a bit about how you define these terms, and (big 
question, I know) how they factor into your exploration via the book?
  

    
Yes, please explain.  I first learned meta in chemistry class
referring to the positions of chemical groups on a benzene ring, and
it's all over the digital photography forums as the non-image
information that is part of the image file.  But I'm hearing it now
used in very general conversations where the context is obviously quite
different.
  

    
I'll start with metadata because it's the hardest to pin down. I use
the term loosely ... too loosely for some, who justifiably charge me
with Aggravated Metadata Abuse.

Metadata is, of course, information about information. Back in the
day, it was easier to know which was the data and which was the meta.
The book on the shelf was data (purists may disagree with me) and the
catalog card was metadata. In fact, this corresponds to the first two
"orders of order" my book postulates. In the first order, you organize
the things themselves: The books on the shelves, the bolts in the bins,
the cans in the larder. In the second order, you physically separate
the metadata from the things, you generally reduce the metadata to what
fits on a card or label, and you organize them: The library's card
catalog, the map of the items in the warehouse. In the second order,
you frequently can manage multiple sorts (subject, author, title),
whereas the first order requires you to put each thing in one and only
one spot, because that's atoms are mean that way.

In the third order, the content and the metadata are all digital. We
can now organize free of the constraints of the physical. The old
principles of organization are ill-suited to this new environment, so
we have to invent new ones...which is what my book is about.

Now, metadata gets mushy in the third order because when both the
content of (say) a book  is on line, we can use that content as
metadata. So, we can ask "What was that tragedy Shakespeare wrote in
1599?" using the author, genre and year as metadata, or we can ask,
"When did Shakespeare write the play that has the line about someone
having 'smote the sledded Polacks'?" The content becomes metadata. So
the difference becomes operational: Metadata is what we know and data
is what we're looking for.

This makes metadata squishier as a concept, but it makes our species
smarter. Everything that is linked to anything else becomes a lever by
which we can pry up new knowledge.

I'll give a faster glossary of the other terms you suggest, Jon, in a
separate message...
  

    
Quick glossary of the other terms Jon suggests...

Taxonomy: The way we organize a set of categories. In a paradigmatic
taxonomy, categories are arranged in neat super-categories that are
themselves clustered into super-super-categories, etc. That is, it's a
tree, like the tree of species, with "living things" as the root, then
"plants" and "animals" as branches, and then branches of the branches,
etc. In such a taxonomy, every leaf hangs from one and only one branch.
(Note: For a while, the working title of my book was "A Leaf on Many
Branches," because that's one of the ways the new digital principles
usefully violate the old.)

Ontology: Taxonomies find single, simple relationships among the many
leaves on its branches. (In a business taxonomy, AKA an org chart, the
relationship is that of authority/power.)  An ontology maps the complex
relationships within a domain, where the relationships can be
different among the pieces, and pieces can have many, many
relationships. The Semantic Web has found special value in ontologies
since they capture more information than taxonomies do and hold the
hope for enabling different domains to interchange information. (In my
book, I am skeptical about the utility of building huge, honking
ontologies that try to capture domains as large as the legal system. I
am more positive about the value of building lots of small, local
ontologies, and then messily stitching them together.)

Folksonomy: <a href="http://www.vanderwal.net";>Thomas Vanderwal</a>
coined this term to denote a taxonomy that arises bottom up by people
tagging socially.  So, if at Flickr, people tag photos of San Francisco
as "SF" 73% of the time and as "Frisco" 2% of the time, you're better
off searching for photos tagged "SF" if you want pictures of Baghdad by
the Sea (which I think we actually no longer call San Francisco, for
sad reasons). 

Now, I've been on a little tear recently to try to make it clear that
the value of folksonomies is not that they replace top-down taxonomies
with a system of categories that better reflects how users think.
Folksonomies have that value, but the real benefit is (imo) that
folksonomies replace rigid taxonomies with flexible, soft-edged,
multiply-meaned clouds of inter-related data. So, even if 73% tag
photos "SF," if you think of the city as "Frisco," you can still find
those photos, because folksonomies are rich with data, including the
fact that items tagged "Frisco" are also frequently tagged "SF."
Taxonomies restrict information in order to make organization clear.
Folksonomies can enrich information.

But, we have in our Western culture thought that the basic order of
the universe itself was taxonomic. The challenge to taxonomies is
therefore not just about coming up with practical ways to retrieve
information. 
  

    
the third order ability to 'filter on the way out' seems to me the catalyst
for social computing, business intelligence and agile development.

flickr and technorati and youtube and others pull the metadata from the
crowd, let them rank the content and all of a sudden knowledge
acquisition is free.  coming from SGML to this is quite a change in the way
we think about metadata, what it's good for, who creates it and what happens
next.

looking back to Cluetrain Manifesto, you saw that this wisdom of the crowd
would find its way into the knowledge base, and you warned the advertisers.
Did they listen?
  

    
How does "tagging" become "tagging socially"?
  

    
(bbraasch slipped in with a post while I was reading and writing - with a 
very good question!)
  

    

(NOTE: Offsite readers with questions or comments may email them to
 <inkwell@well.com> to have them added to this thread)
  

    
Administrivia: David and I will be offline for much of the afternoon. 
  

    
bill braasch, mainstream advertisers are starting to pay attention to
the structural changes in the market. Too often that means, though,
that they put out a call for customers to create the same sort of ads
that the company has been producing. The more threatening consequence
to advertisers is that as our ability to _pull_ the info we need gets
better, the info _pushed_ on us by advertisers seems more intrusive,
more obnoxious, and less relevant.

FWIW, in the past twelve months or so, there's been a marked increase
in the overall interest of marketers in Cluetrain. Sales of the book
have gone up a lot (still low numbers, but not bad now for a book 7
years old), and there's been more interest from marketing conferences.
I figure this is due simply to the fact that there's no longer any
denying that a big and influential chunk of the market is on line.
  

    
Jon, tagging becomes social when it's done via a site (or service)
that enables others to see what you've tagged, and what tags you've
used. At such sites, you can click on a tag and see all the resources
anyone at the site has tagged that way. You can typically even
subscribe to a tag and have the stream of results delivered to you.

There's actually a controversy about this among folksonomists. Some
say that it's only a genuine folksonomy if the taggers tag selfishly,
using the tags that will best help them re-find what they've tagged,
without regard to what might help others find it. E.g., I might tag the
photo of San Francisco as "visit someday" or "grandma's apartment"
which would help me re-find it, but not help anyone else. If I were to
tag altruistically, I might tag it "SF" because I see that's the most
popular tag for San Francisco, even if I tend to think of the city as
"Frisco." By tagging something "SF," I'm helping others to find it. 

The group advocating selfish tagging makes two points. First, the
primary aim of tagging is to help you re-find resources, and for that
you should use the tags most meaningful to you. Second, if people tag
socially, it can distort the folksonomy's representation of the bottom
up landscape of concepts. For example, if I tag it "SF" because that's
the majority tag, I've just increased by one the momentum behind that
tag. Maybe if people had tagged the city purely selfishly, we'd
discover that "Frisco" is really the most popular way of thinking of
it. If you want to see this in action, do a search at eBay for
"laptops," "notebooks," and "portables." That will tell you which "tag"
you want to use when you're trying to sell your
laptop/notebook/portable computer.

For myself, I think we use tags for different reasons. Sometimes I tag
selfishly because I don't care if anyone else finds it. Sometimes I
tag socially because I like feeling that I'm contributing to a public
stream of knowledge. Sometimes I use both types of tags.

Tag me "ambidextrous."
  

    
yeah, there are two kind of tags, but they are both called tags.

I put a flickr slide viewer on the town blog at <http://www.sighbo.org> and
used the tag for the town name.  the town wants to remain anonymous so I
will not type it into this widely read page.  Anyway, I found a couple
hundred photos of people who came to this town to take pictures of
themselves.  there were some nice vistas, familiar scenes too, but too many
pics of people I thought might find it to be embarrassing to be on the town
blog.

I tried 'bobo', as that is what the locals call the place.  Lots of pets are
named bobo.  Nary a pic of the town.

I switched to 'west marin' and got a nice slideshow.

I think we need a tag type attribute.
  

    
Interesting take on it!  (In a group I met with recently there was 
discussion of a hashed unique tag for specific group events.  Still, 
if attention is the game, desire to mimic tags for attention can 
become as important as the problems of synonyms and non-unique names 
or monograms.) 
  

    
Good point, Gail. We're enough at the beginning that the tagging
"namespace" is relatively uncluttered...although Flickr already has
hundreds of thousands of different tags. Still, you can probably get
away with asking attendees at the Worldwide Association Confederation
Guild 2007 conference to tag their blog posts "wacg07." And even if the
Westchester Area Conference of Gofers 2007 meets later and asks its
attendees to use exactly the same tag, we can use the dates of the
posts as additional metadata to separate the WACGs from the WACGs. It
may not be perfect, but will it matter that much if you read a stray
post from the other conference? 

And, where it does matter, we'll come up with something else.
  

    
bbraasch, great example. And the problem you point to will become more
pressing as more tags collide (same tag, different meaning) and
diverge (different tags, same meaning). Typed tags (i.e., meta-metadata
that says that the tag "bobo" refers to a place) make sense in some
applications, but at some point it turns into filling in a form, a task
people view as a chore to avoid. So, I think in general we should be
careful about going down that route.

There are other possibilities, fortunately. E.g., Flickr picks up tons
of metadata encoded in the photo files it gets from cameras. So,
Flickr knows for most photos whether the flash fired, what the aperture
was, what the date was, etc. At some point, cameras will also
automatically encode their latitude and longitude. That information can
be used to disambiguate clashing tags.

And Flickr does a remarkable job in many instances analyzing the
relationships among tags to determine whether this is a photo of the
Island of Capri or of the Ford Capri. That's how Flickr clusters
photos, and it works with surprising precision. So, it turns out that
as we get more and more tags, chaos doesn't necessarily result. In
fact, the statistic analyses get better.
  

    
bbraasch, great example. And the problem you point to will become more
pressing as more tags collide (same tag, different meaning) and
diverge (different tags, same meaning). Typed tags (i.e., meta-metadata
that says that the tag "bobo" refers to a place) make sense in some
applications, but at some point it turns into filling in a form, a task
people view as a chore to avoid. So, I think in general we should be
careful about going down that route.

There are other possibilities, fortunately. E.g., Flickr picks up tons
of metadata encoded in the photo files it gets from cameras. So,
Flickr knows for most photos whether the flash fired, what the aperture
was, what the date was, etc. At some point, cameras will also
automatically encode their latitude and longitude. That information can
be used to disambiguate clashing tags.

And Flickr does a remarkable job in many instances analyzing the
relationships among tags to determine whether this is a photo of the
Island of Capri or of the Ford Capri. That's how Flickr clusters
photos, and it works with surprising precision. So, it turns out that
as we get more and more tags, chaos doesn't necessarily result. In
fact, the statistic analyses get better.
  

    
I was trained as a librarian, worked as an bibliographer, and now try
to keep  an electronic archive, so this subject is near my hand, if not
my heart.

What librarians talk about when they talk about this sort of thing is
classification systems (what has here been called taxonomy) and subject
headings (what has here been called tagging). The two are almost
always used together. A reference work will be arranged according to
some classification system, and then indexed by subject heading. A
library will be organized by the Library of Congress or Dewey
classification system on the shelves, and by subject heading in the
catalog. 

The great problem with classification systems is that one can only
classify systems that change slowly, and human knowledge rapidly
proliferates, splits, combines, and breaks down the divisions between
its various parts. The most successful classification system to date
has been the Library of Congress system, because it is based on Thomas
Jefferson's system for organizing his home library, and Jefferson was
likely to become interested in anything at any time. He built his
classification system to be more expandable than orderly.

The problem with subject headings is that one wants them to be both
consistent and to be the names that people actually use for things. One
wants them consistent so that all information on a given subject is
gathered under one term. One wants them to be the terms people actually
use, because one is, after all, trying to make things easy for people.
Unfortunately, people continually change the names they use for
things. Also, no one really knows how to begin to find out what those
names are.

These are, you could say, the classic problems with sorting large
quantities of information so that you can find it again. Many of the
problems with sorting out the Internet are likely to be variations on
them.
  

    
bbraasch, great example. And the problem you point to will become more
pressing as more tags collide (same tag, different meaning) and
diverge (different tags, same meaning). Typed tags (i.e., meta-metadata
that says that the tag "bobo" refers to a place) make sense in some
applications, but at some point it turns into filling in a form, a task
people view as a chore to avoid. So, I think in general we should be
careful about going down that route.

There are other possibilities, fortunately. E.g., Flickr picks up tons
of metadata encoded in the photo files it gets from cameras. So,
Flickr knows for most photos whether the flash fired, what the aperture
was, what the date was, etc. At some point, cameras will also
automatically encode their latitude and longitude. That information can
be used to disambiguate clashing tags.

And Flickr does a remarkable job in many instances analyzing the
relationships among tags to determine whether this is a photo of the
Island of Capri or of the Ford Capri. That's how Flickr clusters
photos, and it works with surprising precision. So, it turns out that
as we get more and more tags, chaos doesn't necessarily result. In
fact, the statistic analyses get better.