Inkwell: Authors and Artists

    
Jon, that's an interesting question! I've been reading a lot of books
lately by academics who are writing to popular or business audiences,
and it's a tricky balancing act. Most of the time the authors end up
hiding the complexity in the footnotes and references.

With Topsight, I do something similar. The chapters are written to a
general audience, but I also point people to the academic backing
behind each chapter. For instance, when I discuss information maps, I
mention that these are based on the genre ecology models I've developed
in academic papers, and I point to those resources. And in the first
appendix, I tell the readers: If you want to learn more, here are some
further resources.

This might be an odd metaphors, but think of Topsight as an interlaced
GIF. This particular GIF is pretty big and will take a while to
download, but since it's interlaced, you can get a low-res version of
it quickly and at least get a good idea of what you're looking at. And
if you're patient, you can go into more passes -- read the resources in
the back of the book -- and get a much sharper image.
  

    
That's a good metaphor - I totally get it. 

If I was going to use the Topsight method to look at my company, how
would I start?
  

    
The first thing is to get a game plan. As I say early in the book, you
need to pick your focus and figure out what data you can gather.

And by "data," I don't mean big scary things such as running
experiments. I mean systematically shadowing people, interviewing them,
and gathering information that they touch and use. In Ch.2, I talk
about how to gather these at different levels so that you have adequate
coverage when it comes time to analyze them.

So suppose you want to know how people are processing paperwork in
your organization. You'd articulate it as a question so that you have
focus. Then you'd ask: how do I investigate this question so that I get
answers from several of the people involved? You might come up with a
big list of ways: observations, interviews, walkthroughs, photos,
videorecording, system monitoring, reading through emails, etc. etc.
Not all of these ways will be practical or allowable (most people don't
want you snooping through their email), but you should be able to find
a few ways to get coverage.

The research matrix in Ch.2 helps you to make sure that coverage is
adequate. One problem I consistently see is that researchers overrely
on interviews or surveys, thinking that these methods will tell them
what people do. No. They tell you what people *think* they do.
Sometimes they tell you what people think ought to happen. But they
drop out all sorts of things that actually happen. So the research
matrix helps you to make sure there's a good tension in your data, a
good set of perspectives that you can relate together.

Of course, this research matrix is just a starting place. As you
approach people at your site, you might find that they can't or won't
give you certain data. So in the next few chapters, I discuss how to
get these people on board, how to figure out what they can't or won't
allow you to do, and how to modify your approach accordingly.

That's a big deal, in fact. People at your organization really need to
be on board with the process, or they won't participate and your data
will not be so great. So the first part of the book is as much about
persuasion as it is about research design: How do you make the research
into something that will benefit all stakeholders?
  

    
I want to piggyback on Jon's last question about using Topsight in
industry...

Let's say that I want to adopt Topsight for use in an undergraduate
course populated by students from a variety of majors—journalism,
creative writing, public relations, marketing, English studies,
history, political science, etc. (i.e., mostly non-engineering and
non-natural sciences majors). 

How can Topsight help these students generalize understandings of
professional communication?
  

    
I introduce Topsight to my own rhetoric & writing students in terms of
audience analysis. "You've been analyzing audiences through secondary
sources," I tell them, "but now you'll have the opportunity to
understand them through primary research. You can get a better idea of
how they think, what challenges they face, and how they've innovated to
solve those challenges."

What I usually don't tell them is that in doing this, they're learning
one of the basic skills of technical communication. TC involves taking
concepts and information from one domain and translating them
appropriately so that they can be applied in another domain. And
audience analysis - that is, figuring out how and why people think and
act the way they do - is a key step in that process.

Beyond that main skill, we work on other skills. For instance,
Topsight discusses the entire research process as a rhetorical process:
how do we connect people's needs and interests to the research so that
we can provide good recommendations? How do we express those
recommendations? It's not a book on writing recommendation reports, but
readers end up learning the components of a solid rec report anyway. 

The student population that you describe, Brian, already has an
abstract understanding of how to write to different audiences. But in
Topsight, they find a direct application that exercises a number of
technical communication skills. Or at least that's how I see it.
  

    
Thanks, Clay!

An unrelated question: your definition of "true ethnography" back near
the beginning of this thread was interesting to me. My sense is that
the three-pronged definition you offered was accurate a few years back,
but that the scope of what constitutes ethnographic practice is quite
different today.

I know that you read widely in qualitative methods and methodologies;
what do you think of the contemporary ethnographic work that's
happening in industry? I'm thinking here of design ethnography, rapid
ethnography, and UCD/UX ethnographies done by consultancies like
AnthroTech?

Certainly, none of this work obviates your point about bounded,
systematic qualitative case studies! Instead, from my perspective
anyway, it seems to me that I could pick up Topsight and use it to
conduct contemporary ethnographic work. Thoughts?
  

    
Ah, this *is* an issue, Brian. "Ethnography" has been applied pretty
broadly in the last 15-20 years, blurring the distinctions between this
methodology and others. People often apply the term "ethnographic" as
synonymous with fieldwork or qualitative research - I read an otherwise
good study today in which someone characterized her research as
ethnographic because she included a questionnaire! 

But applying "ethnography" as a broad term is problematic for a number
of reasons. Here's a great, very pointed discussion from an
anthropologist's perspective about how the term has been used -- she
doesn't name names, but she's clearly talking about contextual design:

Forsythe, D. E. (1999). It’s Just a Matter of Common Sense:
Ethnography as Invisible Work. Computer Supported Cooperative Work
(CSCW), 8(1), 127-145.

I chatted with an anthropologist friend about this question a few
weeks ago, and he emphatically agreed that the term "ethnography"
should be more tightly bounded. Specifically, he agrees with the
distinction offered by Creswell and others that ethnography studies
*culture*, while other methodologies take different study objects
(e.g., a case study focuses on activity in a bounded context, but not
necessarily on the culture in that context). 

That doesn't mean that we should automatically invalidate contextual
design, rapid ethnography, etc. Not at all! These are often
well-thought-out methodologies that actually use ethnographic *methods*
within bounded contexts -- often described as "ethnographic case
studies." But I'm not a fan of labeling these ethnographies, even
though they use similar methods. The different study object implies a
different configuration.

Here's one good example. In Topsight, I discuss making detailed
observations, taking detailed field notes, and conducting
semistructured interviews immediately afterwards. That works great for
studying specific work activity in bounded contexts: the detailed field
notes extract a lot of micro-level operations and the stimulated
recall interviews allow you to gather on-the-spot interpretations of
what people did. But it's not good for studying a culture. In fact, we
get very different advice for writing ethnographic fieldnotes: we're
told to soak in what's happening, then find some time to ourselves to
recap the observations in fieldnotes, perhaps at the end of the day:

http://spinuzzi.blogspot.com/2013/01/reading-writing-ethnographic-fieldnotes.h
tml

Similarly, ethnographic interviews tend to focus less on micro-level
events and more on generalities, more on the meso- and macro-level
occurrences. The sorts of details that could inform UX, UI, and design
tend to get lost, traded for the broader themes that can inform you
about culture.

Anyway, that's my two cents. I imagine you'll get different answers
from different methodologists!
  

    
Clay, we're at the end of our discussion, and I want to thank you for
taking time to discuss _Topsight_ in such depth. One final question:
having finished the book and had time to reflect on it, have you done
any rethinking? Are there any parts of the book that you would handle
differently, given 20-20 hindsight?
  

    
I'd like to thank Clay for writing Topsight and going the
self-publishing route so that the book reached its readers now and not
next year or later.

I first came across Clay's approach in Tracing Genres, his first book
http://www.amazon.com/Tracing-Genres-through-Organizations-Sociocultural/dp/02
62194910
while working on large intranet and collaboration suites. No matter
how hard I tried, user-centered methods didn't give me the input I was
looking for and eventually I assembled an ethnography-ish package for
myself, my team and my client. 

We criss-crossed the country interviewing, shadowing and observing our
users and documenting their tools, practices and habits. The most
important thing we did was to leave HQ and go to where the work was
actually done. When we came back we threw out our old plans and started
over, and while the end result was pretty good I knew there were
elements missing from our new "method".

Ironically, we were handling, sending, using and transforming those
missing elements throughout the working day. They weren't staring us in
the face, they were in our hands and all around us: documents.

The user-centered model of design looked at users, tools, activities
– lots of things – but not the information itself. My first instinct
when redesigning a huge bank of forms was to simplify them and have
them adhere to the corporate design manual, not investigate where they
came from, what they did and where they went – what Topsight calls the
Handoff chain.

Another important improvement that came from switching to a
Topsight-driven process was how many people we interviewed and
observed. I think Jakob Nielsen's paper "Why you only need to test with
5 users" 
http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
has caused a lot of confusion among UX practitioners, whom I suspect
have misunderstood it to mean that you barely need to interview users
to figure out what to build. First of all, that's not what Nielsen
says. Second, his paper is 13 years old and the world has moved
forward. Third, uncovering how people work understanding their tools
and improving information flow is entirely different than basic
usability testing.

Of course, experienced researchers and UX professionals are far more
rigorous in their research but getting a client to sign off on an
open-ended research project is challenging, to say the least.

That's why we bake Topsight into our process and use it in the right
context. We're lucky – most of our teams work at the clients' locations
for months or years, so getting access to the right people isn't *too*
hard. Our interviews are aimed at the micro- and meso-level details of
the work, not the work culture itself. We're looking for disruptions
and breakdowns – stuff *we* can fix – and if we uncover a problem that
has more to do with how things are organized, what kinds of people are
hired, the incentive structures and so on, we hand that over to our
clients.

Topsight also lends itself well to people-and-process-intensive
projects like intranets, collaboration tools and case management. 

Of course, there are a lot of settings where we don't use Topsight:
small, everyday website improvements obviously benefit more from
A/B-testing and rapid prototyping.

I hope *everyone* working on tools for sharing and collaborating and
in Enterprise 2.0/social business projects will read Topsight. We can't
make things work better if we don't understand how they work. 

Thanks for hosting this discussion, Jon!
  

    
Fredrik - Thanks for the kind words! Topsight has been cooking for a
long time, but it really crystalized when I met with Fredrik's team.
That event and the team's feedback really helped me as I thought
through how to make this approach more accessible.

Jon - Always, always. In fact one thing I discussed with students in a
recent class was the fact that organizations often go through a
long-term cycle. 

1. At the beginning, they're still trying to figure out their routines
and tools, and consequently the organization has to deal with a lot of
disruptions. 

2. Eventually these parts are hashed out, and the disruptions are
tamped down day-to-day. But since the organization's components and its
environment continue to change, long-term contradictions develop under
the surface, becoming more severe and eventually showing up in various
disruptions. Because of sunk costs, people are reluctant to make more
than surface changes.

3. Finally, these contradictions become more severe and the
disruptions can no longer be treated with Band-Aid solutions. At this
point, the organization must consider big changes -- changes in
mission, vision, basic tools, basic division of labor, etc.

I now wish that I had included a chapter on this long-term cycle,
since it helps people to contextualize the organizations they study;
helps them to better understand the sorts of solutions that
stakeholders will entertain and the resistance they might feel toward
some solutions; and prepares people to detect deep contradictions in
the second stage, where they can be more easily addressed. 

But that's fine, I can always put it in the second edition :)
  

    
Looking forward to that update!

Again, many thanks for taking time with us over the last couple of
weeks. Also thanks to other contributors to the conversation. 

The web page for Topsight is here:
http://clayspinuzzi.com/book/topsight/

You can buy (and review) the book at Amazon:
http://www.amazon.com/Topsight-Studying-Diagnosing-Information-Organizations/d
p/1481960067/