Inkwell: Authors and Artists

    
Welcome, Ted!

12,000 words in the topic so far -- I don't envy you the catch-up reading
(not that you're obliged).
  

    
Paul,

I summarize the traffic on the cosmo-dev mailing list every week.  50
messages is short work ;-)
  

    
Hi, Ted -- glad you're here! For what its worth, I'm happy you're joining 
us, but the idea came from the fine hosts of this Well conference, not 
from me or Salon.

Christian, I think the "climb the ladder, then kick it away" approach you 
describe is probably closest to what Andy Hertzfeld was advocating in 
Chandler's early phases. It's worth pointing out that from a very early 
phase of the project OSAF actually did have "running code," if by running 
code you mean something that you could download and execute. The problem 
was more a disconnect between the high expectations people had for the 
project and the extremely limited functionality that running code offered 
for a very long time. You could download and "run" Chandler 0.3 and 0.4 
and so on, but you couldn't do a lot with it.

One lesson I took from that -- and it's a lesson that is now commonly 
embraced by the builders of Web applications, and also one that most 
people at OSAF would, I think, agree with -- is that you should try to 
roll out a new application or service by finishing at least one feature at 
a time in a fashion that users can actually use. (In the 37 Signals 
playbook this is defined as "build half an application, not a half-assed 
application.") That gives you a chance to begin to build a community of 
users and loop their feedback into your further development work. From 
what I can see Chandler is now on the verge of that sort of dynamic, with 
the forthcoming "preview" release. But I think it would plainly have 
benefited from arriving at that point sooner.
  

    
Hi Scott,

Thanks for the clarification on the invite.

We are definitely on the "build a slice of an application path".   The
particular slice for us is the calendar.

I arrived about halfway through the story in the book, so I'm not well
qualified to comment on what happened before I showed up.  But one
thing that has been difficult has been to balance the "platform" and
"application" aspects of Chandler.  Along with that, we were definitely
trying to build out all areas of the application at the same time.  
Since we decided to focus on the calendar, we've been able to get some
traction. 
  

    
Happily, I embrace "build half an application" in my real life, but in this
context that seems a bit of a paradox: does that really make the other
barriers to software development that we've been discussing go away? Or is
it just that they become small enough you can drown them in the tub?
  

    
I don't even think you get to drown them in the tub at that point, but 
perhaps you have a fighting chance of wrestling them to the ground.
  

    
BTW, in my morning reading I see that Ted has an excellent post up on his 
blog all about this basic question we're beginning to identify, about how 
much efficiency or software-development productivity the web-based 
software approach (or what Ted calls "rich internet applications (RIAs)") 
wins us.

The post is at
http://www.sauria.com/blog/2007/03/01/adobe-wants-to-be-the-microsoft-of-the-w
eb/

It's maybe a tad technical for non-developers, but the basic point is 
that, the more ambitious you get in terms of creating a rich environment 
for a Web app, the harder the development gets, unless you're willing to 
say, from the start, that your app only works on one browser. 

It's a fascinating dilemma. My browser of choice is Opera, and boy, life 
is getting harder out there, because a lot of Web-based apps are developed 
to support (a) Firefox, because it's good and oopen, and (b) IE, because 
you can't ignore those gazillions of people and (c) everyone else needs to 
fend for themselves.

To me, the larger point is this: the move to Web-based development will 
solve one set of problems that the previous generation of software wasn't 
very good at (coordinating data on multiple machines, sharing data among 
multiple users, backups and managing data so the user doesn't have to be a 
sysadmin, keeping the application code patched and up to date). But it 
will bring along a whole nother set of problems that we can already see 
the outline of -- these includee cross-browser compatibility and the 
development nightmare Ted explains, security and privacy issues. 
  

    
My first reaction to cross-platform issues is to simplify and avoid
the issue.  Lots of great, easy-to-use websites use very little
JavaScript; take Google search for example.  The UI is very important
but doing technically hard things is not the point; there may be some
easier way to please the user.  Of course, some applications have done
really well using AJAX, but you shouldn't let ambitions about building
a great UI keep you from winning by doing the simple thing.

I don't agree with a lot of Perl philosophy, but I still think Larry
Wall said it best:  Three great virtues of programming are laziness,
impatience, and hubris.

If you have the right kind of laziness then you will avoid hard
problems when you can.  If you're impatient you'll release early and
often and instinctively avoid taking large bets that won't pay off for
years.  Hubris means that despite using minimal effort, you still want
to impress people with something great.

The trap seems to be thinking that in order to do something ambitious,
you have to solve hard problems head-on, rather than dodging them. 
I'm a bit concerned that this discussion (and I suspect the book as
well, though unfortunately I haven't read it yet) is promoting the idea
that software is supposed to be hard and that train wrecks and
disasters are the inevitable price of doing something ambitious, but
that you should be ambitious anyway.  That doesn't seem like a good
mindset for the people on a software project to have because it doesn't
teach good avoidance techniques.
  

    
Do we really have to worry? Part of the lesson of Scott's book for me
was that software developers almost never adopt that mindset about
their own projects, no matter what prior or current evidence seems to
point to.  "All programmers are optimists" and all that.
  

    
>If you have the right kind of laziness then you will avoid hard
problems when you can.  If you're impatient you'll release early and
often and instinctively avoid taking large bets that won't pay off for
years.  Hubris means that despite using minimal effort, you still want
to impress people with something great.<

I like that the above would tend to militate against linear and
hierarchical thinking.
  

    
slipping back to mz:

 >Actually, software is easy.

Coding is easy.  And fun.  And challenging in a problem-solving way.  A lot of
us are good at it.

The hard part of software development for many of us lies in the human factors
completely unrelated to actual coding.  Playing nice with others does not
come easily, for some reason, to a lot of people in this profession.
  

    
(should mention I have not had the chance to get the book yet, but I look
forward to it!)
  

    
>The hard part of software development for many of us lies in the
human factors completely unrelated to actual coding.<

There is even some (scriptural) "evidence" that human factors have
caused some difficulty in the "intelligent design" of the universe. 

Those pesky people creatures are even getting in the way in Iraq.
(maybe a little playing nice?........Naw!)

It is a very interesting outlook when the producers and users of
systems are viewed as unrelated to the actual process. 

BTW, in MBA education, business case study solutions have concluded
with the comment; "The problem is solved, all that remains is
implementation". 
  

    
>It is a very interesting outlook when the producers and users of
>systems are viewed as unrelated to the actual process. 

Oh yes indeed.  Or "unrelated to each other in the actual process".

There have been various attempts in the software industry to address this, 
with Agile/XP/etc being the latest (and maybe the most adopted, but that
is hard to say).

But you are completely right with that point.
  

    
Ted's blog post summed up one set of my answers to Earl's question.
Our challenge in multi-platform support has moved from hardware to
operating systems to windowing platforms to browsers, and now back to
hardware as we try to get Web programs to run on pocket telephones.  We
converge on a platform definition but the definition keeps changing.

I don't think that's the whole story about why software is hard,
though.  Looking at the design side rather than at the implementation
side, there's constant expectations creep.  The goal is to produce a
product that is great when it's delivered, not when the the first
requirements are conceived.  Our product will be compared to other
products that are released while ours is built.

I'm not convinced that it's more difficult now than used to be to
estimate the work required to implement to a set of requirements.  The
state of the industry, though, is that we can't count on requirements
to stay put.  Agile methodology provides a way to allow the development
process to chase the requirements.  I don't think there's an adequate
equivalent on the business side of the software industry.
  

    
 Brian wrote:
 >The trap seems to be thinking that in
 >order to do something ambitious, you
 >have to solve hard problems head-on,
 >rather than dodging them. 
 >I'm a bit concerned that this discussion 
 >(and I suspect the book as well, though 
 >unfortunately I haven't read it yet) is
 >promoting the idea that software is supposed
 >to be hard and that train wrecks and
 >disasters are the inevitable price of doing
 >something ambitious, but that you should be
 >ambitious anyway.  That doesn't seem like a
 >good mindset for the people on a software
 >project to have because it doesn't
 >teach good avoidance techniques.
 
 The book certainly doesn't aim to promote any particular idea; it's one 
story, surrounded by a lot of background research, that you can draw 
different conclusions from. I'm with Brian until he says "...but that you 
should be ambitious anyway." That's certainly not my conclusion! For me, 
it's more like "train wrecks and disasters are inevitable and no one's 
found a sure way to avoid them, but there are some things you can do to 
minimize their likelihood." Those things include working incrementally, 
limiting (or at least staggering) your ambition level, and not making 
unrealistic assumptions about schedules or fixed requirements. 
 
I think I understand what Brian means by "avoidance techniques" and 
"dodging" hard problems but I also think that terminology is guaranteed to 
raise the hackles of the people (project sponsors, business people, 
program managers, etc.) who need to be on board with the developers' plan. 
It sounds like you're saying, "let's punt." I think the same point can be 
expressed by saying, "Let's be realistic, let's acknowledge that there are 
swamps here, and let's agree we don't want to get sucked into them." 

<karish>, that's an interesting suggestion about finding the business 
equivalent of an agile methodology. The thing is, as I understand it, 
agile arose as a developer-side reaction to the churn of the business side 
(changing requirements, etc.). Usually what I hear from developers is a 
demand that the business people they work with get more disciplined and 
rigorous about what they want -- in essence, that they move a little 
*less* fast!
  

    
Coming into the home stretch here I'm going to queue up a few more
questions. Scott you can tackle some or all of them at your leisure.

One funny thing: I've been re-reading D in C and set it down face up
in one of the coffee bars here on the Yahoo campus and a developer
started reading the cover and turned to me and said "I have nightmares
in code." (He was smiling as he said it.)

ok. first of all: early on you explain a bit about how the term geek
was embraced by computer nerds and it got me thinking again about the
different connotations between those two words. To me, geek has become
a term of pride (as you discuss), whereas nerd is still somewhat
negative sounding, although it is also embraced ironically by some.

When I was writing my book I spent a lot of time talking to Craig
Newmark (of craigslist) and he proudly describes himself as a nerd and
points to his pocket-protector days in high school as a formative
influence on his desire to make software inclusive (which I thought was
an interesting insight into the world of social software).

So, any thoughts on nerds vs. geeks and how they relate to software?

I'll put my next question/musing in a separate entry.
  

    
My next question has to do with patterns. That's a particular interest
of mine because my current job involves managing a design pattern
library. Several times you talk about Christopher Alexander's "A
Pattern Language" and there's a whole interesting digression about the
Portland Pattern Repository (Ward's Wiki).

I find it interesting how the pattern metaphor has migrated from
architecture to software design to interaction design (OK that last
jump is less of a stretch).

Any thoughts on how pattern-awareness has impacted software
development? Ward's wiki devotes a lot of energy to noting
antipatterns, kind of warning people away from "worst practices" so to
speak. Is this accumulated lore of any use or does it fall victim to
the whole "this is a special case" thinking you document at
discouraging length in your book?
  

    
ok, that's plenty for now...
  

    
I have one to throw in there too, and maybe Ted can even chime in on
this one -- I gather that the Chandler team is a bit more distributed
geographically these days than they were during the time you were
writing your book, though you wrote about some of the steps in that
direction. Do you think there's any particular significance to this
change? Is it a good sign for Chandler?
  

    
 Paul first: Yes, the OSAF team's more far-flung these days; I know that 
Brian Kirsch moved to Hawaii, Ted is in Seattle, Mike "bear" Taylor is 
back east somewhere, Phillip Eby is in Florida... there are probably 
others too. I don't know how specifically significant this is, except that 
it forced OSAF to move more of its process out from meetings rooms onto 
mailing lists and such, and that's gotta help them as they continue to try 
to get more buy-in from "bazaar"-style open source developers. 
 
 Christian: Geeks vs. nerds -- I did a little research on this and the 
best I could come up with was that "geek" has tended to be more of a west 
coast term, "nerd" an east coast one. Nerd I think has a pedigree that is 
pretty much from school (a la "Revenge of the Nerds") where it originally 
had nothing to do with technology per se and meant any social misfit with 
an overdeveloped brain. "Geek" is even older (as I mention in the book it 
even appears in Shakespeare!), and of course it long meant the mentally 
impaired person who bit the head off the chicken at the sideshow. Both 
words have to some extent been co-opted by the tech subculture as terms of 
pride in self-labeling, and Neal Stephenson wrote a wonderful op-ed piece 
in the NY Times a year or two ago that defined "geek out" as any sort of 
over-the-top passion for the arcane details of any endeavor (so that you 
can "geek out" on train trivia or knitting or whatever).
 
To my ears as well, right now, "geek" feels more fully redeemed than 
"nerd," which still sounds unappealing. But these judgments are, I think, 
highly malleable by time and usage. The first time I heard the word "blog" 
I went "bleaugh!" It sounded grotesque. Now it's just common, 
don't-bat-an-eyelash parlance, and that happened in <5 years. 

As far as the patterns movement goes, I think what's important there is 
that in each case -- beginning with Alexander in architecture and moving 
to the "Gang of Four" in object-oriented programming and now, as you say, 
interaction designers as well -- you have fields that are trying to 
figure out how to pass along learned experience in the absence of 
fundamental laws or scientific principles. In other words, the use of 
patterns is a sign that you're dealing with a craft; the pattern is a 
distilled bit of craft experience, a way of streamlining the communication 
of any piece of knowledge of the form "In these sorts of situations in the 
past, this has helped and that has not."

In that sense, I think that the pattern movement is a huge success, in 
helping solve one of the software field's biggest problems, which is how 
to transfer know-how. But it also represents an implicit acknowledgment 
that there is no rigorous, universal set of software engineering 
principles at the core of the field -- that it remains very much a craft. 
  

    
Paul,

Yes the team has become much more distributed.  In addition to the
people Scott listed,  Reid Ellis lives in Canada, Bryan Stearns has
moved to Oregon, Morgen Sagen lives far enough away from San Francisco
that he is also remote.  On the Cosmo team, 2 of the 5 developers live
outside the Bay Area.

That has definitely made an impact on the open source aspects of the
projects.  Late this summer, an engineer working at another company
produced an entirely new and much higher performance storage engine for
Cosmo.  This is probably the most significant contribution to come
from people not employed by OSAF.

Also, one effect of forcing process out of meetings and into mailing
lists is that there is a record of the rationale for decisions, and
people who were not in the meetings are still able to contribute when
relevant.

I personally regard this trend as a good one.
  

    
That was my intuition too. Thanks for the confirmation.
  

    
Since we're getting close to the end, Scott, I wanted to squeeze in
one more question. My perception, and feel free to correct me, is that
your book is fairly unique in the world of software writing in being
aimed not specifically at the practitioner, but at the interested
layperson.

This book feels very much a part of the legacy of the Whole Earth
Catalog (which we're soaking in right now of course), operating from
the assumption that the world is a complex but ultimately
comprehensible place, and that interested non-specialists have as much
place learning how a particular corner of it works -- and how the work
in that corner might improve the world -- as anyone else.

What I'm trying to get around to here is asking you how you view your
place in the world of software. It's clear that you have a deep
interest in it; that was really driven home by your story about
collecting tiny snippets of different programming languages. You are
not just an anthropologist of an alien culture. And yet you remain a
writer of sentences and paragraphs, not functions and procedures.

Do you have any idea where this interest will take you next? And do
you have any models or precedents you keep in mind to help you
explain, to yourself or others, what it is you are doing and what your
relationship is to the world you are chronicling?
  

    
Paul, I'd be honored and proud to be viewed in some way as part of the 
Whole Earth legacy, and certainly, from my years of presence on the Well 
and my general marination in the writing and ideas of people like Stewart 
Brand, Howard Rheingold and Kevin Kelly, it's fair to see some influence 
there. But I can't say that it was conscious in any way. I think I was 
more directly aware of some antecedents that we've discussed and are sort 
of obvious (Tracy Kidder, Steven Levy, etc.) and maybe some that are less 
obvious. One point of comparison for me -- or, to be more precise, maybe, 
a target of aspiration! -- is John McPhee's writing on geology, where a 
knowledgeable outsider takes you inside an extremely complex field and the 
issues the people in it confront in their work.
 
 Making the book comprehensible and absorbing for the non-specialist was 
absolutely my aim. It was, you could say, one central requirement of the 
project -- and I feel comfortable that I at least partially fulfilled it. 
But there's no question that it was tough to aim for that while also 
knowing that the book's "early adopters" would, inevitably, be programmers 
who would read it from a very different vantage. 
 
 I spent the first 10 years of my writing career as a theater critic, and 
that sort of writing involves a primary responsibility to the general 
reader. Most readers of your reviews are never going to see the play 
you're writing about, but maybe, if you write a lively and engaging enough 
piece and you find ways to connect what the event was all about with 
something else in the readers' world, you can make the 10 minutes they 
might spend reading your review valuable to them. It was also always 
important to me -- but in a secondary way -- that I write knowledgeably 
enough to earn and retain the respect of the people in the field I was 
writing about, the directors and actors and playwrights. So this sort of 
insider/outsider balancing act is one that I've long been familiar with. 
And my status in the theater world was similar to my relationship to 
programming -- not something I've been actively involved in since teen 
years (the last time I appeared on stage!), but have worked on the edge (I 
spent a couple years reading new plays for the American Repertory Theater 
in the early '80s) and have some knowledge of the field from the inside.
 
Where it's all going to take me next I have no real idea. I've been back 
working at Salon since early last year, trying to add some new interactive 
features to the Web site. I thought it would be a way I could take some of 
what I'd learned in writing DREAMING IN CODE and bring it home to help the 
company.  But it's also true that everything I've attempted to do since 
returning to Salon has taken a really long time! While that's been 
frustrating, it has, at least, confirmed the book's perspective  -- that 
making software really is hard, and that it will usually find unexpected 
ways to trip you up.