Toward an Organic View of Data Administration

[This article originally appeared in Data Base Newsletter (January/February, 1996), published by Database Research Group, Inc., of Boston, MA (617) 227-2583, Ronald G. Ross, Editor/Publisher. Reproduced electronically with permission.]

In a way, I guess it's all Ted Codd's fault. The quirky genius who laid his gift, twinkling and elegant like some kind of jewel, before us in 1970 probably never thought he was giving rise to a whole generation of back-seat drivers and obsessive naggers. Yet here we are, still gnawing at an MIS management that avoids us but cannot seem to find the gumption to just tell us to go away. After all, writers and consultants all seem to agree that Data Administration is the key to competitive advantage, that messing with it would expose any business to chaos and annihilation. They can't all be wrong--can they?

Data Administration is one of those professions, like advertising, that spends a considerable portion of its energy trying to prove to itself and to the world that it is doing something of value. When I first got into the field, the most popular topic at seminars and DAMA meetings was "How to Sell DA to Upper Management." Almost ten years later, I look in my latest local DAMA newsletter and read a notice from a long-time member asking for suggestions from other members. The question: "How do you prove DA contributes to the bottom line of the business? What kind of metrics show DA providing a positive return on investment?" We do not appear to have come very far in ten years. Yet during those same ten years, management has cheerfully embraced desktop computing, client-server architectures, just-in-time inventory control, and a host of other innovations more radical and transforming than DA. Why has this simple idea failed so completely to take hold?

This emphasis on selling when discussing DA and management is, when you think about it, pretty patronizing. After all, we have been delivering this message for ten years; do we really think management is so...well...dumb...that they cannot understand what was so clear to us back in 1984? Are we poor explainers? Or could it be that they understand perfectly and are just not buying? We have worked hard to sell this concept over the years, but eventually we have to wonder whether working harder is not the answer--whether, in some way, we may be working hard at the wrong task.

Back to Ted Codd for a moment. His 1970 paper which defined the relational model was a landmark, and is justifiably famous today. It is marvelously complete; a 1995 description of the relational model would add little to what is in Codd's paper. It is thoroughly grounded in mathematical principles; it is internally consistent, elegant, and rational. It defines a universe in which everything makes sense and everything is consistent. To those of us slugging it out in a messy world of MIS politics, hardware limitations, and users who don't know what they want, it held out a promise of an attainable world of order and clarity--an Aristotelian universe of rationality and of rules that are followed. That's an extremely seductive vision to people who like things to make sense; in my view, we have followed that vision into a morass.

The search for an orderly universe of good behavior is an ancient one, and its roots lie deep in us. In its darkest form it can lead to political repression and even fascism. But even in its benign forms it is doomed by the perverse nature of humans, who have feelings, agendas, prejudices, and desires, and who will not behave in convenient ways. So our search for order--for E-R diagrams and Zachman frameworks, for normalized data structures, for predictable and descriptive names--becomes, as we always knew it would, a problem in control.

The City Planning Metaphor

We have all compared system developers with architects designing buildings. If so, then data administrators are like city planners or the zoning commission; their concern is with shaping the overall environment, so that the city as a whole takes a shape that is sensible, efficient, and pleasant. It is instructive to look at city planning for hints as to how we might successfully approach our own task--or for warnings about how the process can misfire.

I work in Emeryville, a small city wedged between Berkeley and Oakland, California. Twenty years ago Emeryville was a nasty Mordor of heavy industry: paint factories, steel mills, and foundries. No one went there who didn't work there or wasn't on their way to someplace else. Today Emeryville is a booming place, full of high-tech offices, condos, a multi-screen movie theater and a couple of shopping centers. How did Emeryville manage to bring about this transformation? If the Emeryville city planners had hired a data administrator, he probably would have proceeded something like this (I'm assuming a male DA for grammatical simplicity):

First our DA goes to Sacramento and spends a few months leaning on the Governor for "support." Eventually the Governor, tired of being bothered, gives our DA an assurance of support and a charter and sends him on his way. The DA shows up back in Emeryville and sets up a "review committee" to approve (or disapprove) all land sales, building plans, and improvement proposals prior to execution--not to mention taking occasional cruises around town to make sure everything is proceeding according to standards. Building projects stall as the committee tries to work through a mass of detailed proposals. Developers, aghast at the hurdles being placed in front of them, start to go elsewhere. Eventually people find ways to circumvent his committee; or, if he is very persistent, there is a barons' revolt and the DA and his committee are deposed.

I am painting in broad strokes here, admittedly; but the point I am trying to make is that the traditional techniques of data administration, when applied to a more concrete environment, are quickly seen to be faulty, for two reasons:

Of course, the Emeryville city planners did not hire a DA. What they actually did was something very different: Instead of forcing the city into a predetermined mold, they set up rules that encouraged a certain kind of development through tax laws, infrastructure investment, etc.

Stewart Brand, in his recent book How Buildings Learn, has this to say about planning and control and the growth of cities:

American planners always take inspiration from Europe's great cities and such urban wonders as the Piazza San Marco in Venice, but they study the look, never the process.... [Dennis] Romano: "Those who now romanticize Venice collapse a thousand years of history. Venice is a monument to a dynamic process, not to great urban planning....The architectural harmony of the Piazza San Marco was an accident. It was built over centuries by people who were constantly worried about whether they had enough money." [p.78]
Of course, city planners have been known to get just as dictatorial as data administrators. Local inspection codes specify the ceiling heights of add-on rooms, the number of outlets per electrical circuit--sometimes even the color of individual houses. There are good reasons for wanting to do these things. But the price is paid in inflexibility and a flourishing carpenter's black market. Here is Stewart Brand again:
Zoning must have worked to some degree, since it has lasted so long. But it succeeded in freezing up cities so tight that new Edge Cities out on the periphery became inevitable, and something was lost with all the comfortable stasis ....People find ways around zoning ordinances--quietly setting up home businesses in their garage or basement, quietly moving into industrial lofts--but like barrio dwellers they can succeed only so far before authority discovers and curtails them. Quelling change, zoning quells life. [p. 79]
In his 1987 article, "Why Data Administration Fails," George Tillmann argues that DA should be considered as purely a support organization, not a policing one. This is how he tells the story:
The development team that once supported data administration came to see it as a large burden with little immediate benefit. As the deadline approached, data administration was put under tremendous pressure to finish its work and release valuable staff members who were needed elsewhere.

A post implementation review blamed data administration for delays, questioned the benefit of participating in the data administration program and created a resentful environment that the new data administration organization had to endure for years.

Clearly there is a balancing act to be performed here, whether planning cities or planning corporate data systems: Too much control at too low a level and you have stagnation, resentment, and bypass; too much freedom and the environment grows out of control and chaotic. Striking that balance demands that we ask ourselves exactly why we wish to exert control. The reasons tend to fall into four categories:

1. Infrastructure

These are the rules that provide interconnections and enable basic systems to function. In a city, they are the rules that say that power is delivered to a house at certain voltages, that water main connections must be of a certain size. In systems, they are architectural rules like standardizing on TCP/IP. Enforcing these rules is generally not a problem, for the simple reason that things will not work otherwise.

2. Esthetics

These are the rules that try to make the overall environment sensible, efficient, and pleasant. In a city, they are the laws governing location of residential and industrial zones, building height restrictions, setbacks, etc. In systems, the analogous rules would be those governing standard development languages and platforms, location of data servers, and other architectural-level issues.

3. Quality & Value

Similar to the previous, only here the goal is to maintain the quality and value of individual structures. This is the purview of the building codes in cities, and of coding and naming standards in systems. Because they operate at the detail level, forcing reviews of minor changes, these are the rules that are most often bypassed and evaded. In cities, these rules are in practice enforced only on large jobs, and the thousands of illegal mother-in-law units quietly added by property owners are benignly ignored.

4. Navigation & Documentation

Rather than exerting prior control over design, these rules seek to make sense out of the actuality so that one can find one's way around, even if the place is chaotic. In systems, this is the area of metadata: physical data models, architecture diagrams, and data dictionaries. In cities, street names have to be unique, and the same on every map; address numbers have to increase monotonically, and the odd-even rule has to work consistently; telephone directories have to be published; and so on.

Encouragement, Not Rule-Making: A New Paradigm?

One reason for the continuing inability of DA to make a real difference in system-building is that it appeals primarily to the urge in many of us to try to impose an ordered perfection on the world. But the world does not work very efficiently or productively that way. At bottom, we cannot force people to behave and we cannot blithely assume they will do what's best for the organization at the expense of their own short-term goals. The writers of the Constitution understood--and their descendants in cities and municipalities everywhere understand--that all government works best when it leverages people's individual self-interests instead of trying to legislate against them.

What do we mean by that in the case of Data Administration? We mean that DA must get out of the "design review committee" mentality and substitute something more value-added and flexible. It must recognize that systems tend to grow organically, and be a part of that process, rather than an instiller of order upon it. Here, then, are my four basic laws of Organic Data Administration:

1. Focus on the big stuff

The goals of DA have more to do with overall system architecture than the specifications of individual data items and minor program enhancements. Resist the temptation to get buried in endless details, and concentrate on the big picture:

2. Make process rules, not artifact rules

Anyone who has tried to devise a rigid, prescriptive naming standard knows that no such standard serves all purposes. It is hard to imagine a design approach so cockeyed that it could never be the right choice. The fact is, we just cannot reduce design to rigid rules without sapping creativity and sowing resentment.

But we can require a walkthrough of a major piece of design. We can install change control procedures where the design choices that have been made (or are about to be made) must see the light of day and endure comment from peers. We cannot legislate the result, but we can put into place processes that, over time, tend to move us toward whatever design goals the organizational culture promotes. (Note that those goals may or may not include such DA favorites as consistency across applications, high integration, and sharing of existing data.)

3. Consult for added value

Whatever disrepute the DAs in your organization may currently labor under, those folks do know how to model; use them. Make DAs available as facilitators, modeling consultants, design advisors. As long as it is offered as a service and not a requirement, you may be surprised at how much business you get.

4. Map the world

And when all is said and done, document everything, no matter how ugly. Publish physical data models, document hardware platforms and geographical locations of data, show the flow of data from system to system. Read books on how to present information effectively and vividly. Instead of trying to reroute the streets, draw yourself a really good map.

DA must be a partner and a contributor, not an enforcer. What rules there are must allow for flexibility of implementation; the system builder must have room to express his or her own creativity. They must be rules of process, not artifact--that is, they must prescribe that certain things happen, not that the result must look a certain way. Lastly, the DA must get used to the idea that systems, like cities, are by their nature fragmented and inelegant and subject to random forces of all kinds; but if you can provide a really good subway system your customers will thank you forever.


Stewart Brand, How Buildings Learn, Viking Press, 1994.
George Tillmann, "Why Data Administration Fails," Computerworld, September 7, 1987.
Edward Tufte, The Visual Display of Quantitative Information, Graphics Press, 1983.
Edward Tufte, Envisioning Information, Graphics Press, 1990.

c. Eric Rawlins 1995

Page created and maintained by
Page last updated: 3/21/96