SLIDE #4: photo of Lawrence Scrivener in simulator on cover of PSA Skylines May 1981
Phil Mercurio: Our next speaker is Alan Barnum-Scrivener; he's a Senior Technical Consultant with Kubota Pacific Computer. He spends his day job empowering scientists doing scientific investigation. I've also found Alan to be a deep thinker in whole systems theory, and a good friend. Alan Barnum-Scrivener.
ABS: Ok, I've come here today to talk about something that, in contrast to many of the things we've been hearing about in the last few days, is an idea that is not currently being implemented anywhere, and I'm not working on it; and it's basically just a vision for the future, and this seemed like an approximately appropriate forum in which to inject this vision.
| As the cost of medical imaging technology continues to drop even as techniques improve and become less invasive, we are clearly headed for an era of universal application of medical imaging to diagnosis and treatment monitoring. Though this technology is being developed for use by medical professionals, a spin-off will be the availability of systems that lay persons can use to image their own bodies. This talk describes such a self-imaging system and its purposes and possible evolution and uses. The benefits of such a system include patient motivation, early problem detection, and the leveraging of medical professional resources, as well as whatever acceleration of healing can occur due to accurately-applied "creative visualization," whether due to actual undiscovered mechanisms or placebo effects. |
First of all my abstract: "As the cost of medical imaging technology continues to drop even as techniques improve and become less invasive, we are clearly headed for an era of universal application of medical imaging to diagnosis and treatment monitoring. Though this technology is being developed for use by medical professionals, a spin-off will be the availability of systems that lay persons can use to image their own bodies. This talk describes such a self-imaging system and its purposes and possible evolution and uses. The benefits of such a system include patient motivation, early problem detection, and the leveraging of medical professional resources, as well as whatever acceleration of healing can occur due to accurately-applied 'creative visualization,' whether due to actual undiscovered mechanisms or placebo effects."
So that's it in a nutshell. That's really the whole talk, the rest is just fluff; but since they gave me 15 minutes I'll go ahead and use it.
A little more detail on this:
The data to input to this device would have to come from non-invasive measurement because you want to be making the measurement frequently. The visualization methods have to be quite advanced, including artificial intelligence applied to the segmentation problem, because there isn't going to be the man-hours to do this kind of thing manually. It has to have a completely intuitive user interface since the average person is going to be using it. It would be nice if it could do some preliminary diagnosis with expert systems techniques. Also it's vital that this all occur in the context of the patient being in a relationship with their physician, so they don't go off half-cocked just playing with their own body, possibly even getting into some bad hypochondriac loops.
The mission of such a system is to allow the patient to gain more knowledge of their own body; to provide them the incentive to do such lifestyle changes that are going to result in better health as well as following whatever treatment they've been prescribed, to increase the average person's intuition about medical systems, and especially to encourage prevention and early treatment of problems.
What will this look like? I had a sketch I was going to put on a slide, but really in its first incarnation it looks like a workstation or a personal computer. As it evolves, it could end up looking like a virtual reality system with eyephones, earphones, gloves, the whole wazoo. It could pass through a phase which it looks somewhat like a simulation vehicle, with people actually having full motion base feedback.
As these kind of display capabilities increase, one would also hope that on the other side the simulation ability of the system would increase. I noticed in listening to the questions that Jaron [Lanier] got this morning that a lot of people are a little unclear about the difference between simulation and graphics display. We show you (I'm from the graphic side of the house), we show you a picture of something and you often leap to the conclusion that there's a model behind that picture that's going to act realistically; well, that's almost never the case, and that's the hard part of the problem. I think Dr. Garfinkel at UCLA is doing good things to push the state of the art in this direction [5]. If we could truly simulate the human body, then ultimately we could automate an awful lot of medical care; there's a concept from science fiction called the "auto-doc," that maybe one day, 500 years from now, it's this thing you get in, and it makes you better. So that's where I'm ultimately headed with this. You should notice that if I'm interested in putting doctors out of business I'm going to do it beyond all our lifetimes, ok?
Where did this idea come from? Where did I get the input that led me to it? Well, first of all, I grew up around simulator technology.
This is my father here who managed the simulation facility at the now defunct Pacific Southwest Airlines.
Here's what the simulator itself looks like. This is a very mature industry now; RediFusion built this motion base, the same people who built the Star Tours ride at Disneyland. They know how to do it; it's a solved problem. And I grew up believing that flight simulation was something that was exciting and possible; I'd always had evidence to that effect.
In 1982 when I was already a computer professional but not yet into graphics, I attended a World Game sponsored by Bucky Fuller, and his whole message was that he wanted to do an experiment, it was a 50 year experiment on how much of a difference an individual could make.
I found this very inspirational, and it was at this conference that I came up with the idea for the Somascope.
It was just a year later that I started working in the graphics industry; that's how I met Phil (Mercurio). I worked at a company called GTI which is no longer in the graphics business. I did customer service, I did customer education, and I went to see all the customers and saw their facilities.
It was all aerospace companies, almost all of it was military work then. I got to see the giant eggshell domes that they project images on with the fighter plane cockpit in the middle. I got to see the giant walls they built with models of airports and various places with these huge gantries that they were flying cameras over, and the people in the cockpit were seeing what the camera saw. That was in the very beginning of the process of converting from mechanical systems to computer generated imagery. I felt even then that there was perhaps some reason I was being exposed to all this technology.
This was what typical out-the-window computer graphics in an airtcraft simulator looks like.
I did a stint for a while at Rockwell International doing space station animation; that was a lot of fun until we lost the space station contract.
Then I went to work for a hardware vendor. The company was originally called Stellar; it merged with its biggest competitor called Ardent to become Stardent, which just this last fall failed and a portion of that technology was purchased by one of the investing partners called Kubota. It's been a really gnarly situation.
In the process, the continuity for me is that I've been in contact now for the last four years with all the customers for these products, what we call graphics super-computers. The typical pantheon of aerospace companies has been there so that everybody from the last slide have been customers of my current employer; in addition, you see more aerospace companies, the National Labs, people working on bigger and better bombs, and things like that.
This is the kind of visualization they were typically doing: airflow over a space shuttle. It's not a very good slide, but in computational fluid dynamics they model how turbulence occurs around wings and things.
However, especially in the last two years we started getting customers in the medical field: UCLA Radiology, Kineseology, Anatomy; UC San Diego School of Medicine; Loma Linda (I actually already knew all the guys at Loma Linda before they bought a computer; maybe that's one reason they did); University of Arizona (Bill Holmes is here today from that group); Health Sciences Corporation (I think the name has changed on that), Dr. Harvey Eisenberg spoke yesterday on medical imaging. I recently went to St. Jude's Hospital where they're doing radiation oncology. (I forgot to put Scripps Clinic on this slide.)
Here's the kind of things they're doing at Scripps Clinic; they're visualizing stuff down at the molecular level. This is a binding pocket of an antibody containing a synthetic peptide which is composed of nine of the residues from the influenza virus hemagglutinin protein. It was visualized by J. M. Rini and Mike Pique and their team.
Dr. George Eisenman at UCLA is visualizing molecular channels in this way. I think it's fascinating that they can get output which looks like a drawing but is actually done automatically by the computer.
![SLIDE #14: front cover from Pixel: The Magazine of Scientific Visualization July/August 1991 [2]](slide14.jpg)
At the Wistar Institute they're visualizing the adenovirus which failed to yield under crystallography but then by doing slices of electron micrographs and reconstructing it they were able to figure out its shape.
![SLIDE #15: Picker International -- SPECT brain study from Computers In Health Care February 1991 [4]](slide15.jpg)
At Harvard University Medical School they're doing radiation oncology treatment planning using our computers where they are visualizing the results of MRI, showing the tumor and then simulating the radiation therapy and so on. (This slide is actually from Picker International, which is doing similar imaging.)
My exposure to all of these customers has only increased my belief that the Somascope is something that can be done and should be done.
So, then the question is how do I expect this to happen if I'm not going to build it, and I'm not working on it?
![SLIDE #17: Figure 34b - Bird's-eye view of UN Geoscope, from Critical Path [3]](slide17.jpg)
Well, I'm following the lead of Bucky Fuller. In 1965 he invented something called the Geoscope, which he originally envisioned as an enormous globe that he would hang from guy wires next to the U.N. building in New York. It would be covered with photos taken from aircraft of the earth. It would have all these lights that would shine on it that would show all these educational things like where there's energy usage, where there're wars, where there's population and so on. This idea evolved over time. (At the 1981 World Game event I attended he had the world's largest world map, which filled a basketball field, and we used it to visualize the effects of all of the world's 50,000 nuclear weapons being exploded at once in the most populous areas.) Eventually Bucky decided that it shouldn't be a giant globe, it should be little computer workstations with geographic information in them. And in fact, both those visions are coming true.
SLIDE #18: front cover from Supercomputing Review Jan. '91 [1]
This is the cover of Supercomputing Review magazine. This data set you're seeing here was developed by Tom Van Sant who's an artist in Santa Monica doing something called the Geosphere Project. He's actually built a sphere on which he's projecting video data of just what I was describing. So just by creating the vision and putting it out in the world Bucky started the ball rolling in such a way that this project eventually came to fruition.
SLIDE #19: International AVS Center business card
Van Sant is also working with the International AVS Center at the North Carolina Supercomputer Center to create a working Earth Situation Room where people can go see how our planet's doing right now. I'll say some more stuff about AVS a little later on.
Meanwhile a whole market has grown up in what are called Geographical Information Systems, or GIS, which are implementations of the more recent workstation concept for the Geoscope.
So, Why? Why build such a device as the Somascope? Well, for one thing, I don't think the average person on the street can really count on doctors to solve all their health problems. There just aren't enough doctors, there's too many people; plus, the person who cares the most is the patient.
Let me read you a quote from Ellen Goodman, in the LA Times, 1987. In a column on health care she said:
"There's a tendency to study single diseases and small body parts instead of lives. The group concerned with the maintenance of lungs doesn't always 'do' ankles and the cancer-prevention team isn't 'into' cardiovascular diseases. As the last generalists, we, the owners of whole bodies, are supposed to think of ourselves as nothing more than the sum of parts and potential diseases to be taken care of with separate regimens."
[-- Ellen Goodman LA Times 5/26/87]
So the people who need to change their lifestyles are the patients. I'll just assert baldly here that the biggest killer in America today is the American lifestyle. Smoking, food, stress, drinking, drug abuse... all the things that are the consequences of our success as a nation are leading to our crisis in health care.
Also, it's at a wonderful time right now when the Cold War has just ended and all these aerospace workers are trying to figure out what to do next. I see this as very analagous to what happened in 1972 when the space program was being downsized; when Nixon had signed an ABM treaty and the aerospace companies were hurting then and they laid off a bunch of people and a bunch of these engineers went out and created the PC revolution because they didn't have jobs and they needed to do something. This is Adam Osborne's theory in Running Wild [9]. I see the same kind of thing is happening today. I'm not sure the companies are going to survive but the engineers are going to find a way, and a lot of that way I see is probably going to be in medical imaging.
I need to say that in addition to just motivating the patient there is a psychosomatic effect. I'm sure you all realize this. The placebo effect is kind of a paradox really, because you can tell patients, "This is just a placebo, but other people have got results from it," and it'll work just as well as lying to them. So it's something to do with belief. Witch doctors have known this for the longest time, and hypnotists have known this -- you can hypnotize people and tell them that you've given them honey and their blood sugar goes up. In people with multiple personalities, their eyes will change color, their reaction to medications will change depending on the personality. So clearly there is something in this psychosomatic effect. When you think about the word psychosomatic it just means "mind/body;" and we treat it as if its this peripheral and trivial effect, but from a medical point of view isn't "mind/body" all there is?
So the next question is when. When is all this going to happen? Well, let me try to give you a real quick, in the 2 minutes that remain, history of western civilization here.
SLIDE #22: woodcut of Vesalius reprinted in "The Vesalius Project" [12]
400 years ago, Visalius was the first person to do public dissection, which was very controversial, and up until that time the inside of the body was taboo, we didn't know what was in there.
SLIDE #23: phot of Oppenheimer and Von Neumann from A Computer Perspective [7]
40 years ago, computers were something that were multimillion dollar things occuring in engineering labs and people thought they would only build a few dozen and they'd be done with it all.
SLIDE #24: front cover from KPC Titan 3000VS brochure
4 years ago my company and some others were producing graphic supercomputers the size of a refrigerator that cost about a hundred thousand dollars.
SLIDE #25: photo of Titan 750 from KPC corporate brochure
We've dropped that down to the size of something that sits next to your desk and costs 60 thousand dollars, and it's clear where the trend is going.
SLIDE #26: X Windows architecture overview from O'Reilly Volume 3 [10]
We've also seen the proliferation of standards, the X-Windows system which allows multiple computers to do graphics.
SLIDE #27: sample Motif window from Motif Style Guide [8]
We've seen the Motif window managing system which does for the user interface what X does for the machine interface.
SLIDE #28: front cover from AVS 2.0 manual set
We're settling some issues now in our industry which have bugged us for a long time which is getting all the systems to work together completely without problems. I've seen the rise of visualization software, something called AVS -- the Application Visualization System. Several of the speakers have actually shown you slides using AVS but they haven't mentioned it. It's an extremely intuitive user-friendly software interface for visualization, you can see a demo of it next door.
SLIDE #29: first page of "Shaped-based Interpolation" by Herman et. al., from IEEE Computer Graphics and Applications May 1992 [6]
We're at the point now where some really important problems are being solved. The interpolation problem: this is just from this month's IEEE Computer Graphics and Applications, by a group at the University of Pennsylvannia. Sai Raya, who currently works with Dr. Eisenberg and came from the University of Pennsylvannia, has also made great strides in this area [11]. There are also advances being made in the segmentation problem: just last week, I was at UCLA and they were doing segmentation using multi-spectral data, borrowing techniques from the people who are doing planetary imaging, Mars flyovers. They take all these different spectral bands and they use that to correlate to figure out which feature is which. Turns out that they can do that with brain data too, so that they can automate the process of figuring out which parts of the image are actually which organs.
While this is great research is going on, we have a state of affairs today where the patient is "out of the loop." I got a sonogram recently, and the technician looked at the moving image. I got to watch my own heart beating on the monitor. When he had a good view, he "froze" the picture. The doctor looked at the still picture and made a diagnosis. All I ended up with when I left was a stack of insurance forms. I couldn't help thinking how much better it would be if the doctor had looked at the moving image with me, and left me with a copy.
SLIDE #30: preliminary artwork for The Body amusement park ride, for proposed WonderWorld theme park in Corby, Northamptonshire from "Animated Architecture" [14]
Plus we're seeing a bigger interest in the entertainment industry and the insurance industry. I wish I had a picture of the Body Wars ride. There's actually a simulator ride at Disney World, sponsored by Metropolitan Life, where you can go through the human body. It uses the same RediFusion motion base as the Star Tours ride. It's almost like being in the movie Fantastic Voyage. I was very impressed by it. I actually started crying on the ride; the woman next to me began salivating, and the man on the other side began prespiring; something to do with body fluids; but I think it had to do with the fact that it was a return to the womb experience. When we were in the blood stream, we could feel ourselves being pulled through the blood every time the heart beat.
The insurance companies are clearly interested because treatment is much more expensive than prevention. There are also patient-driven pressures in this direction. In Los Angeles there is a chiropractor who produces what he calls X-ray movies of joints in motion. He advertises this technology on the radio. The patient is X-rayed in a series of positions, and the resulting sequence of frames is animated. It shows very clearly what a problem is with a joint. And the patient gets to take it home on a videotape.
So that's the kind of momentum that's leading up to the Somascope, and my prediction is that it's going to happen, in the first incarnation -- the workstation -- within 5 years, and in the interactive motion-base vehicle form within 15 years.
Lastly, who. I would have liked to put together a bibliography for this; I didn't quite finish it. I will tell you who I am.
SLIDE #32: my business card
Kubota Pacific, my current employer, was kind enough to send me to this conference so I'm plugging them for the moment. Also, if you'll come next door and sign up on my sign-up sheet I'll send you a complete copy of this paper including the finished bibliography.
So that's it. Thank you very much. [APPLAUSE]
Phil Mercurio: Thanks a lot, Alan.
References [1] Barnes-Svarney, Patricia "On a (Computer-Generated) Clear Day You Can See Forever" Supercomputing Review January 1991 [2] Burnett, R. M., Stewart, P. L., Fuller, S. D., and Mathias, Craig "Image Reconstruction of Adenovirus" Pixel: The Magazine of Scientific Visualization July/August 1991 [3] Fuller, R. Buckminster Critical Path especially Chapter Five: "The Geoscope" St. Martin's Press 1981 [4] Frisiello, Roxanne St. Claire "Desktop Supercomputers Advance Medical Imaging" Computers in Health Care February 1991 [5] Garfinkle, Alan "A Mathematics for Physiology" The American Physiological Society 1983 [preprint] [6] Herman, G., Zheng, J., and Bucholtz, C "Shape-based Interpolation" IEEE Computer Graphics & Applications May 1992 [7] Office of Charles and Ray Eames A Computer Perspective: Background to the Computer Age Harvard University Press 1973 [8] Open Software Foundation. OSF/Motif Style Guide Release 1.1 Prentice Hall 1991 [9] Osborne, Adam Running Wild: The Next Industrial Revolution Osborne/McGraw Hill 1979 [10] Quercia, Valerie and O'Reilly, Tim X Window System User's Guide for Version 11 (Volume Three of The Definitive Guide to the X Window System) O'Reilly and Associates 1989 [11] Raya, Sai, Udupa, J. and Barrett, W. "A PC-Based 3-D Imaging System: Algorithms, Software and Hardware Considerations" Computerized Medical Imaging and Graphics Vol. 14, No. 5, Sept. 14, 1990 [12] Roper, Stephen D. "The Vesalius Project" Academic Computing October 1989 [13] Scripps Research Institute Scripps Research Institute Scientific Report 1990-91 (Volume 17) [14] Walker, Derek (guest editor) "Animated Architecture" (special issue of Architectural Design) Academy Editions/St. Matin's Press 1982 For information on upcoming "Medicine Meets Virtual Reality" conferences write: Medicine in VR / P. O. Box 23220 / San Diego, CA 92193
Last update: 28-Feb-1998 by ABS