=======================================================================
Cybernetics in the 3rd Millennium (C3M) -- Volume 2 Number 9, Sep. 2003
Alan B. Scrivener --- www.well.com/~abs --- mailto:abs@well.com
=======================================================================
Do Nothing, Oscillate, or Blow Up:
An Exploration of the Laplace Transform
The last two issues of C3M ranged pretty far afield from what most
people think of as cybernetics. I got an unusually high number of
positive comments back, but I also got my first unsubscribe request.
So this month I am returning to the mainstream, and plunging into
some heavy-duty mathematics as well. Here goes...
I am sometimes amazed to encounter people who say they are very
interested in cybernetics and/or systems theory but they don't like
math. It seems to me that without math these pursuits quickly devolve
into just word games, or at best the kind of paying attention to
externalities and relationships that brought us group therapy in
psychology and man-in-the-loop testing in engineering. Math is
vital to the study of systems because it helps us understand all of
the things systems are CAPABLE of doing, that is, all possible
behavioral modes of a given set of assumptions.
In the 1930s Lewis Fry Richardson was a pioneer of the application
of mathematical models to social science. His "Generalized Foreign
Policy" (1939) offered a mathematical model of arms races, and
generated a firestorm of criticism from the "experts" in foreign
policy and diplomacy. His next book was "Arms and insecurity: a
Mathematical Study of the Causes and Origins of War" (1949).
( www.amazon.com/exec/obidos/ASIN/0835703789/hip-20 )
In it he devoted quite a few pages to rebutting his critics.
They had claimed that you couldn't reduce the complexities of
international relations to mathematical equations, because it
required human judgment and intuition to analyze these types of
problems. Richardson's main argument was that when people propose
a hypothesis in this (or any) field they often state the "obvious"
conclusions from that hypothesis incorrectly; their conclusions
do not follow "logically" from their premises. Only the rigor
of math can verify such conclusions.
Likewise, more recently, misguided critics have attacked the
methodology of computer modeling as if working things out
"in your head" represented a superior and more reliable way
to test if conclusions follow from premises.
For example, in his essay "Understanding the counterintuitive
behavior of social systems" (1971) Jay W. Forrester described
how government funded low income housing projects would usually
aggravate the lack of affordable housing, by attracting more
people to an area than the projects provide. His models showed
that somehow creating more jobs instead would cause private investors
to overbuild new housing, increasing supply and reducing cost.
The essay is reprinted in "Collected Papers of J. W. Forrester" (1975).
( www.amazon.com/exec/obidos/ASIN/1563271923/hip-20 )
So, if math is so vital to cybernetics and systems theory, why
do many of the fans of these fields hate and avoid math? I think
it is because it is so badly taught to our children. Kids are
very aware of rules, and have high standards of integrity.
Our standard math curriculum lies to them, and this turns a
lot of them off. We could tell them, "We're going to make
up some rules this year and follow them and next year we're
going to change the rules." But no, we say, "You can't subtract
a larger number from a smaller one," and then the next year
we say, "Surprise! You can after all, and the answer is a new
kind of number, called a negative number." That scrapes a few
of them off. Then we pull the same trick with division, and
surprise them with fractions. Then we do it again with square
roots, telling them you "can't" take the square root of a negative
number, only to go back on our word and introduce "i" the "imaginary"
quantity. By this time we've lost almost all of them. Even
the people who TEACH math to grade schoolers are pretty nervous
about i in my experience. And yet "complex" numbers (formed
adding "real" and "imaginary" numbers) are among the most powerful
and elegant tools in mathematics. (This goes along with my theory
that the purpose of public education is to inoculate people against
knowledge so they don't catch it later in life.)
I was fortunate in that I had some very good teachers, including
my father who taught me at home, and the inoculation never "took"
with me. I hung in there through the lies and "got it" time and
time again. By 9th grade I had noticed a pattern: We were taught
to count. Then counting was generalized to addition (which was
"closed" over the counting numbers, or positive integers, i.e., add
any two counting numbers and you get another counting number).
Then we learned the inverse of addition, subtraction: which was
not "closed" over the counting numbers. They had to introduce
negative numbers to make a complete set of numbers.
Then addition was generalized to multiplication (which was
"closed" over the integers), and we learned the inverse of
multiplication: division, which was not "closed" over the
integers. They had to introduce fractions to make a complete
set of numbers (and we still couldn't divide by zero).
Then multiplication was generalized to powers (which was
"closed" over the rational numbers -- as long as the exponents
were whole), and we learned the inverse of powers: roots, which
were not "closed" over the rationals. They had to introduce
irrationals to make a complete set of numbers (and we still
couldn't take the square root of minus one).
At this point I came up with what I call Scrivener's Conjecture:
Every time we generalize an operator and then take its
inverse we will have to invent a new kind of number.
So I tried it out. I invented an operator I called "gorp"
(for no particular reason) which generalized powers. I defined
gorp(2, n) to be n^n. (I am using BASIC's notation for exponents
here since this plain text format doesn't allow much else.)
Then I defined gorp(3, n) to be n^(n^n) since putting the
parentheses the other way, (n^n)^n would reduce to n^(n*n) which
didn't seem as interesting. Of course gorp(4, n) would be
n^(n^(n^n)), and so on. The I defined the inverse, "prog" (gorp
spelled backwards) so that if m = gorp(p, n) then n = prog(p, m).
Here is a table of some values:
n p gorp(p, n)
-- -- -----------
0 2 [undefined]
1 2 1
2 2 4
3 2 27
0 3 [undefined]
1 3 1
2 3 16
3 2 3^9 = 19,683
Clearly this function rises much faster than anything else I knew
of; also, clearly, prog(2, 0) had to be a new kind of number,
since there is no n such that n^n = 0. (Though 0^0 is undefined,
its limit is 1.)
But the math I was being taught took a different turn. Powers
were not generalized to gorp, but to the EXPONENTIAL function,
b^x where b was a constant, and its inverse was the LOGARITHM,
where if a = b^x then log(a) = x (to the base b). Logs are
very useful (and they are undefined at 0, with a limit of
minus infinity) but they weren't the same as prog, nor did
they yield any new types of numbers. For a while it looked like
the imaginary quantity i, and complex number z = a + bi,
were the last new types of numbers to be defined in western
math.
It did seem very curious to me that a funky new irrational
number called "e" was introduced out of nowhere, and used as
the base of the so-called "natural" log, but since e was
approximately 2.7182818284 and not defined in terms of any
roots or other irrationals I knew about, it didn't seem
very "natural" to me. That is, until I learned that if you
take the area under the curve of the inverse function,
y = 1/x, evaluated from the vertical line x=1 to the vertical
line x=k for some number k, the resulting function is log(k)
to the base e! Huh? Where did that come from? (In calculus
notation, the definite integral from 1 to k of 1/x dx is log(k)
to the base e.)
Then one day in about 11th grade an older student told me that
I wasn't supposed to know this yet, but e^(Pi*i) = -1, or
as he glibly said, "e to the Pi i is minus one!" Boy was I
confused. First of all, what did it mean to take a real number
to an imaginary power? How could you multiply e times itself
"i times" anyway? And secondly, e was from logs, Pi was from
circles, and i was from square roots. How could these unrelated
numbers combine in such a goofy way to make something simple like
minus one? I didn't figure that one out for years.
Every now and then I'd ask a mathematician about my
conjecture, and my ideas for gorp and prog. One told me
it sounded a little like Ackermann's function, a super-
quickly growing function which has been studied since 1928.
Interesting, but not helpful (at least to me).
( www.nist.gov/dads/HTML/ackermann.html )
My friend Bill Moulton alerted me to the work on
"hypernumbers" by Charles Muses. He co-edited a book called
"Consciousness and Reality: The New Pivot Point" (1972) which
included his own essay, "Working With the Hypernumber Idea."
( www.amazon.com/exec/obidos/ASIN/038001114X/hip-20 )
Muses claimed that Hamilton's discovery (or was it an invention?)
of quaternions in 1843 represented a new form of imaginary number,
which did not obey the commutative law: a*b did not equal b*a.
(A quaternion contains 4 components, much like a complex number
contains two. They've never made much sense to me, and they
fell out of favor in mathematics early in the 20th century.)
He goes on to describe a series of such new numbers, ultimately
reaching seven of them, counting real and imaginaries as types
one and two. Each new type breaks another law, such as
the associative law, until the seventh does not even obey identity,
i.e., a=a no longer holds. Muses draws spiritual lessons from
all of this, equating the seven types of hypernumbers with seven
stages of the evolution of human consciousness. (I am reminded
of the alchemists, who made a similar association with the stages
of transmuting base metals into gold.) I have studied this paper
extensively over more than a decade but I've never "gotten" it.
By coincidence (or maybe not) Muses has written extensively
on cybernetics. He passed away in 2002, and "Kybernetes: The
International Journal of Systems & Cybernetics" (which he
contributed to frequently) devoted a special issue to him,
Volume 31 Number 7/8 2002, "Special Issue: Charles Muses - in Memoriam."
( matilde.emeraldinsight.com/vl=3034422/cl=67/nw=1/rpsv/cw/www/mcb/0368492x/v31n7/contp1-1.htm )
This work is also related to "Surreal Numbers" (1974) by Donald Knuth.
( www.amazon.com/exec/obidos/ASIN/0201038129/hip-20 )
Both Muses and Knuth introduce the idea that there are positive and
negative forms of zero (!), each of which satisfies the equation
x^2 = 0, which also relates to Kurt Godel's famous Incompleteness
Theorem -- Godel proved that these forms of roots of zero cannot
be definitely proved to exist or not to exist.
But all of these revelations inspired me to go back again and
look at my conjecture. After learning calculus I was able to
determine that for positive x the derivative of y = x^x was
y' = log(x) * x^x, that is equal to 1 when x = 1 (obviously) and
has a limit of 1 when x = 0, and between the two values it forms
an asymmetric "dip" whose minimum is 1/e of all things! When I
learned to manipulate complex numbers I was able to compute
the real and imaginary parts of z^z where z = a + b*i. When
I learned computer programming I was able to draw graphs of the
function, and later do 3D surface plots of the real and imaginary
parts over the complex plane. These are beautiful -- I will share
them in a future C3M if I can find or rewrite the code I used to
generate them -- but I never was able to use these steps to reach
a definition of a new kind of number.
Along the way, though I was able to learn why the mysterious
e^(Pi*i) = -1 is true. It is a specific result of the general
form of Euler's Formula:
e^(i*theta) = cos(theta) + i*sin(theta)
When theta equals Pi, cos(theta) is zero and sin(theta) is one.
This "magic" formula, which like e^(i*Pi) = -1 I learned from
an older student before I was supposed to know about it, is massively
useful. For example, you can use it to find the so-called "trig
identities" which I had to memorize in high school, such as
sin(2*x) = 2*sin(x)*cos(x).
( www.math2.org/math/trig/identities.htm )
I was shown how to derive these "the hard way" but it never stuck
with me. Using Euler's Formula makes it a breeze, and I never
had to memorize another trig identity again. But where did
Euler get this amazing equality? How do you take the imaginary
power of something?
The answer is to be found in the tool known as the Taylor Series.
I studied this in college calculus class, and was able to pass the test,
but never had a clue what the symbols meant. It fell to a physics
professor, David Dorfan, to provide an intuitive understanding.
( scipp.ucsc.edu/personnel/profiles/dorfan.html )
I still remember the day in electromagnetics class that he asked
if any of us could explain the concept. There was silence. We'd
all studied it; calculus was a prerequisite for his class. I
remember him muttering in his charming, clipped accent (British?
South African?) about "what are they teaching you in the math
department," before going to the board and drawing a few figures
and explaining it all in about ten minutes. "How do you think they
compute sines and cosines for the tables?" he implored us. (This
was before affordable scientific calculators, and we used books of
tables of numbers to find the values of trig functions.) "Do you
think they draw giant circles and measure them?"
Here, then, is a brief explanation of the concept which I wrote
for a book I'm currently working on, "A Survival Guide for the
Traveling Techie" (more on that another time):
The Taylor Series allows you to approximate certain well-behaved
functions with simple arithmetic. For a given function of x --
f(x) -- you create a series of terms using x, x squared, x cubed,
and so on, based on knowing the function's value for some single
value of x (often called a) along with it's derivatives at x = a.
You need to able to find its first derivative, second derivative,
third derivative, etc., or in other words: rate of change, rate
of change of rate of change, rate of change of rate of change of
rate of change, etc., or in still other words: slope of the graph,
slope of the graph of the slope of the graph, slope of the graph
of the slope of the graph of the slope of the graph, and so on.
So when you go to predict the function, you use a potentially
infinite expression, but you only add as many terms as you feel
like doing the arithmetic for; if you add up N terms, we say the
result is an Nth order approximation.
Let's look at a simplified example that uses discrete data.
A say we want the value of the function where x = a + 1.
A zeroth order approximation of the function would be zero.
No matter what the value of f(a) and its derivatives are,
who cares, the result will be zero. And in some cases this
is not a bad approximation. It's like assuming nothing
will happen. Sometimes you're right. A first order
approximation would be whatever the function was at x = a.
It will just stay the same. This is true for all constant
functions. It's like assuming the same thing will keep
happening again. Sometime it does. A second order
approximation involves looking at how the function's
value has been changing, say over the interval from a - 1
to a. Call that difference delta (it's not really calculus
without a Greek letter here and there) and say that the
prediction at x = a + 1 is equal to the value at x = a
with delta added.
The actual definition of the series is a summation of an
infinite series involving all the infinite derivatives of f(x).
|
(
www.well.com/~abs/Cyb/4.669211660910299067185320382047/t1img684.gif )
(In this equation z is used instead of x and z-sub-0 instead of a.
These are "dummy variables anyway so the names don't matter except
stylistically. The expression f superscript n of z-sub-0 means the nth
derivative of f evaluated at z-sub-0, not the nth power.)
A more thorough treatment of the Taylor Series can be found at
Eric W. Weisstein "Eric Weisstein's World of Mathematics (MathWorld)."
(
mathworld.wolfram.com/TaylorSeries.html )
Dorfan helped me to develop an intuition for the Taylor Series,
but it wasn't until I was out of college for a few years that I
realized its relevance to complex numbers. Again I wanted to make
some progress on my conjecture, I got a small book on "complex
analysis" out of the library and read it. (I've forgotten the title
and author now, but it was standard stuff.) I remember vividly that
I was camping with friends in the piney woods in the Cuyamaca Rancho
State Park in the mountains east of San Diego, at Green Valley Falls
Campground.
(
parks.ca.gov/default.asp?page_id=667 )
[Pointless aside: information on Green Valley Falls can be found
at the gorp (!) web site, named for a type of trail mix.]
I was sitting on a picnic table reading when I came to the
explanation of how e^x and sin(x) are related. A friend of mine
happened upon me and said, "Alan, you've got a huge grin or your
face. Why?" I grappled with how to explain it, and finally just
said, "I just found out the answer to an esoteric question in
math that has bothered me for almost ten years, and it's very
simple and beautiful." But I didn't attempt the explanation --
my friend was not a math type. I will attempt it now, though.
The Taylor Series for both e^x and sin(x) are very easy to
derive, because the derivatives of these functions are so
simple. The derivative of f(x) = e^x is f'(x) = e^x, itself. That's
right, it's its own derivative. So it is its own second derivative
(f''(x) = e^x), third derivative (f'''(x) = e^x), fourth derivative
(f''''(x) = e^x) and so on as well. It is the only function that has
that property, which turns out to be very significant in the
study of Ordinary Differential Equations (ODEs) and dynamical
systems theory using those equations. So in a Taylor expansion
with a = 0, f(a) = e^0 = 1, f'(a) = 1, f''(a) = 1, f'''(a) = 1
all the way up to infinity, and so the terms of the series resolve
to just the rest of the expression, (x - a)/n! where n! is n factorial.
(Both 0! and 1! are defined to be one.) So you can approximate e^x with:
x^0/0! + x^1/1! + x^2/2! + x^3/3! + x^4/4! + x^5/5! + ...
simplified to:
1 + x + x^2/2 + x^3/6 + x^4/24 + x^5/120 + ...
You can see how that n! in the denominator makes the terms
for higher powers drop off really fast. This series converges
very quickly and you can get good quality approximations with
only a few terms.
Now consider f(x) = sin(x). The derivative of a sine is a cosine,
and the derivative of a cosine is a minus sine. So when a = 0,
f(a) = 0 (sine of zero is zero), f'(a) = 1 (cosine of zero is one),
f''(a) = 0 (minus sine of zero is also zero) and f'''(a) = 1
(minus minus cosine of zero is still one) and we're back where we
started. So you can approximate sin(x) with:
0*x^0/0! + x^1/1! + 0*x^2/2! + x^3/3! + 0*x^4/4! + x^5/5! + ...
Those multiplications by zero cause the even terms to vanish
and you're left with:
0 + x + 0 + x^3/6 + 0 + x^5/120 + ...
Likewise cos(x) works out to:
x^0/0! + 0*x^1/1! + x^2/2! + 0*x^3/3! + x^4/4! + 0*x^5/5! + ...
or:
1 + 0 + x^2/2 + 0 + x^4/24 + 0 + ...
The odd terms have dropped out in this case.
Now for the magic: these equations involve x only as whole
number powers: x^1 (i.e., x), x^2, x^3, x^4 and so on. Well,
we know how to take whole number powers of imaginary quantities,
right? By definition i^2 = -1, so we know i^1 = i, i^2 = -1,
i^3 = -1*i, i^4 = 1, and it repeats. So: WE CAN FIND THE VALUES
THE EXPONENTIAL AND TRIG FUNCTIONS WHEN X IS IMAGINARY!
Try it yourself, plug i into each series above and see what you get.
What you'll notice is that e^i gives a series that can be sorted
out, every other term, with a real series for cosine and an
imaginary series for sine. What it ends up showing you is
Euler's Formula:
e^(i*theta) = cos(theta) + i*sin(theta)
No wonder they called him a genius, and no wonder I was grinning
on that picnic table!
Now I know some of you already knew this, and some of you
followed my argument and are now going "Wow!" and some of you
are just plain lost. Sorry about that. One of the ironies
of my own education was that in my junior year of college
I gave up being a math major and dropped a class called "linear
algebra" because it was boring and I just didn't get it. Plus
I had no clue what it was for. I decided instead to pursue an
individual major in "understanding whole systems" with Gregory
Bateson as my adviser. If I'd stayed in that class I would
have learned the above facts in short order. Many years later
I discovered that the most powerful mathematical tools for
studying the general behavior of systems are found in linear
algebra. In other words, of all the math courses at UCSC, it
was the one MOST RELEVANT to what I wanted to study. (I didn't
figure this out for another 10 years.) I blame the way the material
was taught. At that time, 1973, it was a HUGE TABOO in higher
math education to APPLY any of the knowledge or to teach students
how to do so. I came up with this analogy: if they taught mountain
hiking like they taught math, you would be forced to wear blinders
so that you could only see your feet. They wouldn't let you see the
mountain from afar before you climbed it, and they wouldn't let you
see the view from the top -- if you made it.
Another irony: right as I left Santa Cruz, a group of rebels was
forming (in the low-rent Applied Sciences building, isolated from the
mainstream math and science students) which later became known as
the "Chaos Collective," early pioneers of chaos theory, which eventually
was instrumental in healing the rift between pure and applied math
and science in the 1980s. This story is told in "Chaos: making a New
Science" (1987) by James Gleick.
(
www.amazon.com/exec/obidos/ASIN/0140092501/hip-20 )
Okay, fast forward another decade or so, after some background.
My parents were both born and raised in Memphis, Tennessee,
but they transplanted to San Diego, California in 1959, and my
sisters and I grew up there. Our only nearby kinfolk were my
dad's cousin, Robert Scrivener, "Uncle Bob," and his wife Dorothy,
"Aunt Dot." We saw them every Thanksgiving and Christmas, on
birthdays, and other times too, since they were our only relatives
within a thousand miles. Eventually Uncle Bob passed away, and Aunt
Dot married a man named Charles Curtis ("Uncle Charlie"). Though
neither of them was a blood relative we still saw them a few times
a year. Eventually Uncle Charlie passed away, and later so did
Aunt Dot. My parents were executors of her estate, and she left
almost everything to her church. My folks ended up with a pile
of stuff the church didn't want, and so they invited us kids to
pick through it before the rest was tossed in a dumpster. Among
the old National Geographics and Arizona Highways magazines I found
Uncle Charlie's book collection, which divided into three categories:
math puzzles based on number theory (such as Fermat's Last Theorem),
electromagnetic physics texts, and textbooks on electrical circuit
theory. I found myself wishing I'd known he had these interests when
he was alive. Aunt Dot had been a registered nurse and had no
interest that I could see in math or engineering, and it never came
up. I never even knew what Uncle Charlie did for a living.
But his textbooks have enriched me. They proved to be a starter
for a whole collection; several friends saw them on my shelves and
contributed some college texts of their own to my growing
collection, on network theory, control theory, mathematical
modeling, and even cybernetics. Opening up these texts on
occasion I found the math daunting. One concept that I kept
running into -- which I was in complete ignorance of -- was the
Laplace Transform. I thought that maybe some day I would learn
what it was.
In 1988 In began working in the field of scientific computing,
and had to "port" the FORTRAN programs of scientists to a new
model of min-supercomputer (if that isn't an oxymoron, like
"jumbo shrimp.") I ran across the term Laplace Transform often
in the comments to the code. I remember it reminded me of the
slogan, "LA's the Place!" My office was right next to the Los
Angeles Airport (LAX), and every time I passed the little bars
in the departure areas I would see these banners advertising a
drink called a "Green Eyes," the "official" drink of LA, and
the 1984 LA Olympics. Since my wife has green eyes I ended up
trying one. It was a supersweet "candy drink" but I liked it and
ended up buying a set of novelty glasses decorated with green palm
trees, the slogan, "LA's the Place!" and the recipe for the drink:
3/4 oz. (22.5 ml.) Midori
1 oz. (30 ml.) Rum
1/2 oz. (15 ml.) Cream of coconut
1/2 oz. (15 ml.) Lime juice
1 1/2 oz. (45 ml.) Pineapple juice
Blend with crushed ice
(I never met anybody besides the LAX bartenders during the 12 years
I lived in the LA basin who'd heard of the drink, let alone knew it
was the "official" drink of LA.)
That was as close as I ever got to understanding a Laplace Transform
(those Green eyes really got you drunk fast; that was quite a transform!)
until a few weeks ago. I once again cracked one of Uncle Charlie's texts
and ran smack into the loopy L symbol used to represent the Laplace
Transform, and I decided it was time.
I knew who Laplace was. He gave us probability theory pretty
much as we know it today, solving the problem of how to fairly
split up the pot among gamblers in an unfinished, interrupted card
game. From reading "Men of Mathematics" (1937) by E.T. Bell,
(
www.amazon.com/exec/obidos/ASIN/0671628186/hip-20 )
I knew he had lived through the French Revolution and Napoleon's
subsequent rise to power. I knew he had made important contributions
to celestial mechanics. From "A History of Mathematics" (1968) by
Carl B. Boyer,
(
www.amazon.com/exec/obidos/ASIN/0471543977/hip-20 )
I learned he was one of the "three Ls" of 18th century France:
Lagrange, Laplace and Legendre, all of whom lived to ripe old
ages. An on-line biography can be found at the School of Mathematics
and Statistics, University of St. Andrews, Fife, Scotland.
(
www-groups.dcs.st-and.ac.uk/~history/Mathematicians/Laplace.html )
A number of portraits of him are on-line as well.
(
www-groups.dcs.st-and.ac.uk/~history/PictDisplay/Laplace.html )
The significance of his work is discussed in the on-line "Wikipedia."
(
www.wikipedia.org/wiki/Pierre-Simon_Laplace )
To begin my quest, I went to Amazon.com and rather arbitrarily
selected a text, "Complex Variables and the Laplace Transform
for Engineers" (1961) by Wilbur R. LePage, which I ordered.
(
www.amazon.com/exec/obidos/ASIN/0486639266/hip-20 )
When it arrived I was a little disappointed that it was intended
for graduate students who had already learned to solve problems
with the Laplace Transform but wanted a deeper understanding.
But I forged ahead. I did quite quickly get the definition of
the Laplace Transform: if f(t) is defined for all real numbers
greater than or equal to 0, then L(f(t)) is defined as the definite
integral from 0 to infinity of f(t)*e^(-s*t) dt = F(s).
(
www.well.com/~abs/Cyb/4.669211660910299067185320382047/l1img987.gif )
Its remarkably simple. I also learned what kinds of problems
it can solve, and how it is used. A classic problem in circuit
theory is: given a circuit made of a capacitor (C), a resistor (R)
and an inductor or coil (L), all in series with an input source of
current with a driving function Va, and measure the output function
Vb across the resistor; find the system function H which expresses
the output in terms of the frequency of the input and the values of
C, R and L. Now replace the capacitor with a whole sub circuit made
of the same type of components and solve again. The Laplace Transform
using this magical property: if you cascade (or "convolve") functions
such as some f(x), g(x) and h(x) into f(g(h(x))), you can find the
Laplace Transform of each function, simply add them up, then apply
the Inverse Laplace Transform and, "Bob's your uncle!" (as they say
in the UK), there's your answer. I was reminded of the power of
logarithms to change multiplication into addition, which is what
makes slide rules work. To clarify here, logs and other operators
work on numbers, to make new numbers, while transforms work on
functions to make new functions. There was a cute comic book to
teach calculus called "Prof. E McSquared's Calculus Primer" (1989),
(
www.amazon.com/exec/obidos/ASIN/0971462402/hip-20 )
that showed transforms as robots that took functions in and
ejected the resulting transformed functions, to illustrate what
derivatives and integrals did. Jim Blinn used a similar technique
in his computer effects for the "Mechanical Universe" TV series.
(
www.pbs.org/als/mech_univ1/ )
But I still didn't have what I wanted. The text I was using
taught me how to plug in the symbols and solve some problems,
and I'm sure there are armies of engineering students out there
who've been trained to do just that. But I didn't have any circuit
design problems to solve; I was after the "deeper understanding"
promised in the introduction and I hadn't gotten it yet.
One clue was that the Laplace Transform is sort of a generalization
of the Fourier Transform. This I was familiar with from my
scientific computing days. Everyone wanted to know how quickly
the new mini-supercomputers could do FFTs -- Fast Fourier
Transforms. I educated myself about this tool, and it was
pretty easy to develop an intuition for the concept. Anyone
who has watched the LED display on a stereo's graphics equalizer
has seen an FFT in action. The input signal, a series of numbers
in the "time domain" (i.e., where the speaker cone is located at
a series of moments in time) is transformed into the "frequency domain"
(what frequencies are present in the signal). This makes sense to
us because it is how our ears work: the little tiny hairs in our
cochlea (inner ear) vibrate nerve endings to tell us what frequencies
are present in the sounds we hear, sort of like how a piano's wires
will vibrate in resonance to sounds in their vicinity.
But here is where I was stuck: the Laplace Transform goes from
the time domain to the "s" domain. Time vanishes and the function
is expressed in terms of a new variable, s, which nobody seemed to
be able to explain.
What I was looking for was an intuition for the meaning
of the equations, so I went to google.com and typed in:
"Laplace Transform" intuition
One web site said s was the "Laplace variable." Oh, great.
another said "The s-domain is simply another way of analyzing
mechanical and electrical systems." I already knew that.
I was offered the clue that s sometimes represents a frequency.
But what is it the rest of the time?
A good summary of the material in my text was at "Eric Weisstein's
World of Mathematics (MathWorld)" at the Wolfram Research web site,
but it didn't attempt to explain s.
(
mathworld.wolfram.com/LaplaceTransform.html )
Finally I happened upon the web site of Duncan K. Foley of the
Department of Economics Graduate Faculty, New School University
in New York.
(
homepage.newschool.edu/~foleyd/ )
In a PDF document entitled "Laplace Transforms"
(
homepage.newschool.edu/~foleyd/GECO6289/laplace.pdf )
he wrote:
From an economic point of view we immediately recognize the
Laplace transform as the present discounted value of the stream
of returns f[t] at the interest rate s. If f[t] is continuous
and differentiable at all t >= 0, then it is possible to recover
f[t] from L[f][s] through the inverse transformation...
This has the economic meaning that if we know the present
discounted value of a stream of returns at every interest rate,
we can recover the whole pattern of the stream of returns.
Ahah! I got it. The "present discounted value" is what an annuity
is worth if you cash it out now. Let's say you go to your accountant
and say, "I won the lottery and they're paying me $10,000 a year, but
I want all the money now. I heard an ad on the radio for these folks
who will give me a settlement now, and they offered me $200,000 cash.
Is this a good deal?" Your accountant says that depends on what
interest rates do, and she gives you a spreadsheet that gives you
a "present discounted value" when you plug in a guess as to what the
interest rate will be. (One simplification -- this model assumes
interest rates are going to hold constant at some rate from tomorrow on.)
In this example, f(t) is the lottery payout function over time,
s is the interest rate, and F(s) is the spreadsheet she gives you.
The amazing thing is, as it says above, "if we know the present
discounted value of a stream of returns at every interest rate,
we can recover the whole pattern of the stream of returns." This
is analogous to the result in Fourier's work, that the time series
can be completely reconstructed from the frequency distribution.
Okay, one more thing for extra credit: if I understand this
correctly, s can be a complex variable. Imagine s = v + w*i,
then v is an "interest rate" and w is a frequency. Taking e^s
gives a combination of exponential growth or decay (e^v)
and harmonic oscillation (e^w*i).
One of the things I learned when I worked on the Support Vector
Machine project in 2001 (see C3M vol. 2 number 3) is that
when you take two strings of numbers (vectors) and "dot product"
them, that is, take (x1, x2, x3, ...) and (y1, y2, y3, ...) and
compute x1*y1 + x2*y2 + x3*y3 + ..., you are measuring the
CORRELATION between the two vectors. In my computer graphics days
I learned to use dot product to find the cosine of the angle between
two 3D vectors of unit length. If they are pointing in the same
direction the cosine is 1, and if they are at right angles the cosine
is zero, so this can be thought of as quantifying how they correlate.
Generalize this to N dimensions, and then to a continuous case.
The Fourier Transform correlates a function with a series of
harmonics; the Laplace Transform correlates it with a combination
of harmonics and exponential curves.
if you have read my "Curriculum for Cybernetics and Systems Theory"
which inspired this e-Zine, you may recall the description and graphs
in the section "Where Cybernetics and Systems Theory Came From"
(
www.well.com/~abs/curriculum.html#From )
Maxwell's analysis of the behavior of governors produced
predictions that combine harmonics and exponential curves,
as the graphs show. For a long time these were thought to be
the only behaviors systems could exhibit.
I still have several bankers boxes filled with issues of
the CoEvoltuion Quarterly and the Whole Earth Review from
the 1970s and 1980s, and I have searched them in vain for the
quote I want to share with you now. Following the publication
of an article by Howard Odum on global energy dynamics, which
included a block diagram of his model of world energy use,
a reader wrote in to say (I'm paraphrasing), "I'm an electronics
engineer, and that diagram looked like an electrical circuit,
and what I know about electrical circuits is they can only do
three things: they can do nothing, they can oscillate, or they
can blow up."
This is an interesting insight, but it is technically not true.
The fourth alternative is they can exhibit chaos, by seeking a
strange attractor. But it only happens in non-linear systems,
which are systems where the Laplace Transform in useless.
Laplace didn't know about chaos because he didn't have a computer.
I have a photocopied page from Kenneth Boulding's "Conflict and Defense:
A General Theory" (1962) -- at least I think that's where it's from;
I neglected to note the book title on the page.
(
www.amazon.com/exec/obidos/ASIN/0819171123/hip-20 )
Wherever it's from, it describes the trajectory of a deterministic
system through a state space, complete with some 2D diagrams,
emphasizing that the path must be unique and so can't fork or cross,
and then asserts:
A moment's consideration will convince you that (since the path
must be unique) a state determined behavior must either converge...
to a fixed state called the 'equilibrium point', or enter a
'behavioral cycle'... Either mode of behavior is called a stable
equilibrium because, unless there is a disturbance which moves the
state point (or alters the subsequent transformation), its behavior
remains invariant.
Again, this is wrong. More than a moment's consideration was
needed to discover that this argument only applies to the 2D case.
In 3 or more dimensions you can find strange attractors
(
sprott.physics.wisc.edu/fractals/animated/nhrisk.gif )
such as Rossler bands
(
sprott.physics.wisc.edu/fractals/animated/ROSSLER.GIF )
Birkhoff Bagels, and other monstrosities. More more such images
see Sprott's Fractal Gallery on-line.
(
sprott.physics.wisc.edu/fractals.htm )
Clearly, 300 years plus of complex analysis using the powerful and
elegant tools bequeathed to us by Laplace have produced generation
after generation of mathematicians, engineers, economists and other
systems experts with intuition lopsidedly biased towards the
linear systems that can be studied analytically. Now with
cheap computers in the hands of new students there is hope that
more "computational experiments" a la Woldfram can run the intuition
the other way.
=======================================================================
newsletter archives:
www.well.com/~abs/Cyb/4.669211660910299067185320382047/
=======================================================================
Privacy Promise: Your email address will never be sold or given to
others. You will receive only the e-Zine C3M unless you opt-in to
receive occasional commercial offers directly from me, Alan Scrivener,
by sending email to abs@well.com with the subject line "opt in" -- you
can always opt out again with the subject line "opt out" -- by default
you are opted out. To cancel the e-Zine entirely send the subject
line "unsubscribe" to me. I receive a commission on everything you
purchase during your session with Amazon.com after following one of my
links, which helps to support my research.
=======================================================================
Copyright 2003 by Alan B. Scrivener
