91av

JOHN HOLLAND spends his summers in a house in the woods near the shore of
Lake Michigan, a stone’s throw from the Canadian border. He describes his
hideaway as “a place of peace, long horizons and the ever-changing face of
nature”. This seems the perfect setting for a man who draws so much inspiration
from nature: he has been called Mr Genetic Algorithm for his invention of
computer programs that change in ways that echo evolution. “Biology is such a
rich source of ideas,” he says. “My forté is to take those questions,
chew off a lot of stuff, and finally form what for me is usually a computer
model that gives a cartoon sketch of the question.”

These days, the models he’s thinking about are of complex systems, from
economies to ecosystems, which behave in ways that stump even the most
progressive mathematician. Holland, then, has shed his old sobriquet and donned
another: Mr Emergence.

Emergence is ubiquitous in nature. It is what turns the simple interactions
between pairs of ants into complex colony-wide behaviours, changing the colony
into what appears to be a single “superorganism”. And it is the process that
allows the human immune system to create a seemingly endless variety of
antibodies from a small number of genes. In every case, the whole is so much
more than the sum of the parts.

Holland describes the phenomenon as “much coming from little”, which creates
an aura of mystery and paradox. “Our present understanding of emergence is not
much better than the child saying that Jack Frost explains the wondrous colours
of autumn,” says Holland. “That kind of explanation might stir our imagination,
but ultimately it is unsatisfying.” Although some who study complexity suggest
that this mystery puts emergence beyond the reach of scientific inquiry, Holland
disagrees: “With the right kind of science, one day we will come to understand
it.” He admits, however, that “we are a long way from that point as yet”.

What is known about emergence is that it appears in systems in which a few
simple rules govern the interaction of the component parts. Take a cellular
automaton, for instance. This consists of a grid of squares, each of which
changes its state—its colour, say—at regular intervals according to
rules that depend on the state of the squares around it. These rules can be
simple, yet the behaviour of the entire device can be very complex, with
“objects” emerging, travelling and interacting in their own distinct ways
(see Diagram).
Holland wants to find a path for understanding the dynamics of
such interactions in the real world as well as artificial ones, to discover how
“much can come from little”.

To understand emergence, we first need a theory to explain it. Yet so far it
isn’t even clear which features are essential to emergent systems and which are
incidental. And there is no mathematics capable of modelling it. This is the
tricky terrain that Holland is negotiating today—identifying features that
are common to emergent systems and building an intellectual framework within
which to consider them.

Even to reach his present position, Holland has travelled a long way. Though
he didn’t realise it at the time, his journey began many years ago—with
games. “One of my earliest memories is of playing checkers [draughts] with my
mother, when I was about four years old,” he says. “And, as I had a younger
sister, I enjoyed having my mother’s undivided attention. Sibling rivalry, you
know.” A few years later, his sister was allowed to join Holland in the game
circle, the two playing bridge with their parents.

Playing God

Games became an obsession for Holland, not so much for the playing as for the
process. “The minute I understood that games were based on a few simple rules
that are agreed on by everyone, and that endless consequences flowed, different
every time, I was delighted, fascinated,” Holland says. “Pretty soon, I was
making up games of my own, making whole new artificial worlds and seeing what
happens. It was like playing God.” Holland has been making up games ever since,
only these days they are more appropriately described as scientific
models—abstract cartoons that capture the essence of some part of nature
cast as computer algorithms.

Holland discovered early on that he had a facility for mathematics, which was
much encouraged by his parents. In 1946 he went to the Massachusetts Institute
of Technology to pursue the subject in depth, but soon found that he was more
interested in exploring how electronic computers might be used to capture
thought processes. He continued this odyssey at IBM, where he became entranced
with a new theory of how the brain learns and adapts, the brainchild of Canadian
psychologist Donald Hebb. In Hebb’s theory, the connections—or
synapses—between the brain’s nerve cells constantly change in response to
the organism’s experience. Holland became one of a few pioneers who tried to
model this notion in artificial neural networks. Today such networks are used as
powerful computers
(“Cell Wars”, 91av, 21 February, p 36).

The experience of this early exploration influenced much of his thinking for
the next three decades. Also influential was the work of Arthur Samuel, whom IBM
hired to develop reliable vacuum tubes. But, like Holland, Samuel spent most of
his time working on what really interested him—he wrote a program that
could not only play checkers, but could also learn. The checkers player soon
surpassed its inventor in skill. By 1967 it was playing at world championship
level and is still unbeaten today, one of the great achievements in this brand
of model building.

Simple rules, a universe of possibilities, and with the capacity to adapt,
the checkers-player program was a source of wonder to Holland. In 1952,
realising that programming computers for IBM was not his future, he enrolled as
a postgraduate student in mathematics at the University of Michigan, Ann Arbor,
where he has been ever since.

Two intellectual ships sailed across Holland’s bow during his doctoral
studies, throwing him off course both literally and figuratively. The first was
reading Herman Hesse’s novel Magister Ludi (Master of the Game or
The Glass Bead Game), in which people are involved in a complex game that
involves weaving endless novel themes by moving glass beads on an abacus. “Of
all the things that have influenced the way I do things, Hesse’s novel ranks
first,” Holland says. “It’s the essence of what I mean by inventing things.”

The second was another book, The Genetical Theory of Natural
Selection by R. A. Fisher, the foundation of modern population genetics. In
this case, it led Holland to play God again—to develop his genetic
algorithms, which harness evolution to find optimal solutions to problems. Long
regarded by many of his peers as distinctly weird and unlikely to be useful,
genetic algorithms are used today as powerful tools for solving complex problems
of many kinds, in business as well as in science.

Emergence, as a general phenomenon to be explained, is relatively new.
Holland says that for a long time he understood that his genetic algorithms were
what complexity theorists now call complex adaptive systems. Namely, systems in
which what emerges is not only greater than the sum of the parts but it also
changes—adapts—as a result of experience. But it wasn’t until the
late 1980s, when he started visiting the Santa Fe Institute in New Mexico, the
mecca of complexity theory, that he came to see emergence as a ubiquitous
phenomenon.

“I started having conversations with lots of people, in physics, in
economics, in biology,” he says, “and I came to see that the rules of the game
were much the same, no matter what field you look at.” He soon became convinced
that emergence was Big, in a way that people at Santa Fe had not fully
formulated, so much so that in his recent book* he wrote: “We will not
understand life and living organisms until we understand emergence.”

As to the precise nature of emergence, Holland is necessarily vague. “None of
us has a solid grasp of emergence, far less a full definition,” he concedes.
“Emergence is multifaceted, and if you try to be too precise, you will lose what
you’re after.”

Science rules

Holland’s approach to pinning down emergence is through his models. “I’m
talking about models in the sense that Newton’s laws of motion form a model of
the Universe,” he says. “These laws have a lot in common with a game. The
equations describe `rules’ of the game in which `moves’ can be made with the
help of the tools of mathematics.” And just like the rules of a game, physical
laws do not encompass everything that can flow from them. For instance, “Newton
could not have guessed that his equations would reveal the gravity-assisted
boost that takes space probes to outer planets,” Holland notes. “The same is
true of the five axioms of Euclidean geometry, which are still providing
surprises after centuries of study.” These cases echo what has happened with
chess: “The rules of the game have been known for a millennium but new and more
powerful combinations are constantly being discovered,” Holland says. “This
should make us realise how little we probably know about the Universe, for which
the rules are unknown and [which] is vastly more complex than chess.”

A mathematical model of emergence would be like manna from heaven. It would
certainly be a miracle, because most established mathematics describes linear
systems in which, for example, the doubling of an input doubles the output.
Emergent systems are anything but linear. The state of each square in a cellular
automaton, for example, is not decided by a simple sum or product. “Ninety-five
per cent of the mathematics I know is by the board for my kind of work,” Holland
says.

One device that can handle large numbers of nonlinear interconnections is the
computer—Holland’s favourite tool. He imagines models of emergence in a
setting that mimics an idea that goes back to classical Greece. “The Greeks
argued that all machines can be constructed by combining six elementary
mechanisms: namely the lever, screw, inclined plane, wedge, wheel and pulley,”
Holland says. “In an intuitive way, this leads me to look at emergence in terms
of elementary mechanisms and procedures for combining them. It’s the interaction
among these mechanisms, or building blocks, that generates emergence.”

Holland’s models, which he calls constrained generating procedures (CGPs),
are made up of a number of virtual mechanisms, each of which computes an output
that depends on the state it is in and the inputs it receives. This may seem an
unremarkable device, but as Holland says, it’s the interactions that count. He
begins to plug the output of one mechanism into the inputs of others, and
creates feedback loops or attenuates the signals travelling between mechanisms.
With such techniques he can build first a cellular automaton, then a neural
network and even Samuel’s checkers player. By feeding one type of mechanism into
another, he can create CGPs that change their own structure, much as a stock
market changes its nature as its members make their trades.

With his models, Holland can also explore the idea that one emergent system
can itself become a mechanism— or building block—for a “higher”
system. For example, a pond is a complex adaptive system made up of the
organisms—the building blocks—living in it. The pond is then a
building block for the meadow ecosystem, a complex adaptive system at a
higher level, and the meadow ecosystem is a building block in the regional
ecosystem, and on it goes, right up to the global ecosystem.

“It’s building blocks all the way down,” Holland quips, “and I’d like to get
closer to the notion of what that means.” But getting closer isn’t going to be
easy. Once again, mathematics—or the lack of it—gets in the way. “I
need to be able to work with the notion of layers of models, and mathematicians
haven’t thought much about that,” says Holland.

Arcane in the extreme to mere mortals, Holland hopes that this new class of
models will lead him to a new form of mathematics that will penetrate the
mysteries of emergence. In any case, he expects—has faith
anyway—that this approach will bring into view the mechanisms of
emergence, and will create “a context so stark that nothing lies hidden in the
complexity, nor is there any room for mysterious, unexplained activity.”

In contrast with most people who study complexity, Holland sees his work as a
continuation of the traditional reductionist approach in science. “Reductionism
has been tremendously powerful, and it’s amazing how much has been achieved that
way,” Holland notes. “You take a system, study the parts, and you can understand
a lot about the system.”

But to some people this approach sounds coldly analytical, lacking in wonder.
“Not for me,” he responds. “I find it more exciting, not less, when I understand
how something works. There’s a tendency for people to talk about emergence as
surprise. `Emergence is what surprises you’, that kind of thing. So if you come
to understand the system, how and what it generates, then it is no longer
emergence, by definition. To me that’s wrong. It is true, though, that surprise
can be a signal of emergence, but it doesn’t define emergence. For me, there’s a
sense of wonder, of awe, about emergence, and that won’t disappear when I
understand it.”