The Mind Made Matter

"If a stone could think he would think that he wished to drop.”

– Ludwig Wittgenstein, Cause and Effect: Intuitive Awareness

Chopin’s Preludes for solo piano are regarded as some of the more trailblazing compositions in the history of piano music.

The preludes were written between 1835 and 1839 in a remote and chilly Carthusian cell in the monastery of Valldemossa in Majorca. Chopin described his cell as “shaped like a tall coffin.”

Chopin's Preludes followed the template of Bach’s 48 Preludes and Fugues published in his Well-Tempered Clavier from 1722. The novelist George Sand, Chopin’s partner at the time wrote, "His genius was filled with the mysterious sounds of nature, but transformed into sublime equivalents in musical thought…” As it turns out both Natural history and instrumental history.

No workroom could be worse equipped for acoustics than the cell in Valldemossa. And the piano on which Chopin composed the Preludes was locally constructed by an otherwise unknown carpenter, Juan Bauza. The piano was by the standards of its time a dud. Paul Kildea writes Chopin's Piano, his thoughtful history of Bauza, “it was out of date before it was completed… unable to support thicker or longer strings, greater tension or larger compass… its wooden frame hostage to the island’s fierce climate."

And yet it was on this nominally wretched instrument that Chopin wrote the Preludes and not on the far more sonorous Pleyel pianos on which he was accustomed to composing. It is as if the further Chopin moved from his Platonic musical ideals, into the texture and tension of physicality, the more his imagination found the space to become creative. In Majorca he composed not as Bach’s epigone but as his heir apparent.

From the pub-like keys of a practice piano in a lugubrious Köln winter in 1975, Keith Jarrett improvised a composition that would go on to become the best-selling Jazz album of all time. In reflecting on the condition of his piano, Jarrett himself described it as not worthy of recording.

It was over a hundred and thirty years after Chopin that the thirty-year-old Jarrett planned to perform a solo piano recital on a Bosendorfer 290 Imperial concert grand. On the night of the performance, Jarret was delivered a vastly less edifying, and out of tune, Bosendorfer baby grand. As Jarret describes it, "We had the wrong piano rented… I had a Bosendorfer I really didn't like…I had not slept for two days… everything was wrong. When you have problems one after another you forget what you are doing.”

Like Chopin's Bauza, Jarret's physical instrument had "ideas of its own." And these did not align with anyone's ideal tonalities. The mini grand sounds less like a piano and more like a fortepiano and it wants to be played like a harpsichord. Jarrett achieved his effects throughout the first movement by abstaining from the use of the sustain pedal. Jarrett’s isorhythms of involuntary serialism are reminiscent of the contemporary experiments of Philip Glass, Terry Riley, and Steve Reich with sparse arpeggios and polyrhythms.

It took a harmonically compromised piano to lure Jarrett away from experimental jazz into a space hitherto dominated by modern classical composers. And it took a similar experience to dislocate Chopin into the baroque lyric of his Preludes. Both Chopin and Jarrett were experiencing exbodiment – that state of cognition characterized by a collaboration between organismal mind and environmentally sourced matter. One where matter becomes a computational collaborator and not merely a mechanical servant.

Bee Compasses

In 1973 the Nobel prize for Physiology or Medicine went to a rather unlikely triumvirate: Karl von Frisch, Konrad Lorenz, and Niko Tinbergen. This was the first and the last time that the prize was awarded for animal behavior – ethology. To put this in perspective, all Nobel prizes in the previous several decades had been awarded for reductionist approaches to molecular medicine. Following the 1973 award not a single prize has been awarded to whole organism biology. For the Nobel committee, Physiology and Medicine are merely redundant synonyms for cell biology.

Even more extraordinary is that Karl von Frisch had at no point in his career shown any interest whatsoever in human beings or their cells. The closest he got to cells were the hexagonal lattices of honeybee hives. This contrasts favorably with Konrad Lorenz who had a rather unhealthy fascination with human copulation, aggression, and domestication, which would ultimately draw him into the delirious orbit of Nazi science. In a series of “studies” in the 1940s, Lorenz explored the way urbanization was leading to genetic degeneration which he thought best remedied through eugenics. While Lorenz goose-stepped his way through the Second World War, the ornithologist Niko Tinbergen fought for the resistance and was imprisoned by the Nazis. And Von Frisch, who had throughout his career worked with gifted Jewish scholars and women, and who possessed through his mother distant Jewish relatives, was threatened with unemployment by the Reichsministry of Education, and narrowly escaped imprisonment.

In a 1968 review of the life and work of von Frisch, the late Edward Wilson wrote, “Every successful scientist has a small number of personal tools with which he levers discoveries out of nature. Von Frisch had two in which he attained great mastery. The first was the repeated exploitation of the passage of honeybees from nest to flowers and back again, a complex sequence of behavioral events that is nonetheless easy to manipulate and to monitor. The second was the method of Pavlovian training, by which von Frisch associated the stimuli to be studied with a subsequent reward of food.”

And using these experimental tools von Frisch was able to discover and decipher one of Nature's most extraordinary signalling systems, the so-called dance language of the honeybee. In brief, bees returning from a source of pollen communicate the direction and the distance of nutritive pollen to the colony through an elaborate sequence of movements set against a honeycomb. The angular deviation of a forward waggle motion from the overhead position of the sun in relation to the comb describes the orientation of a food source. The amplitude of the waggle encodes the distance. Thus, the stable position of the comb allows the bee to use gravity as a reference.

In many ways the hive with its combs serves as an inclinometer rather like an astrolabe. The astrolabe is a simple two-dimensional model of the celestial sphere which can be used to measure the precise position of celestial objects above the horizon. And in this way determine time and latitude. In order that an astrolabe can function reliably, a plumb line is required to ensure that it is positioned perpendicular to the earth or ocean. The hive combs likewise need to preserve a fixed angle with respect to gravity, whereby the orientation of the waggle dance, serves rather like the angular deviation of a celestial cue relative to the horizon.

The hive and combs are part of the system of computation required for effective foraging. And like an abacus and a slide rule, employ persistent features of physical geometry to allow for very precise behavior. A foraging bee is part body, part collective, and part physical hive. The functional unit of navigation is like Chopin's Preludes and Jarrett's Köln concert – the behavioral manifestation of an evolutionary trajectory embedded in a life-constructed physics.

Niches of Power

Ecologists study the interaction of species with their environments. Following Thomas Malthus, Charles Darwin had made environmental resource limitation the foundation of his regulatory theory of natural selection. Competition and cooperation in the face of environmental scarcity makes for adaptation. And the environment changes slowly against which organisms evolve relatively quickly.

Early formal ecologists realized that natural selection implies that the precise structure of the environment matters and that it should be described as carefully as the organism. Whereas organisms possess obvious features such as anatomy and physiology, environments possess niches and trophic interactions – energy flows. There are architectures of ecology just as there are of organisms.

Joseph Grinnel described the niche as an abiotic (non-living) recess in which the organism is tucked. Charles Elton sought to find some space in this recess for other organisms. But it was George Evelyn Hutchinson who made the first step toward formalizing the niche when he conceived of an "n-dimensional hypervolume" of survival-relevant resources. For Hutchinson, life might take place in three dimensions of space, but it is involved in vastly more dimensions of interaction and dependency: the dimensions of sunlight; water; oxygenation; etc. Hutchinson's dimensions are quantities analogous to the coordinates of space. They are the coordinates of existence. 

It did not take too long for the question to arise, where do all these the niches come from? Grinnel had suggested that they come from physics. Elton and Hutchinson further proposed they also derive from other organisms. Then a series of researchers including Richard Dawkins in 1978, Richard Lewontin in 1982, Charles Jones and colleagues in 1994, and John Odling Smee and colleagues 1996, participated in a movement to undermine the whole organism environment dichotomy. According to these researchers, environments are in large part extended phenotypes or constructed niches or engineered ecosystems. The world is like a body that develops and grows out of both self and non-self. The niche had been a convenient fiction that makes natural selection efficient. The non-fiction account makes the niche an adaptive extension of the organism evolved to overcome the limitations of the body and generate a super-physical system.

A nice example of the extended phenotype is found in a study from 2019, in which Sarah Han and colleagues discovered how the web of the triangle weaver spider provides a potent source of external power amplification. The key idea is to slowly accumulate potential energy and then rapidly dissipate kinetic energy that can exceed the limits of physiology. The spider can increase the tension in the web silk by ratcheting their legs inwards through a leg-over leg loading motion. This iterative process takes place over prolonged intervals of time taking minutes to hours. When potential prey is caught in the web, this tension is released by rapid withdrawal of the limbs, causing the web to contract locally around the prey from all directions in a second or less. The spider-web extended system exploits the tensility and ductility of silk to overcome the kinematic constraints associated with hydraulic pressure exoskeletons. 

Niche construction and its cognates raises the obvious question, how are we to distinguish between organism and organism-built environment? In a paper from 2009, my colleagues and I answered this question with a mathematical theory of niche monopolies. We proved that exbodiment can only evolve through niche construction if the organism can monopolize its niche the way a germ cell (egg or sperm) monopolizes resource-harvesting by soma (tissue cell). A body is nothing other than a constructed niche with a guaranteed closed feedback loop. Bodies are the environment ruled over by a germinal despot. 

Cortical Chopsticks

Spiders and their webs, like birds and their nests, and bees and their hives, have evolved elaborate extended phenotypes which go some way to dissolving the boundary of organism and environment. The skills required to manipulate their complete exbodiment could be said to be largely innate.

Humans on the other hand need to learn to use the tools they build. Chopin did not instinctively exude his Bauzer piano postpartum and then play it as naturally as wiggle his fingers. The cognitive and neural implications of this fact are intriguing. The ability to construct a physical artifact is largely dissociated mentally from its effective applications. This means that the neural encoding of the exbody needs to learn efficiencies approximating those observed in, for example, the encoding of sound in the inner year, or smell in the olfactory bulb, or scenes in the visual cortex. This goes beyond constructed niches which modulate local physics, toward modifying through representation, the whole adaptive plan of the organism.

For example, what might we expect to happen as we develop expertise with a common tool? Does the tool slowly morph into an embodiment – an extension and amplification of our limbs – and if so – how does the brain encode such a process? Sawamura and colleagues sought to answer this question by tracking the acquisition of chopstick expertise in subjects forced to use their non-dominant hand. They found that as trial subjects practised over a six-week period, concomitant with skill acquisition, brain activity shifted from prefrontal cortex to premotor cortex. A pattern that is often interpreted as a shift from intentional and analytical solutions to motor programming more typical of limb motion. Expertise on this task would seem to transform the tool from an object of analysis to a part of the body. 

An earlier paper by Kitamura and colleagues on virtual chopsticks, which could more easily be manipulated into unconventional motion, reached essentially similar conclusions. Which suggests that our understanding of physical objects can also be understood in terms of kinematics – or spatial degrees of freedom – that live outside the confines of the body, physical or otherwise.

Mental Tapes

The first recordings of Chopin's piano music can be traced to cylindrical phonograph recordings made in 1919 by Alfred Cortot. Most likely recorded on a Steinway piano whose bombastic tones could not be further from the harp-like innocence of the Bauza.

Computer memory did not make use of ferric oxide audio tape until its use in the UNIVAC computer in 1951. Prior to UNIVAC computer memory was non-persistent and stored either in arrays of vacuum tubes or in the travelling waves of mercury display lines.

It is fair to assume that in 1936 when Alan Turing wrote his classic paper “On Computable Numbers, with an Application to the Entscheidungsproblem,” that is with application to the decision problem, Turing had never used a tape recorder and most certainly not a computer. The first university computer was the ENIAC completed at the University of Pennsylvania in 1945.

Turing's 1936 paper is a pessimistic answer to a question posed by the optimistic mathematician David Hilbert in 1928. Hilbert asked, and firmly believed any answer to be affirmative, whether there was a universal method, or mechanical process, that might demonstrate that any mathematical assertion is true or false. Turing showed that this question was in the most general sense unanswerable, that is undecidable.

The way that Turing solved the Hilbert problem is beautifully physical and not at all like the abstractions typically associated with mathematical proof. In order to demonstrate undecidability Turing invented a fictitious computing machine that calculates by modifying persistent marks on a tape – Turing internalized exbodiment. With his rare intelligence Turing uses a physically impossible infinite model inspired by a finite physical device to refute an infinite proposition.

The key elements of the universal Turing machine are the infinite tape which includes its programs, the data upon which it will operate; a tape head that reads and writes to the tape according to its program, and a halting criterion which ends the programs with a true or false statement. Turing proved that there are problems that can be presented in the data for which the machine will never halt. These problems are formally undecidable – they have no computable solutions.

Unlike the web and the chopstick that through evolution or learning could become an exbodied part of a spider or a human, both of which extend beyond the limits of the physical, the Turing tape is a reabsorbed physical device. And by virtue of its abstraction, it can be extended beyond physical limits into the spooky expanse of the infinite. The Turing tape is the perplexing imagination of a mental exbodiment. And it is one that serves to illustrate both the flexibility and limits of our reason.

The Exbodiment Helix

Chopin might have composed in his head but he performed on a piano. The physical artifact remains an important part of what Descartes called the world of extension. Turing invented a tool that he could then devour with his mind. He employed what I call the exbodiment helix in order to support a mental simulacrum of a physical object.

The individual use of an existing tool – Chopin's Piano – and the mental representation of a physical object – Turing's tape – are not categorically distinct events but points on an exbodiment helix. The helix captures how over the course of time individuals internalize knowledge to mind and then outsource expertise to matter. The radius of the helix is increasing through both the growth of material tools and technologies and in mental models and theories. It is not a question of mind versus matter but the mind made matter and matter made mind.

Most of us are familiar with the game of chess less in terms of the concept and more in terms of the physical board and pieces. An eight-by-eight grid of alternating black and white cells half populated by 32 pieces of varying degrees of value.

Great chess players to include master players can play chess with lesser mortals when blindfolded. The blindfolded player does not play on a cortical virtual-reality board but something vastly more abstracted – a kind of gestalt space-time system that captures constellations and patterns rather than the shapes and coordinates of individual pieces. This was partially illustrated in experiments by Saariluoma and Kalakoski when they replaced pieces with simple dots and then asked players to play with no evident loss of accuracy.

An obvious fact of blindfolded play is that it can only be achieved after years of sighted play. The physical board needs to exist as a training scaffold first in order to create the long-term memory structures that short-term memory exploits blindfolded.

And as the great historian of chess, H.J.R. Murray described, the game itself has evolved. And in such a way as to build a more effective scaffold. Starting with the elimination of all random elements (chess started as a die-based game), imposing the alternating black and white squares (chess started with monochromatic squares), and imposing an upper bound on play through the three-fold repetition rule and 50 move rule (to save on working memory). 

Expertise in chess moves up through the exbodiment helix with individual skill and cultural mnemonics defining an increasingly efficient path toward mastery.

The same helical relationship is observed in master abacus users. The abacus is a physical calculation device in which the position of beads on vertical rods denotes numbers through a positional value system, 1s, 10s, 100s and so forth. Just as in decimal notation. Calculations are carried out within the frame by vertically displacing a number of beads into each of these order of magnitude slots.

Just as with chess, after years of abacus experience, experts can visualize an abacus blindfolded. And use the mental abacus to complete calculations at a rate unimpeded by movement or friction. In a series of intriguing experiments on the abacus, expertise has been shown to be associated (as with the chopstick) in the shifting of neural activity from prefrontal cortex and verbal working memory towards visual-spatial working memory.

And the abacus provides a beautiful illustration of the workings of the exbodiment helix. Since visual perception shows a bias in amplitude corresponding to the cardinal orientations – the oblique effect – the abacus is designed with clear horizontal and vertical elements. Try to use an abacus when it is rotated away from the horizontal (even if the pieces can be secured!).

What we learn from by studying the history of the creative imagination is that the individual mind lives within a collective intelligence largely expressed through material objects. Whether we are talking of calculators and compasses, instruments and maps, or books and puzzles, we individually absorb well designed functional schema from matter, and occasionally give back our own incremental representations to the ambient culture. A world out of balance, and destined to reach a standstill in its helical progress, is one that imagines it might exist entirely in the ether.