Image and Inference

N.B. The thing ^ is Priit Parmakson's adaptation -- reconfiguration -- of one of my images.

James Franklin, "Diagrammatic Reasoning and Modelling in the Imagination: the Secret Weapons of the Scientific Revolution," in, G. Freeland & A. Corones, eds., 1543 and All That: Image and Word, Change and Continuity in the Proto-Scientific Revolution (Dordrecht, 1999) (footnotes omitted):

"One must consider the affection which is produced in the soul, and in that part of the body which contains the soul -- the affection, the lasting state of which we call memory -- as a kind of picture" : Aristotle 145 "There are no such things as mental pictures" : Ryle 146 We live at the end of a period which, perhaps more than any other, has hidden the pictorial life of the mind from intellectual view. Philosophy in the mid-[20th] century regarded "sense data" as fictions, arrived at by (bad) inference and suitable for disposal as an undergraduate exercise. It was seriously maintained that all inner representation was propositional. (That is philosophy in the English-speaking tradition; continental philosophy being, if anything, even more word-oriented). Psychology in the behaviourist decades was not much concerned with any inner life, pictorial or not, and "imagery" tended to be associated with the Gestalt school, labelled "unscientific". Introspection, despite being reproducible with reasonable consistency, was ruled out as a source of experimental data. Frege, Russell and Hilbert, followed by Turing and the computer scientists and Artificial Intelligentsia, imposed on the learned world a view of inference as the manipulation of uninterpreted symbols according to formal rules. No room for pictures there. Even mathematics, once centred on geometry, maintained hardly any formal role for pictures, holding that geometrical intuition is unreliable. Galton was surprised to find even at the end of the nineteenth century that scientists were claiming to think in symbols, not images, and supposed that science had atrophied the imagination. It is true that there has been a kind of tradition in the physical sciences of remarks on the need for mental geometrical intuition, but it is one of those "traditions" that presents itself as a novelty every time it is reinvented. At about the time Ryle was denying the reality of mental pictures, Einstein was saying:

The words or the language, as they are written or spoken, do not seem to play any role in my mechanism of thought. The psychical entities which seem to serve as elements in thought are certain signs and more or less clear images which can be "voluntarily" reproduced and combined . . . The above mentioned elements are, in my case, of visual and some of muscular type. Conventional words or other signs have to be sought for laboriously only in a secondary stage, when the mentioned associative play is sufficiently established and can be reproduced at will.

Generally, though, the image was left to intellectual marginals: diagrammatic inference to the engineers, with their slide rules, flow charts and circuit diagrams, and mental images to the Freudians and their dream fantasies.

The result is that the late-millennial intellectual has several layers of defence against accepting the medieval unselfconsciousness about using the imagination as a tool for doing science. First, we doubt the existence of the inner life at all. Or, we regard it as reached only by a chain of inference, not open to view. Or, we regard it as vague, and primarily emotional, not precise and scientific. Or, we presume the "stream of consciousness", if there is one, is a flow of words, the preserve of psychiatrists and novelists. Or, if we do take the visual side of the imagination seriously, we think of it as something like a muse, "inspiring" artists and poets in a manner too sublime to analyse. We have every excuse for misunderstanding.

At the leading edge of science, this picture is no longer true. Two developments especially have made the difference: psychological experiments on mental images, and scientific visualization by computer. The psychological work is especially relevant. Around 1970, Shepard and Metzler found that the time subjects took to decide whether one 3D figure could be rotated to fit in the same space as another was proportional to the angle required, suggesting that the subjects were actually performing the rotation in some kind of mental space. Since then, similar techniques have been used to investigate how subjects imagine themselves in an environment while mentally searching it, and how they construct a mental model from a description of a scene. It is found generally that reasoning about space is done by means of mental models, not via chains of propositions. In view of what was said above about the importance of reasoning in three dimensions, it is significant that 3D imagining is found to be remarkably powerful: one can, for example, mentally scan across an imagined space from any viewpoint: the space is mentally encoded in 3D, but can be "displayed" from any viewpoint in 2D.

The other development that has given images scientific respectability is the very recent availability of computer-intensive visualisation tools, used for such purposes as understanding complex fluid flows. The images are not themselves mental, but of course the point of computer visualization is to create pictures that lead to understanding, that is, cause suitable mental representations. To some extent, the two themes are connected by research in education, which finds that mental visualization is a necessary skill for learning in the sciences. Anyone with some sympathy for the idea that ontogeny recapitulates phylogeny will be ready to look for some parallel process in the history of science.

These developments are yet to flow through into general intellectual consciousness, but progress has been sufficient to assert, at least, that Aristotle was more right than Ryle, and that there are no longer any barriers in principle to taking literally what the medievals were saying about mental images.