The Tangle Test For Intelligence
Yet more varieties of spatial reasoning
(DRAFT: Liable to change)
Aaron Sloman
http://www.cs.bham.ac.uk/~axs/
School of Computer Science, University of Birmingham
(Philosopher trying to disentangle Computer Science)
Installed: 19 Aug 2019
Last updated:
This is http://www.cs.bham.ac.uk/research/projects/cogaff/misc/tangle-test.html. A PDF version may be added later. Some background is below. This is one of many examples of spatial/mathematical intelligence/consciousness on this web site.
Tangle test for mathematical/spatial intelligence/consciousness
This paper elaborates on a robot challenge posted on twitter as "A tangle test for intelligence" on 9th Aug 2019: https://twitter.com/aaronsloman/status/1159849248931889152. (The Chewing test for intelligence posted in 2014: http://www.cs.bham.ac.uk/research/projects/cogaff/misc/chewing-test.html was more "tongue in cheek".)Figure Tangle, below, shows a computer cable, with one end connected to a floor socket and most of the loosely tangled cable resting on a horizontal surface with the other end free. I noticed it while listening to a presentation on the use of "deep learning" to characterise visible (2D) structures in a medical image on the basis of samples of the image. Nothing was said about characterising the (3D) brain structures from which the image was derived (by a projection process).
Thinking about the problems not addressed in the talk (also not addressed in other recent talks and papers on vision and image understanding in AI) led me to reflect on what information I could extract from the messy visual data available as I looked at the cable shown in the figure. The resulting challenge on twitter was to get a one-armed mobile robot with a gripper and a video camera to work out how to straighten out such a cable, without actually doing it. What visual and cognitive competences would it need, and how could they be implemented? This is a first draft partial analysis of the challenge, which has deep connections with Immanuel Kant's philosophy of mathematics.
For example, in the above picture, two portions of the cable near the lower end are obscured by the table, leaving open the possibility that there is not one continuous cable. In "real life" a simple movement to a different viewing location can eliminate that possibility (assuming bits of matter do not rearrange themselves when out of sight).
[Some video examples showing multiple effects of camera motion in different directions are available here: http://www.cs.bham.ac.uk/research/projects/cogaff/movies/chairs. Explaining what's going on in the mind or brain of a viewer of those movies, including explaining how multiple concurrent changes in different parts of the image are combined as evidence, is left as a task for the reader -- or a future generation of researchers.]
Informal mathematical reasoning about affordances
In my visual experience of the cable in the above photograph, and presumably yours, there is rich semantic content going far beyond 2D image structure and the 3D visible surface structures in the scene. Your visual system combines information about a large disjoint collection of visible portions of the cable, some straight or nearly straight, others curved, including forming loops (where one part of the cable crosses another part) and a tangled network of spatial relationships, including nearer, further, going behind, changes of curvature, and many more. It is important to separate two questions:An important feature of this sort of scene is that there are many locations where partial occlusion produces clues as to which of two portions of the cable is further from the viewer, and approximate alignment of occluded edges provides clues regarding continuity, although not all clues are totally unambiguous. Understanding those relationships could be part of the answer to this additional question:
1. How can a brain represent a relation of "connectedness" that can include arbitrarily many components, in a potentially unlimited variety of spatial relationships, and which can be used not only in judgements ("these visible fragments are connected") but also in questions that can reasonably be asked in situations where current perceptual contents do not provide an answer, e.g. "are these visible fragments connected?".2. How can a brain discover, or have good reason to suspect, that a particular collection of perceived fragments are parts of a connected whole?
3. How can a brain work out which changes of viewing direction, or view-center, and which changes of head motion (including movements to a new viewing location) could provide useful disambiguating information?
At first I thought it was obvious that there was only one continuous cable because I could see only two ends, one under the table and another lying on the table, until I noticed that there is another possibility, namely that parts of the scene include one or more "cable rings" with no ends, also lying folded on the table. There is a way to check that that is not the case by visually following the 3D route of the cable between the two visible ends, and making sure that that includes all the visible portions of the cable, but ensuring that there are no loops requires a process that takes time: it is not immediately obvious in a scene as complex as this, especially when not all portions of the cable are simultaneously visible.
Such reasoning is inherently mathematical -- concerned not with statistical evidence or probabilities, but with what is possible, impossible or necessarily the case in relation to spatial structures and processes. I think Kant recognized that this is part of ordinary human intelligence, closely related to mathematical intelligence of the kind required for ancient mathematical discoveries presented in Euclid's Elements, and other discoveries, e.g. the discovery that the neusis construction, available in Euclidean geometry, which can be used to trisect any angle, which is not possible for most angles in Euclid's geometry Sloman(trisect). The impossibility is not easy to prove, and was first proved using the cartesian coordinate representation of geometry and the theory of prime numbers!
Following the route of the cable to determine that there is only one cable is not easy to do with a normal human short term spatial memory, though, in principle, it would be possible to paint the cable, following all the turns and under-passes, to ensure that the route from one end along the cable to the other end does not leave out any visible portion of cable (i.e. an unpainted portion of cable).
This may not be possible with a photograph, where following the cable route in a densely packed portion of the scene can be difficult. However, someone in the same room, viewing the cable itself can use changes of viewpoint (i.e. moving your head) to disambiguate the connectivity, which doesn't always work with a 2D image. (Why not?) A further option, in the "real life" situation is to use fingers to follow the route of the cable temporarily separating parts if necessary.
That is a piece of topological reasoning that I expect many, perhaps all, viewers of this text can follow, though they probably could not have followed it when two years old, which raises questions about what happened to their brains between then and now. (Compare Sauvy and Sauvy (1974).)
Moreover, I later realised, after a little thought that in principle a mobile robot with a visual system like mine and one arm holding a gripper could disentangle the cable, using a succession of grasp, move, release actions, if it is a single cable, and later posted the note on "A tangle test for intelligence" to twitter. Part of the challenge is working out a strategy for grasping and moving that is guaranteed to work if there is only a single cable with no tight knots.
Working out a strategy and understanding why it works requires mathematical intelligence that is part of ordinary spatial intelligence, required for practical tasks that can be encountered in many different contexts. (Not all humans develop their spatial intelligence to the same extent, so some people may find this example hard to follow.)
Further thought revealed that there are some interesting questions about how I knew that the problem was solvable, and I realised that it is yet another example that can be shown as illustrating Kant's view of mathematical discovery.
Background
This is part of two large overlapping projects: (a) CogAff (Cognition and Affect) project http://www.cs.bham.ac.uk/research/projects/cogaff/ investigating information-processing architectures supporting forms of information processing in biological organisms and possible future intelligent machines; including machines with affective states and processes such as motives, preferences, attitudes, interests, emotions, moods, obsessions, hopes, fears, values, ideals, long term ambitions, long term grief (which contradicts most published theories of emotion), and (b) the Turing-inspired Meta-Morphogenesis project http://www.cs.bham.ac.uk/research/projects/cogaff/misc/meta-morphogenesis.html, attempting to study changes in uses of information, information formats and media, and information-processing mechanisms and architectures -- between the very earliest (proto?) organisms and current and future possible organisms -- of all shapes, sizes, habitats, behaviours, etc. Transitions in types of information processing produced by biological evolution, and by individual or group development, form a complex branching network of mechanisms and functions, not a linear sequence.The project's aims include investigation of actual and possible products of actual and possible designs for information-processing functions and mechanisms in an enormous variety of actual and possible types of organism. Despite that variety we can't assume biological evolution and human science and engineering have already revealed all (or even most?) of the types of information, types of information use, and types of mechanism that evolution can produce. The diversity is in part a product of the fact that biological evolution, individual development and social/cultural evolution can all, in different ways, produce new construction kits for building information processing mechanisms, as discussed here: http://www.cs.bham.ac.uk/research/projects/cogaff/misc/construction-kits.html. Each new evolved or engineered construction-kit, can contribute to the variety of organisms or machines, in combination with other construction kits.
Note on "information": I use "information" in the sense of the 19th century novelist Jane Austen (i.e. usable semantic information content), not in the sense of 20th century mathematician/engineer/scientist Claude Shannon, as explained in http://www.cs.bham.ac.uk/research/projects/cogaff/misc/austen-info.html. Shannon's concept of "information" refers to mathematical properties of a subset of information-content bearers, e.g. strings of symbols, and is only of secondary importance in this project.
Evolved mathematicians
This is part of the Cogaff (Cognition and Affect) Project http://www.cs.bham.ac.uk/research/projects/cogaff/, which includes an attempt (originally begun in my 1962 DPhil thesis) to understand how biological evolution could have produced ancient human minds able to make mathematical discoveries in geometry and topology, and other animal minds also able to make and use related discoveries, and use them in spatial reasoning, but without the reflective and communicative abilities required for mathematical thought and discussion. I discovered that Immanuel Kant's view of mathematical discoveries as non-empirical, non-analytic (not based solely on logical derivations from definitions), and non-contingent (i.e. concerned with what's impossible, or necessarily the case) was closer than any other philosophy of mathematics to my experience of doing mathematics, especially solving problems in geometry and topology.I learnt about AI several years after completing the thesis and hoped that I could use AI to demonstrate Kant's ideas about mathematical discovery (by working out how to design a baby robot that could grow up to be a mathematician making discoveries about mathematical impossibilities and necessities non-empirically, in something like the manner hinted at by Kant -- mostly through his examples.
I have not yet found a way to do that, and neither can anyone else, as far as I know, explain or model ancient mathematical capabilities and corresponding capabilities in pre-verbal toddlers and other intelligent animals.
I am now exploring (tentatively) the idea that it cannot be done on digital computers (able to make only discrete changes in its forms of representation) but could be done on a suitably designed, possibly sub-neural, chemistry-based machine, since one of the features of quantum physics is that it explains how chemistry can support both discrete and continuous changes in structures and relationships for reasons closely related to Schrödinger's discussion in his (1944). Newtonian physics cannot explain such phenomena, since it requires everything to be explained in terms of interacting point-masses with inertial properties and gravitational attraction, but no means of forming interacting objects like gear wheels, nuts and bolts, pulleys, levers, springs, etc. Without chemical bonds based on quantum effects not even a lever could exist, let alone a digital computer.
This raises questions about the space of possible computations that might be implemented on a Turing-like machine that is not restricted to a discrete tape, discrete symbols, and discrete state transitions. I suspect Turing was thinking about that problem when he wrote his (1952) paper on chemistry-based morphogenesis. This document merely discusses an example, without reaching any conclusions. It is one of a large collection of work-in-progress documents on my web site, including http://www.cs.bham.ac.uk/research/projects/cogaff/misc/impossible.html (also PDF).
REFERENCES AND LINKS
Erwin Schrödinger (1944) What is life? CUP, Cambridge,I have an annotated version of part of this book here
http://www.cs.bham.ac.uk/research/projects/cogaff/misc/schrodinger-life.html
Aaron Sloman (1962 -- digitised 2016),
Knowing and Understanding: Relations between meaning and truth, meaning and
necessary truth, meaning and synthetic necessary truth
DPhil Thesis, University of Oxford, Bodleian Library. Now online here:
http://www.cs.bham.ac.uk/research/projects/cogaff/sloman-1962
The Meta-Morphogenesis project (2011-...)
http://www.cs.bham.ac.uk/research/projects/cogaff/misc/meta-morphogenesis.html
A. M. Turing, (1952),
'The Chemical Basis Of Morphogenesis', in
Phil. Trans. R. Soc. London B 237, 237, pp. 37--72.
(Also reprinted(with commentaries) in
S. B. Cooper and J. van Leeuwen, EDs (2013)).
A useful summary for non-mathematicians is
Philip Ball, 2015,
Forging patterns and making waves from biology to geology:
a commentary on Turing (1952) `The chemical basis of morphogenesis',
Royal Society Philosophical Transactions B,
http://dx.doi.org/10.1098/rstb.2014.0218
Tom McClelland, (2019) The Mental Affordance Hypothesis, Mind, 2019, https://doi.org/10.1093/mind/fzz036
J. Sauvy and S. Sauvy, (1974)
The Child's Discovery of Space: From hopscotch to mazes -- an introduction to intuitive topology,
Penguin Education, 1974, Translated from the French by Pam Wells,
http://www.amazon.co.uk/The-Childs-Discovery-Space-Hopscotch/dp/014080384X
Aaron Sloman Trisect (2015), How to trisect an angle (Using P-Geometry). http://www.cs.bham.ac.uk/research/projects/cogaff/misc/trisect.html
A partial (possibly out of date) index of discussion notes in this directory is in
http://www.cs.bham.ac.uk/research/projects/cogaff/misc/AREADME.html
This work, and everything else on my website, is licensed under a
Creative
Commons Attribution 4.0 License.
If you use or comment on my ideas please include a URL if possible, so
that readers can see the original, or the latest version.
Creative Commons License
Maintained by
Aaron Sloman
School of Computer Science
The University of Birmingham