Videos involving plant matter

Garden-process qualia

Or more generally: complex fast-changing qualia

(To be expanded)

HISTORY

This document
Installed: 17 Oct 2015
Updated:19 Oct 2015; 8 Dec 2015; 9 Dec 2015; 24 Dec 2015; 18 Feb 2016

This file is accessible at:
http://www.cs.bham.ac.uk/research/projects/cogaff/misc/vision/plants/
This is one of many discussion documents on the same web site identifying problems about biological vision (not only in humans) that I think have not been considered by most researchers in vision, especially researchers in machine vision, who set themselves targets that omit many of the features of biological vision systems, including the role of vision in making discoveries in geometry, topology and other areas of mathematics. Even blind mathematicians may be using brain mechanisms that evolved as parts of visual systems in their ancestors.

I'll start with the problem of seeing what's going on in videos of a garden. However, similar problems are presented by TV advertisements where there are often complete scene changes every second or less, nowadays. Advertisers make assumptions about human visual processing capabilities that I think have gone unnoticed by most vision researchers (in psychology, neuroscience, AI, robotics).

NOTE: (Added: 24 Dec 2015)
There are some more detailed challenges related to static images here:
http://www.cs.bham.ac.uk/research/projects/cogaff/misc/vision


VIDEOS OF A GARDEN TAKEN FROM CHANGING VIEWPOINTS

A not very high quality video camera was moved around in a Birmingham garden. Several short video recordings were made.

CHALLENGES FOR VISION RESEARCHERS

Consider what goes on in you

(a) when you look at these videos
(b) when you move around a garden like the one shown
What would justify a claim that something similar is going on in a machine?

While looking at these videos consider the question:

What would have to happen for a machine (e.g. a robot) to experience the videos as you do (under various conditions, e.g. with one eye or two, viewing on a small sized or a large screen, etc.).

How could you demonstrate that this was happening when the robot viewed the videos, or when it moved round this garden following the trajectory and gaze directions of the camera used for these recordings?

VIDEO 1: Two versions of the same video: MP4 and WEBM
(I hope all browsers can cope with at least one of them.
Both work with vlc on my linux machine.)
(Apologies for sneeze in the middle)

http://www.cs.bham.ac.uk/research/projects/cogaff/movies/garden-sep-2015.mp4 (36MB)
http://www.cs.bham.ac.uk/research/projects/cogaff/movies/garden-sep-2015.webm (27MB)

VIDEO 2: Two versions of the same video: MP4 and WEBM

http://www.cs.bham.ac.uk/research/projects/cogaff/movies/garden1.mp4 41MB
http://www.cs.bham.ac.uk/research/projects/cogaff/movies/garden1.webm 27MB

VIDEO 3: Two versions of the same video: MP4 and WEBM

http://www.cs.bham.ac.uk/research/projects/cogaff/movies/garden2.mp4 42MB
http://www.cs.bham.ac.uk/research/projects/cogaff/movies/garden2.webm 26MB

VIDEO 4: Two versions of the same video: MP4 and WEBM

http://www.cs.bham.ac.uk/research/projects/cogaff/movies/garden3.mp4 70MB
http://www.cs.bham.ac.uk/research/projects/cogaff/movies/garden3.webm 49MB

VIDEO 5: WEBM video (added 1 Sep 2017)
http://www.cs.bham.ac.uk/research/projects/cogaff/movies/garden5.webm 9.2MB

What should an AI system be able to do to demonstrate that it has seen these videos as a human can?

DON'T ANSWER THUS: "It should build a changing 3-D model of the scene including the camera trajectory and be able to use the information to show a different video of the same scene for the same time period but with the camera moving through a different spatial trajectory -- a fly-through video."
That's typically how AI vision researchers test the 3-D stereo capabilities of their vision systems.

HOWEVER
MY BRAIN CAN'T DO THAT, SO WHY SHOULD AN INTELLIGENT MACHINE BE ABLE TO?

What does a human brain do with visual information? There has been a vast amount of research on human and animal vision. Most laboratory research either investigates failures or limitations, or very narrowly focused competences.

Finding out what a nest building bird, or a squirrel uses vision for, or how a pre-verbal toddler uses vision to make discoveries in 3-D topology is much harder:
http://www.cs.bham.ac.uk/research/projects/cogaff/misc/toddler-theorems.html#pencil
I don't think anyone has a comprehensive answer -- though many fragments of human and animal visual competences have been documented.

Related research
Some of the issues are addressed in this very short survey paper:
https://www.staff.ncl.ac.uk/alex.thiele/29.pdf
Alexander Thiele, "Vision: A Brake on the Speed of Sight"
Current Biology, Vol 15 No 22 (2005)

We move our eyes more often than our heart beats. Our brain seems to cope effortlessly with the consequences of these rapid visual alterations, but a new study shows that similar scene changes in the absence of eye movements delay the speed of information processing. So are there costs in constantly shifting our focus of gaze?

Part of the answer that is usually omitted is: we make mathematical discoveries. For more on that take a look at this web site, after looking at the videos and questions above.
http://www.cs.bham.ac.uk/research/projects/cogaff/misc/impossible.html
     Some (Possibly) New Considerations Regarding Impossible Objects
     Their significance for mathematical cognition, and current serious
     limitations of AI vision systems.

Additional evidence is presented in this video
http://www.cs.bham.ac.uk/research/projects/cogaff/movies/ijcai-17
Linked to this workshop paper
http://www.cs.bham.ac.uk/research/projects/cogaff/misc/ijcai-2017-cog.html
(Also PDF)

"Fact is richer than diction".

J.L. Austin in Philosophical Papers (OUP, 1961), page 143.
Originally in "A Plea for Excuses", Proceedings of the Aristotelian Society, 1956-7

The garden videos seem to illustrate Austin's dictum.

We could generalise what he wrote, to: "Seeing is richer than doing".

Another challenge to human and machine vision, drawn to my attention by Aviv Keren, involves flame-breathers: https://vimeo.com/132024990

The apparatus required to produce that video, shown here https://vimeo.com/129062369 was far more complex (and costly) than my cheap little camera, used in the garden videos.

THE REALLY HARD PROBLEMS OF CONSCIOUSNESS

Most people think they know what it is to be conscious as opposed to being unconscious, and what it is like to have various kinds of consciousness.

There's much self deception in this

Compare Gilbert Ryle on the "polymorphism" of many concepts. We can expand his idea to "parametric polymorphism" an important idea that's more widely used in computer science than in philosophy.

NOTE ON A PAPER IN JCS 2003

In 2003 Ron Chrisley and I published a paper that answered some of the questions about how a machine can have qualia for which its internal references use a "private" language.
http://www.cs.bham.ac.uk/research/projects/cogaff/03.html#200302
A. Sloman and R.L. Chrisley (2003), Virtual machines and consciousness,
Journal of Consciousness Studies, 10, 4-5, pp. 113--172.
Some of the ideas are developed in the context of Virtual Machine Functionalism:
http://www.cs.bham.ac.uk/research/projects/cogaff/misc/vm-functionalism.html

In 2012 Marcel Kvassay wrote "Machines, Intelligence, Consciousness", a tutorial overview of the paper comparing our theories with those of David Chalmers and others:
http://marcelkvassay.net/machines.php (HTML)
http://marcelkvassay.net/pdf/machines.pdf (PDF)

Most philosophers are ignorant about virtual machinery, unfortunately. Working out how to extend current virtual machines to support human and animal visual consciousness is a hard research problem. Part of the problem is to specify the requirements. This web site makes a small contribution.

NOTE:
The 2003 paper claimed "consciousness" is a cluster concept'. I now think that's a mistake. Gilbert Ryle (in The Concept of Mind 1949) came closer to the truth in use the notion of a polymorphous concept. Since then the importance of parametric polymorphism has emerged in the context of programming language design (for example) and that captures more accurately the diversity of usages of concepts like "conscious", "efficient", "better", and many more. See also
http://www.cs.bham.ac.uk/research/projects/cogaff/misc/family-resemblance-vs-polymorphism.html

A paper on evolution of consciousness is in preparation here:
http://www.cs.bham.ac.uk/research/projects/cogaff/misc/grounding-consciousness.html
Dynamic metaphysical grounding of consciousness in evolution.

QUESTIONS THAT COULD BE ASKED ABOUT EACH VIDEO
How much 3-D structure can you see?

  1. None ?
  2. A Little ?
  3. A Lot ?

Does it make any difference whether you look at the video with your left eye, your right eye, or both eyes?

If you watched the video in its natural size (width 640 pixels) try again with a full-screen view, and vice versa. Are your answers to the questions different for the natural and full-screen views. (If you are viewing on a tablet or mobile phone this distinction will not make any sense.)

Please email answers to a.sloman AT cs.bham.ac.uk

There are some snapshots from one of the videos, with questions for the viewer, in a separate file here:
http://www.cs.bham.ac.uk/research/projects/cogaff/movies/garden-static.html

Comments to the maintainer:
Aaron Sloman
EMAIL: a.sloman @ cs.bham.ac.uk
Thanks.
http://www.cs.bham.ac.uk/~axs
Related web pages:
http://www.cs.bham.ac.uk/research/projects/cogaff/misc/vision