Date:Mon, 8 Dec 1997 10:33:33 +0000 
Original message: here.
Reply-To: "PSYCHE Discussion Forum (Biological/Psychological emphasis)"
Sender:    "PSYCHE Discussion Forum (Biological/Psychological emphasis)"
From:         Aaron Sloman < A.Sloman@cs.bham.ac.uk>
Subject:      More on consistency and consciousness: reasons for view B

To recapitulate:

View A is that the contents of consciousness must be internally
consistent because the brain uses some mechanism for enforcing this.

View B is that sometimes the contents of consciousness are consistent
and sometimes they are not: i.e. internal consistency is not an
*absolute* requirement for the contents of consciousness, but may often
be a consequence of other requirements.

I said there were three types of reasons for preferring B:

1. No known type of mechanism could implement the general consistency
   constraint.

2. Empirically there are counter examples.

3. There are more biologically plausible and biologically useful,
   mechanisms whose side-effects would include achieving consistency
   most of the time, without totally ruling out inconsistency.

I'll now elaborate on each of these in turn.

1. No known mechanisms could do it.

People who have worked on the general problem of inconsistency
detection and eradication know that it is inherently intractable, for
the reasons indicated in an earlier message: i.e. in the most general
case it is undecidable (Go"del's incompleteness theorem, etc), and even
decidable cases (e.g. propositional logic) are combinatorially
explosive, and therefore intractable in general.

Even if the brain does not use propositional or predicate logic it seems
(in the case of humans) to have combinatorial capabilities both as
regards the variety of 2-D images that can be "parsed" and the variety
of 3-D percepts that can be formed. Motion merely adds to the
combinatorial complexity.

Anyone who doesn't believe that inconsistency detection is intractable
in general, should try to design in information processor which can tell
whether an arbitrary collection of propositions is consistent or not, or
an arbitrary representation of a visual scene, including all variants of
Penrose/Escher figures that can be taken in as a whole. Finding a
non-exponentially explosive (in space or time) way to check consistency
of an arbitrary propositional expression could bring fame and fortune
and answer one of the open questions in the theory of computation.

Some people may think that parallelism in neural nets can defeat the
combinatorics, but although I have not thought it through, I suspect
there is probably a theorem about limitations of perceptron-like neural
nets on this sort of task, analogous to the Minsky/Papert results
regarding parity and connectedness: such essentially global properties
require serial mechanisms (or enough perceptron layers to simulate
serial processing).

Marvin, is that right? (Perhaps I am out of my depth here.)

Vision has to work fast, at least if you are a bird, squirrel, chimp,
tennis player, car driver, etc. So an engine that sits chugging away
looking for ever longer sequences of inferences till all possibilities
are exhausted is unlikely to work for the purposes of vision.

If there are at most N layers of consistency checking in the visual
networks preceding the stage of visual processing which feeds into the
contents of conscious experience (whatever that means, but forget that
for now), then perhaps a class of inconsistencies detectable in N steps
could be eliminated, but not others. We seem to be able to detect 2-step
and 3-step visual inconsistencies fairly easily, at least as adults.

(What about other animals and young children?)

(Maybe quantum-gravity computers can do better than human brains??
Is Penrose listening?)

2. There is empirical evidence that View A is false.

I have presented a number of cases of different sorts in previous
messages, saved in the file

    http://www.cs.bham.ac.uk/~axs/misc/binocular-rivalry

One class of counter-examples discussed previously involves examples of
perception of "long range" Penrose/Escher figures where detecting the
inconsistency requires more than two or three steps of transitive
inference, e.g. a Penrose decagon. It's easy to see such a complex
figure as locally consistent everywhere without noticing that if you
perform an inference chain involving as many steps as the sides of the
polygon that it leads to an inconsistency: X is nearer than Y and X is
further than Y.

As Stan Klein and Irvin Rock (among others) admit, pictures of
impossible objects can be perceived as impossible objects. (I've seen an
advertisement in which the typical line drawing is replaced by a superb
coloured, textured picture of a Penrose/Escher object, which is easily
experienced as a realistic photograph of a globally impossible object.)

Even in the case of the simple Penrose triangle I think I have no
problem experiencing it as a complete but incoherent 3-D object,
likewise the devil's pitchfork etc. That's why those pictures are fun:
they are not like the necker figure which (normally) flips between two
consistent interpretations.

My binocular view of two pencils passing through the same bit of space
is inconsistent in a different way. Likewise the motion after-effect
described previously, in which motion is observed but nothing changes
its location.

Bernie previously reacted to some of this by asking whether there is
evidence that two (differently shaped?) objects can be seen in exactly
the same place. I previously tried to indicate that on one
interpretation of the question this is possible (a cube and a sphere of
the same diameter embedded in each other), and on another interpretation
it is impossible because the question is then incoherent: A cube cannot
occupy the same volume of space as a sphere because if it did it would
be a sphere, not a cube.

Likewise: one reason it is impossible to see the necker cube
simultaneously in both of the "standard" ways is not because the brain
has a mechanism imposing a consistency constraint but because it is
logically impossible and therefore no mechanism is needed to rule it
out.

That's because seeing both interpretations in the same place at the same
time would require seeing a particular edge as both sloping further away
to the right and sloping nearer to the right, and that would require the
existence of neighbouring bits of the edge, e1 and e2, such that e1 is
nearer than e2 and also further than e2. I.e. the very description is
incoherent, if the two views are supposed to share edges in 3-D. It's
really just the same point as the earlier point that if a cube were in
exactly the same space as a sphere it would not be a cube. Likewise if
an edge sloping away to the right were seen in exactly the same location
as an edge sloping nearer to the right, then it could not be an edge
sloping nearer. If they are in the same place they have the same slope.

Well then, is it possible to see the necker drawing as depicting not two
differently oriented cubes in exactly the same place, but as two
wire-frame cubes with corrresponding edges sloping differently but
*completely aligned*, i.e. either with edges passing through each other
or one nearby and one further away, one cube flipped one way and the
other cube flipped the other way, while the edges of BOTH project onto
the same lines on a 2-D image plane?

On that intepretation the cubes are not sharing all their edges, but the
edges of the two cubes are paired so that they cannot be visibly
distinguished.

That experience would depend on the ability simultaneously to experience
a 2-D line, e.g. this: _____________________________,  as two coexisting
3-D lines, one sloping away to the right and one sloping away to the
left and both projecting to the same 2-D line. Well, I've just tried,
and I *think* I did it but frankly I am not even sure how to tell the
difference between trying and failing and trying and succeeding.

It's like the example I gave in an earlier message of seeing a square
circle from the edge. Can we denounce as a liar an individual who
sincerely claims to see a line that way? Maybe, like Einstein or Mozart
she can do things we can't?

So: empirical evidence shows that incoherent visual experiences are
possible, as indicated above and in my previous messages (and as stated
by Stan Klein and by Irvin Rock). Sometimes they are detected as
incoherent, e.g. the penrose triangle, sometimes not, e.g. penrose
dodecagon, or a long devil's pitchfork, or cluttered Escher drawing,
etc.

I am not sure that a very young child would detect the inconsistency
in the simple cases.

(I once asked a five year old child where two cars would meet if they
start off in this configuration, and drive towards each other, where A
is an ordinary slow car and B is a very fast racing car:

    A -->                               ^      <=== B
                                        C

He pointed at a location nearer to B, e.g. C. I asked why, and got the
answer:

    "It goes faster so it will get there sooner."

He was totally unaware of the inconsistency in what he was saying (and
believing, presumably. Some of Piaget's observations are also relevant.)

If View A is expressed in a particularly strong form requiring not just
visual percepts to be internally consistent but the total contents of
consciousness then there are of course even more counter examples, e.g.
perceptual experiences which are known to be wrong.

Example of the flipping folded card: Get a card about 20cm by about
10cm. Fold it down the middle to form a V with an angle of about 30
degrees (or more, or less: sizes and angles are not critical).
Then either put it on the table with the V edge on the table, forming
two walls meeting at a corner or balance it with the ends of the V on
the table and the folded edge horizontal above the table, like a ridge
tent.

Stare at it from a few feet away with one eye for a while, and you
should be able to make it appear to flip from one of the two
configurations to the other.

When it has flipped, move your head from side to side a little and watch
it twist and slide on the table. I KNOW the card is rigid and stationary
and I simultaneously SEE it twist and move. Anyone who claims that I
really must be alternating between the two states of consciousness must
be too much in the grip of a theory to face facts.

Or maybe we have different sorts of brains.

Summary of second argument against the consistency hypthesis: there are
empirical counter examples if you look for them.

A qualification is in order, unfortunately:
Some of the issues which appear to be empirical may not be because the
questions asked can be shown to be incoherent.


3. Argument 3: there's a better explanation of consistency

The third argument was hinted at in previous messages by Minsky and one
of my earlier postings and possibly others. A version of it can perhaps
be read into George Mckee's message of 25th Nov about high-order
processes though I am not sure.

The argument is that some of the cases where consistency is found may
actually be cases of something else: lets call it "invocation of
perceptually learnt models or schemata".

The basic idea is very old: it pervades the writings of Richard Gregory,
is developed in some detail in Minsky's paper on Frames (circa 1973?)
goes back much further to Kant and Bartlett, is implemented in a number
of AI vision systems and will probably often be re-invented, though
possibly with newer more powerful implementation mechanisms.

Visual mechanisms in part have the role of identifying perceived objects
or processes either as belonging to some class (e.g. flower, chair,
furniture, spinning, sliding, fighting, breaking, supporting, etc.) or
being a known individual (Fido, Freda or Freddy, the moon, or the Eiffel
Tower, etc.) For now lets ignore the difference between classification
and (re)identification, and just use the neutral notion of "applying a
schema" (rather than "recognizing a pattern", to allow for the
variability in the data and the flexibility in the mechanisms, as
noted by Kant and others centuries ago).

Some of the categories used may be innate (e.g. edge, surface,
edge-feature, curvature, orientation, closure, linear radial and
circular optical flow, etc. in humans, and probably other things in
other animals, e.g. the tongue-aiming bug-detectors in frogs) others
learnt (e.g. learning to read words or music, learning to identify
plants or birds, learning to see a pair of identical twins as looking
very different (which took me a couple of months) and learning to
recognize most of the "affordances" in our environment, in Gibson's
sense).

For now it doesn't matter which are innate or which learnt: both are
derived from experience of conditions for success and failure of
actions, or merely by induction from salient frequently encountered
examples, whether the derivation is done by the individal or by the gene
pool working through many individuals in an evolutionary process.

The only important point is that as long as the process of derivation
from ACTUAL examples is fairly direct, the derived schemata will be
internally consistent. Where the derived schemata involve combinatorial
generalisation (e.g. induced grammars, induced composition rules for
geometrical structures or forms of motion) then it's possible to create
a schema or schema instance which is not derived from any actual object
and which turns out to be internally inconsistent. (Like the village
barber who shaves all and only those who don't shave themselves.)

The rest of the story should be fairly obvious. We need a principle
which can be roughly formulated thus, ignoring for the moment what
percepts or thoughts might be.

    The visual system tries to find a way of accounting for as much of
    the available information as possible by creating percepts
    instantiating the smallest possible number of high level schemata
    (in the sense of Kant and Barlett).

    The cognitive system attempting to understand some complex
    collection of information attempts to create a thought using
    as few schemata as possible which together subsume all the
    information.

or in other words:

    If we can recognize some intelligible high level structure in a
    perceived scene or thought-about situation we will normally do so.

where intelligible means something like "accommodated in known
schemata". (The schemata need not be previously known: maybe some are
created on the fly. A lot of art is like that.)

(The biological advantages to an animal of being able to do this, and
the engineering advantages to a robot, should be obvious, but we could
discuss them.)

Corollary: wherever schemata directly abstracted from previously
encountered concrete cases are deployed the thought or percept will be
internally coherent.

However, sometimes the data driving the process will invoke novel
combinations of schemata and then there's no guarantee that the result
will be coherent. Penrose/Escher figures and the devil's pitchfork are
examples. Likewise puns?

CONJECTURE: some of the innately determined perceptual schemata will be
implemented at a very low level in brain mechanisms using mutual
excitation and inhibition in "winner takes all" networks, so as to
ensure that wherever possible local ambiguities can be resolved by
combinations of top down and bottom up and sideways processing in
something like a "relaxation" process. (An old idea. E.g. I learnt it
over 20 years ago from Geoff Hinton, but perhaps others had thought of
it earlier.)

Sometimes these networks will be driven by data supporting a small
number of stable states equally well (e.g. necker figure) and then
various mechanisms can make the network flip from one stable state to
another.

In some cases extra bottom-up clues (shading, texture, etc.) or top-down
thought processes ("this is coming towards me") can drive sub-schemata
to form smaller stable, but no longer mutually consistent alliances.

That would account for some of the strange incoherent views of the
necker cube.

Maybe similar winner-takes-all multi-stable networks are involved in the
management of the contents of thought processes, though here we don't
get a rich dense collection of inputs directly clamped by the
environment, and there seems to be more scope for structural variation.

(Consider the differences and similarities between proving a deep
mathematical theorem in your head, reminiscing about your holiday,
planning a meal for your friends, composing a sonnet or limerick,
wondering whether someone reciprocates your feelings, thinking
about consciousness, etc.)

Because of the combinatorial richness of linguistic processes, and their
disconnection from current sensory inputs, linguistically mediated
thoughts will be even less constrained to be consistent than perceptual
contents, although when you are thinking about actual situations they
will normally be consistent because anything actual must be. (I
am speaking more loosely than some philosophers and logicians
would like).

This is why it is much easier to produce ambiguous or inconsistent
sentences, puns, linguistic jokes of various kinds, and nonsense
sentences, than pictures with those qualities. But of course expert
cartoonists have been doing it for a long time: e.g. drawing a face
which is clearly recognizable as a certain politician and as a certain
animal.

To summarise point 3:
Although perceptual contents are normally internally consistent
and thought contents often are, this consistency is not something the
brain can enforce. Rather it's a SIDE-EFFECT of powerful biologically
useful processing mechanisms which by and large have developed to fit
the actual world, not a world full of things that are impossible in our
world.

That's just as well, since if the brain needed to detect and enforce
general semantic consistency in any particular sub-structure it would
require mechanisms which, as far as we know, cannot exist.

By allowing (as Stan Klein and Irvin Rock do) that the search for a high
level consistent integrating schema subsuming the current visual
information can fail, we allow the possibility of a globally
inconsistent percept involving smaller scale locally consistent
percepts.

And that seems to fit empirical data about inconsistent percepts very
nicely, while also explaining the cases where a coherent whole does
occur as a side effect of a *soft* requirement of a biologically
effective system, not a hard constraint imposed by some impossible
mechanism.

If all this is just a long winded way of saying the following (quoted in
a previous message):

    No matter what we do, the visual system tries to find a single
    coherent conscious interpretation at any given moment.
        (Page 87 of "In the Theater of consciousness")

I.e. *tries* but doesn't necessarily *succeed* then Bernie Baars is
saying the same as I am, and I apologise for misreading it previously.
(Especially as I find a lot in common with other aspects of his
theories.)

CODA:

At the Elsinore conference last August, Doug Watt repeatedly claimed
that any good theory of consciousness must accommodate what he referred
to as "disorders of consciousness". I agree: disorders and variations
both across and within species and individuals need to be accounted for.

 From the little I know about autism, it seems that one feature of some
autistic people involves visual processing remaining fragmented, without
the global coherence that comes from applying a high level (abstract)
concept.

This could be part of the explanation of the ability of some of them to
produce surprisingly accurate paintings and drawings: because their
processing of visual input stops at the kind of local 2-D structure
which an artist needs to replicate, instead of moving on to the
integrating 3-D structure which is then much harder to project back onto
the page.
    (Compare: Lorna Selfe,
    Nadia: a case of extraordinary drawing ability in an autistic child,
    London, Academic Press, 1977.)


Aaron