From Aaron Sun Jun 4 02:16:05 BST 1995 Newsgroups: sci.lang,sci.psychology,rec.arts.books,comp.ai.philosophy,comp.ai,sci.cognitive References: <3q2put$p07@Mercury.mcs.com> Subject: Promises, obligations, etc. (Was Chomsky on Consciousness and Dennett) [I've added comp.ai and sci.cognitive, as this is getting technical. comp.ai readers who want to read the earlier parts of the thread can look in comp.ai.philosophy] jmc@SAIL.Stanford.EDU (John McCarthy) writes: > Date: 26 May 1995 21:21:55 GMT > Organization: Computer Science Department, Stanford University > .... > Here's a matter in which I agree with Austin and Searle. Austin asked > whether a promise was just a true statement of intention, and > concluded it wasn't, because making a promise created an obligation to > fulfill it. Our formalization of speech acts as computer output > should include the fact that making a promise creates an obligation. > It is part of the verification of an Elephant program that it only > creates obligations it is authorized to create and that it does its > part in fulfilling obligations it creates. Notions like "promise", "obligation", "permission", "authorization", (lets call these `normative' concepts) have some interesting properties, which make them different from non-normative `descriptive' concepts that lead to philosophical problems in AI (e.g. "belief", "desire", "understanding", "intention", "qualia"). The differences have to do with (a) the presuppositions of their use and (b) the practical consequences of their use. There are also other differences to do with the kinds of speech-acts that are typically involved in using the concepts (e.g. when using the normative concepts within the social system in which they occur one is typically commending, approving, disapproving, blaming, etc. whether explicitly or implicitly, or actually, or hypothetically. I'll return to this below.). Roughly speaking, what I have called the normative concepts, like obligation, permission, etc. presuppose something like a social system in which collections of communicating agents interact and do things like: judge one another's behaviour as right or wrong, good or bad attempt to reward or punish one another's behaviour set up agreed general (long term) standards for controlling behaviour (laws, rules, conventions), set up individual (or short term) `contracts' imposing temporary standards on a subset of behaviour set up cooperative mechanisms for detecting violations, and for enforcing or encouraging conformity to the standards. set up procedures for ascertaining causal responsibility when violations occur. Different combinations of these (and similar) features may be found in different societies. Normative phenomena are emergent in that they constitute yet another virtual machine level, implemented in lower level machines and ultimately in physics. But they are not derivable from physics: The same physical laws can support different, mutually inconsistent, norm systems (like different human cultures). I have set that normative phenomena require a system of interacting agents. There may be limiting cases of these phenomena which involve only one individual (who makes promises to himself, creates obligations for himself, for example), but these are limiting cases that differ in a variety of ways from the social cases, just as social cases differ among themselves. Depending on the precise nature of the social system, and the cognitive and other capabilities of the agents involved, different concepts of obligation, authorisation, permission, etc. may exist. For example: in some systems you may be able to remove an obligation by going to a priest and being absolved; or by buying yourself out of it; or by producing evidence that you acquired the obligation under duress; or by simply reaching agreement with the other individuals concerned to cancel the obligations. In other social systems, obligations, like death sentences in some societies, may be non-cancellable. In some societies there may be only a subset of obligations that are cancellable. In many societies, though not necessarily all, the notions of obligations, and the like are linked to the notion of a currency, which can be used to pay for things (goods, services, opportunities, etc.) E.g. in England, paper money includes the words "I promise to pay the bearer on demand the sum of XXX pounds", along with the signature of the chief cashier of the Bank of England. (I can't recall what's on Scottish bank notes.) Exactly what sort of obligation this promise creates is not clear (the cashier could simply hand you back the note as a way of keeping the promise). But the monetary system works because of all sorts of associated obligations, e.g. obligations of shops to accept money in payment for goods, obligations of employers to give money in return for labour, etc. In general the notion of an obligation exists along with the notion that something bad will happen (or, more subtly, should happen) if the obligation is not met. - In the simplest cases there is a simple causal link (e.g. Because I undertook a mountain journey by car, I was obliged to change my tyres, as otherwise an accident would have occurred.) - In other cases the links are more indirect and less well defined (e.g. what sort of punishment the social system will apply to transgressors may be up to the whims of queens or kings or judges). - In other cases the agents concerned may base their notion of obligation on a subtle metaphysical notion of it being bad to do certain things (irrespective of any causal consequences or human judgement of it being bad). The development of this metaphysical notion is a very powerful and relatively cheap form of social control (cheaper than a police force). Whether it works depends on the existence within agents of cognitive apparatus that can be made to think about good and bad in metaphysical ways (e.g. apparatus that supports the mechanism of conscience and guilt feelings, providing internalised control). From all this I conclude that John's remark > Our formalization of speech acts as computer output > should include the fact that making a promise creates an obligation. ^^^^^^^^ needs some elaboration. If this is a "fact" it is a fact that is relative to a social or cultural system, and exactly what sort of fact it is (i.e. what the existence of the obligation amounts to) will differ from one social system to another. Of course, I am not saying this cannot all be formalised: only there will be different formalisations for different social systems. There's another subtlety: the statement that making a promise creates an obligation can be made by someone who is PART of the society and ACCEPTS its norms, in which case uttering the statement may be more than asserting a fact: it may be part of the process of inculcating, re-inforcing, supporting the norms and influencing others to conform. Alternatively, the statement can be made `from outside' by someone who is merely observing the system, like an anthropologist from another culture. The internal and external utterances may use the very same words, but actually involve very different speech acts, with different sorts of implications. (E.g. the `internal version' may imply `to break a promise is a bad thing and should be avoided', while the `external' version does not express any such commitment from the speaker.) Sometimes people within a society learn to make the statements from the external standpoint (e.g. when they reject the norms of the society). Again, I am sure all of this can be formalised. I'd give higher priority to formalising the descriptions of the cognitive mechanisms and social systems that support different kinds of obligations, permissions, etc. than to simply providing formal `translations' of those utterances (as is typical in deontic logic). Doing all that would require reinventing (and probably improving on) much social philosophy, with an AI framework, i.e. designing a working social system. People who work on agent theories and distributed AI will eventually get around to (re)discovering all this, if they haven't already. (I seem to recall that some of it was in Carl Hewitt's theory of "actors" reported in IJCAI 1973, or thereabouts.) I've had a quick look at Marvin Minsky's The Society of Mind to see whether his analogy between a mind and a society of agents made use of all this stuff about obligations, permissions, authorisations, contracts, promises, etc. As far as I can see, it hardly figures at all (except where he mentions Freud's censor, and perhaps implicitly when he talks about the roles of reward and punishment in learning). For instance the index does not include any of the key words discussed above. But maybe I've just not remembered enough about the contents of the book. (Marvin - if you are reading this would you care to comment?) (Incidentally, I believe Plato had a theory of mind as a social system that explicitly involved internal laws, law-breakers, law-enforcers, etc. But my knowledge of this is all second-hand, and Plato's version was potentially circular, if all the agents also had complete internal social systems ad infinitem.) I suspect that the more independent and self-motivating the sub-agents in a mind are, the more that mind will need a system of internal norms, obligations, permissions, contracts, trials, enforcement, punishment, etc. to minimise the degree of incoherence. Different minds may differ in how far and how they resemble such societies with norm-systems. =================================================================== I've commented in previous posts on the architectural presuppositions of what I called the `descriptive' concepts, above. These concepts, e.g. knowing, perceiving, understanding, deciding, wanting, believing, etc. are also emergent concepts, but they are related to their implementation in a different way from the normative concepts. Aaron --- From Aaron Sun Jun 4 02:16:05 BST 1995 Newsgroups: sci.lang,sci.psychology,rec.arts.books,comp.ai.philosophy,comp.ai,sci.cognitive References: <3q2put$p07@Mercury.mcs.com> Subject: Promises, obligations, etc. (Was Chomsky on Consciousness and Dennett) [I've added comp.ai and sci.cognitive, as this is getting technical. comp.ai readers who want to read the earlier parts of the thread can look in comp.ai.philosophy] jmc@SAIL.Stanford.EDU (John McCarthy) writes: > Date: 26 May 1995 21:21:55 GMT > Organization: Computer Science Department, Stanford University > .... > Here's a matter in which I agree with Austin and Searle. Austin asked > whether a promise was just a true statement of intention, and > concluded it wasn't, because making a promise created an obligation to > fulfill it. Our formalization of speech acts as computer output > should include the fact that making a promise creates an obligation. > It is part of the verification of an Elephant program that it only > creates obligations it is authorized to create and that it does its > part in fulfilling obligations it creates. Notions like "promise", "obligation", "permission", "authorization", (lets call these `normative' concepts) have some interesting properties, which make them different from non-normative `descriptive' concepts that lead to philosophical problems in AI (e.g. "belief", "desire", "understanding", "intention", "qualia"). The differences have to do with (a) the presuppositions of their use and (b) the practical consequences of their use. There are also other differences to do with the kinds of speech-acts that are typically involved in using the concepts (e.g. when using the normative concepts within the social system in which they occur one is typically commending, approving, disapproving, blaming, etc. whether explicitly or implicitly, or actually, or hypothetically. I'll return to this below.). Roughly speaking, what I have called the normative concepts, like obligation, permission, etc. presuppose something like a social system in which collections of communicating agents interact and do things like: judge one another's behaviour as right or wrong, good or bad attempt to reward or punish one another's behaviour set up agreed general (long term) standards for controlling behaviour (laws, rules, conventions), set up individual (or short term) `contracts' imposing temporary standards on a subset of behaviour set up cooperative mechanisms for detecting violations, and for enforcing or encouraging conformity to the standards. set up procedures for ascertaining causal responsibility when violations occur. Different combinations of these (and similar) features may be found in different societies. Normative phenomena are emergent in that they constitute yet another virtual machine level, implemented in lower level machines and ultimately in physics. But they are not derivable from physics: The same physical laws can support different, mutually inconsistent, norm systems (like different human cultures). I have set that normative phenomena require a system of interacting agents. There may be limiting cases of these phenomena which involve only one individual (who makes promises to himself, creates obligations for himself, for example), but these are limiting cases that differ in a variety of ways from the social cases, just as social cases differ among themselves. Depending on the precise nature of the social system, and the cognitive and other capabilities of the agents involved, different concepts of obligation, authorisation, permission, etc. may exist. For example: in some systems you may be able to remove an obligation by going to a priest and being absolved; or by buying yourself out of it; or by producing evidence that you acquired the obligation under duress; or by simply reaching agreement with the other individuals concerned to cancel the obligations. In other social systems, obligations, like death sentences in some societies, may be non-cancellable. In some societies there may be only a subset of obligations that are cancellable. In many societies, though not necessarily all, the notions of obligations, and the like are linked to the notion of a currency, which can be used to pay for things (goods, services, opportunities, etc.) E.g. in England, paper money includes the words "I promise to pay the bearer on demand the sum of XXX pounds", along with the signature of the chief cashier of the Bank of England. (I can't recall what's on Scottish bank notes.) Exactly what sort of obligation this promise creates is not clear (the cashier could simply hand you back the note as a way of keeping the promise). But the monetary system works because of all sorts of associated obligations, e.g. obligations of shops to accept money in payment for goods, obligations of employers to give money in return for labour, etc. In general the notion of an obligation exists along with the notion that something bad will happen (or, more subtly, should happen) if the obligation is not met. - In the simplest cases there is a simple causal link (e.g. Because I undertook a mountain journey by car, I was obliged to change my tyres, as otherwise an accident would have occurred.) - In other cases the links are more indirect and less well defined (e.g. what sort of punishment the social system will apply to transgressors may be up to the whims of queens or kings or judges). - In other cases the agents concerned may base their notion of obligation on a subtle metaphysical notion of it being bad to do certain things (irrespective of any causal consequences or human judgement of it being bad). The development of this metaphysical notion is a very powerful and relatively cheap form of social control (cheaper than a police force). Whether it works depends on the existence within agents of cognitive apparatus that can be made to think about good and bad in metaphysical ways (e.g. apparatus that supports the mechanism of conscience and guilt feelings, providing internalised control). From all this I conclude that John's remark > Our formalization of speech acts as computer output > should include the fact that making a promise creates an obligation. ^^^^^^^^ needs some elaboration. If this is a "fact" it is a fact that is relative to a social or cultural system, and exactly what sort of fact it is (i.e. what the existence of the obligation amounts to) will differ from one social system to another. Of course, I am not saying this cannot all be formalised: only there will be different formalisations for different social systems. There's another subtlety: the statement that making a promise creates an obligation can be made by someone who is PART of the society and ACCEPTS its norms, in which case uttering the statement may be more than asserting a fact: it may be part of the process of inculcating, re-inforcing, supporting the norms and influencing others to conform. Alternatively, the statement can be made `from outside' by someone who is merely observing the system, like an anthropologist from another culture. The internal and external utterances may use the very same words, but actually involve very different speech acts, with different sorts of implications. (E.g. the `internal version' may imply `to break a promise is a bad thing and should be avoided', while the `external' version does not express any such commitment from the speaker.) Sometimes people within a society learn to make the statements from the external standpoint (e.g. when they reject the norms of the society). Again, I am sure all of this can be formalised. I'd give higher priority to formalising the descriptions of the cognitive mechanisms and social systems that support different kinds of obligations, permissions, etc. than to simply providing formal `translations' of those utterances (as is typical in deontic logic). Doing all that would require reinventing (and probably improving on) much social philosophy, with an AI framework, i.e. designing a working social system. People who work on agent theories and distributed AI will eventually get around to (re)discovering all this, if they haven't already. (I seem to recall that some of it was in Carl Hewitt's theory of "actors" reported in IJCAI 1973, or thereabouts.) I've had a quick look at Marvin Minsky's The Society of Mind to see whether his analogy between a mind and a society of agents made use of all this stuff about obligations, permissions, authorisations, contracts, promises, etc. As far as I can see, it hardly figures at all (except where he mentions Freud's censor, and perhaps implicitly when he talks about the roles of reward and punishment in learning). For instance the index does not include any of the key words discussed above. But maybe I've just not remembered enough about the contents of the book. (Marvin - if you are reading this would you care to comment?) (Incidentally, I believe Plato had a theory of mind as a social system that explicitly involved internal laws, law-breakers, law-enforcers, etc. But my knowledge of this is all second-hand, and Plato's version was potentially circular, if all the agents also had complete internal social systems ad infinitem.) I suspect that the more independent and self-motivating the sub-agents in a mind are, the more that mind will need a system of internal norms, obligations, permissions, contracts, trials, enforcement, punishment, etc. to minimise the degree of incoherence. Different minds may differ in how far and how they resemble such societies with norm-systems. =================================================================== I've commented in previous posts on the architectural presuppositions of what I called the `descriptive' concepts, above. These concepts, e.g. knowing, perceiving, understanding, deciding, wanting, believing, etc. are also emergent concepts, but they are related to their implementation in a different way from the normative concepts. Aaron --- From Aaron Sun Jun 4 23:14:22 BST 1995 Newsgroups: sci.lang,sci.psychology,comp.ai.philosophy,comp.ai,sci.cognitive References: <3q2put$p07@Mercury.mcs.com> <3qr1fv$h7a@percy.cs.bham.ac.uk> Subject: Re: Promises, obligations, etc. (Was Chomsky on Consciousness and Dennett) [I've removed rec.arts.books, which I had previously not noticed. It does not seem appropriate.] Brian J Flanagan writes: > Date: Sun, 4 Jun 1995 10:29:10 -0500 > Organization: University of Iowa, Iowa City, IA, USA Thanks for your comments. I don't have further thoughts about any of them, apart from wishing to clarify one point. I wrote (AS) > > Normative phenomena are emergent in that they constitute yet another > > virtual machine level, implemented in lower level machines and > > ultimately in physics. But they are not derivable from physics: The > > same physical laws can support different, mutually inconsistent, > > norm systems (like different human cultures). (BJF) > Talk of "emergent" phenomena makes me uneasy. It is like the cartoon > where two scientific eminences are standing at a blackboard which is filled > with impressive-looking mathematical arcana--except for the blank portion > where the miracle occurs. I have a general policy when intelligent people seem to be making sincere claims about things for which there are apparently no scientific justifications. The policy (learnt from Karl Popper's writings, believe it or not) is to try to find the strongest and most interesting true interpretation that I can give to their theories and then to offer a clear analysis and defense of those theories using that interpretation. The analysis and defense will not always be accepted by the original theorists. Thus I think I can understand the truths that lie behind claims that qualia exist, and have argued elsewhere that intelligent robots with human-like capabilities will have (e.g.) visual qualia (defined in terms of the ability to pay attention to certain kinds of intermediate data-structures required for visual processing. By elaborating on that idea we can remove the stigma associated with the notion of qualia and replace philosophical debate with scientific investigation as to which sorts of qualia occur, what their function is, which animals or machines have them, etc. Similarly I think that people who talk about certain phenomena as "emergent" have an important point (whether they realise exactly what it is or not). Namely, the point is that there are many phenomena that can correctly be described in terms of different ontologies, where one ontology O1 (e.g. a physical ontology, or perhaps the ontology defined by the machine language of a particular type of computer) provides an *implementation* for another ontology O2, and O2 may be described as "emergent" if the following relationships hold between O1 and O2: 1. The concepts of O2 are not definable in terms of those of O1. (e.g. the concepts "word", "sentence", "paragraph", "page", etc. used in a word-processor are not definable in terms of the concepts of physics, nor in terms of the machine code of the underlying computer.) 2. The laws of behaviour of O2 (e.g. the rules for breaking lines in a paragraph of text) are not derivable from the laws of O1, since the rules of O2 can be changed without changing the laws of O1. (E.g. you don't have to change physics, or switch to a new CPU, in order to change text formatting rules in a wordprocessor.) 3. O1 is not logically necessary for the implementation of O2, in the sense that in principle the implementation could be based on a different ontology provided that it supported sufficient structural variability and appropriate causal machinery. 4. The events in O2 can be causally related with one another, and can both cause and be caused by events in O1. (This is a claim whose full defence requires a solution to one of the hardest problems in philosophy: the analysis of the concept of cause.) 5. There need not be regular correlations between objects and events in O2 and objects and events in the implementation ontology O1. (e.g. there is probably no physically describable set of phenomena that regularly accompanies or underlies the creation of an obligation.) (I am aware that I am presupposing several points needing further analysis here, including what an ontology is, and what it means for concepts to be definable or not definable in terms of other concepts. I am not assuming that there are always only two levesl of ontology: implementations may go via several intermediate ontologies.) I think there are many examples of emergence in the sense defined (roughly) above. Most of the phenomena that software engineers (or humble programmers) are concerned with (e.g. events and structures in a lisp or prolog or Ada virtual machine, or in an office automation system) are emergent in this sense. In those cases, the relationship between the ontologies is fully understood insofar as we have implemented one of them in terms of the other. In these cases, emergence involves nothing mysterious or anti-scientific. (Chemistry may be emergent relative to quantum physics, in this sense, though we did not do the implementation.) There are many cases of emergence that are not so well understood, including emergent social, economic, legal, political phenomena, and many of the phenomena studied by biologists. Part of the point of my previous message was to show that there are different kinds of emergence. I conjecture that we shall eventually have a full understanding of the nature of mental phenomena as emergent phenomena that are implemented in brain structures and processes, and which may also prove to be implementable in other mechanisms, though that's a matter for further investigation. | it seems to me that there is substantial agreement between you & | Minsky, tho' you formulate your ideas in distinctly different | idioms. That is my impression also, though we come to our conclusions in different ways, and therefore do not necessarily agree on every detail. Aaron ---