Call for Proposals
on
Cognitive Systems ¹

Purpose of this document

This background document aims to provide a clear definition of the scientific and technological domain cognitive systems as covered by the call² and anticipate the information needed for individuals to provide proposals that are interesting, achievable and within the scope of the call.

Domain and Objective of the call

Cognitive Systems are systems that perceive, understand, learn and develop through individual or social interaction with their environment. This call solicits proposals for multi-disciplinary research on artificial cognitive systems. While research on cognitive systems may derive inspiration from biological intelligence, Cognitive Systems is a separate scientific discipline, concerning artificial systems that combine perception, action, reasoning, learning and communication. As a scientific discipline, Cognitive Systems seeks to provide an enabling technology for robotics and automation, natural language understanding, man-machine interaction and complex real-world systems. However Cognitive Systems is not about applications in any of these domains.

Cognitive systems will require convergence of action, perception and reasoning. Action, taken in a broad sense, provides the foundation for semantics. Actions may involve applying and controlling a mechanical device. Alternatively, they may have the form of communicating or interacting with humans. Actions may also have the form of changes to the internal state of the system, such as a change in focus of attention, with no immediate external manifestation.

Perception is the interpretation of sensory input. Cognitive systems must be perceptually enabled in order to generate appropriate actions and behaviours. In its most sophisticated form, perception provides a model of a situation that enables reasoning. However, perception may also directly result in the selection of behaviours, the execution of actions, or a change in focus of attention.

Coordinating perception and action requires reasoning. Such reasoning can occur over multiple time scales and at multiple levels of abstraction. At the lowest level of abstraction, a system may exhibit reflexive behaviours in which sensory signals are mapped directly to actuator controls. Reasoning at this level may be used to select and regulate the transformations. At intermediate levels, compositions of actions or behaviours bring the system and the world to desired state. Reasoning may be used to select and apply a predetermined plan of action. Reasoning may also be used to adapt an existing plan or to generate new plans to attain a goal from a new situation, or to learn new abilities.

To be general, a cognitive system must be able to form and exploit new concepts. The system must learn. Such learning is not restricted to recognition and categorization. It extends to automatic acquisition of perception-action cycles, to parameter control and to formation of abstract concepts and behaviours. The ability to learn perception-action cycles, to learn control and coordination of perception-action, to learn procedures to accomplish goals, to learn new concepts, and to learn and improve new plans of actions are all important problems for cognitive systems.

Clarifications of the call

The goal is to create and develop a scientific foundation that will apply across many domains of engineering science. Such a foundation will require innovative theoretical insights about perception, understanding, interaction, learning and representations.

The call for proposals on Cognitive Systems seeks to break with current practice by encouraging efforts that integrate competences from multiple disciplines in order to develop a theoretical foundation for cognitive systems. No single scientific competence is expected to be sufficient. Projects should bring together a strong team with an established reputation for research excellence in an appropriate variety of disciplines.

Consortia should be prepared to provide conceptual innovation, backed by experimental demonstration and comparative performance evaluation with integrated systems. Integrated projects (IPs) should unify an appropriate multi-disciplinary team to address an important "grand challenge". Specific Targeted Research Projects (STRePs) may be focussed on specific hard problems. Both IPs and STRePs should make a substantive theoretical contribution, a comparative justification for the methods proposed, accompanied by concrete validation based on experimental demonstration and comparative performance evaluation of an artificial system. A network of excellence is sought to provide a forum to unify the cluster of cognitive system projects.

Proposals are expected to provide fundamental insights into several of the following questions:

The nature of cognition: What makes a system cognitive? What are the requirements for a cognitive system? What properties characterise cognitive systems? What categories of cognitive tasks can be defined? To what extent is embodiment necessary for cognitive systems? To what extent are perception, reasoning, language, and embodied action necessary for cognition? Is action required to define perception?

Architectures for cognition: What architectural models can be used to design cognitive systems? How can perception, action, learning, communication and self-description and self-awareness be integrated? What is the nature and function of memory? Can we build systems that are auto-descriptive, auto-critical, auto-regulating and auto-healing?

The nature of knowledge: What kinds of informational states, memory and knowledge are useful to identify? How can knowledge enable generation of new knowledge? What are the roles and nature of spatial, temporal and causal concepts? What is the role of language in cognition and of cognition in language? How can meaning be characterised?

Perception: Is action necessary for perception? How can affordances be learned and perceived? Is the distinction between top-down and bottom-up processes useful in perception?

Learning: How can a system learn of competences, affordances, categories and concepts? What are the different modes of learning needed in a cognitive system? How can new knowledge or skills be integrated coherently with old knowledge or skills without compromising the stability of the system?

Autonomous systems: What are the varieties and mechanisms of autonomy? What is required for a system to be autonomous? What is the relation between cognitive systems and autonomous systems? To what extent are emotions and other affective states and processes necessary for autonomy?

The notion of self: What does it mean for an artificial system to be aware? What is the functional role of consciousness in an artificial system?

Social Interaction: How do considerations of communication, cooperation, and competition impact on cognition?

Goals: How can goals be identified to a cognitive system? How does language impact on goal specification? Can cognitive systems be instructed to achieve goals and at what level and in what manner? Can goals be specified at all or must they be learned?

Application domains should not be considered ends of the projects in themselves. Instead, they should serve as a source of research questions as well as to provide tests that makes it possible to demonstrate and evaluate the impact of their conceptual and technical innovations. Proposals should rely on domains that involve a rich set of actions or interactions. Examples of application domains include

1. Adaptive intelligent environments.

2. Interactive talking heads that integrate visual, acoustic and natural-language abilities and the capability to learn by interaction.

3. Robotic assistants or robot companions.

4. Active learning assistants to individuals or groups recording, structuring and exploiting sensory, biological, biographic, communicative or medial information.

5. Teaching machines.

6. Autonomous robots.

7. Control of complex processes.

8. Non-disruptive personalisation of software.

9. Visual surveillance.

---

5.1.1. Annex

5.1.2. Objectives and focus of call for proposals addressing Cognitive Systems as published in the European Commission’s IST Work Programme 2003/2004:

5.1.3. 2.3.2.4 Cognitive Systems

Objective: To construct physically instantiated or embodied systems that can perceive, understand (the semantics of information conveyed through their perceptual input) and interact with their environment, and evolve in order to achieve human-like performance in activities requiring context-(situation and task) specific knowledge.

Focus is on:

- methodologies and construction of robust and adaptive cognitive systems integrating perception, reasoning, representation and learning, that are capable of interpretation, physical interaction and communication in real-world environments for the purpose of performing goal-directed tasks. Research will aim at realising complete systems with real-time performance and/or bounded rationality, have well developed memory capacities (e.g. short term, long term, iconic, associative) with efficient representation, and that acquire representations as needed to realise performance goals. The emphasis is on closing the loop in realistic test cases.

A main target of this research is interdisciplinarity, i.e., to carefully consider the integration of different disciplines including computer vision, natural language understanding, robotics, artificial intelligence, mathematics and cognitive neuroscience and its impact on overall system design. IP³s are expected to leverage these communities to integrate methods and insights towards the objective of realising entire systems and to promote community building. NoE⁴s will provide a channel for fostering foundational research, for developing and maintaining common resources, specifically, of open systems and training environments to study learning and evolving systems.

Version 2.2 final, 19 June 2003

Contact: Colette Maloney (Colette.Maloney@cec.eu.int)