This paper, combining the standpoints of philosophy and Artificial Intelligence with theoretical psychology, summarises several decades of investigation of the variety of functions of vision in humans and other animals, pointing out that biological evolution has solved many more problems than are normally noticed. Many of the phenomena discovered by psychologists and neuroscientists require sophisticated controlled laboratory settings and specialised measuring equipment, whereas the functions of vision reported here mostly require only careful attention to a wide range of everyday competences that easily go unnoticed. Currently available computer models and neural theories are very far from explaining those functions, so progress in explaining how vision works is more in need of new proposals for explanatory mechanisms than new laboratory data. Systematically formulating the requirements for such mechanisms is not easy. If we start by analysing familiar competences, that can suggest new experiments to clarify precise forms of these competences, how they develop within individuals, which other species have them, and how performance varies according to conditions. This will help to constrain requirements for models purporting to explain how the competences work. The paper ends with speculations regarding the need for new kinds of information-processing machinery to account for the phenomena.

Keywords: cosy; irlab

This paper summarises ideas I have been working on over the last 35 years or so, about relations between the study of natural minds and the design of artificial minds, and the requirements for both sorts of minds. The key idea is that natural minds are information-processing virtual machines produced by evolution. What sort of information-processing machine a human mind is requires much detailed investigation of the many kinds of things minds can do. At present, it is not clear whether producing artificial minds with similar powers will require new kinds of computing machinery or merely much faster and bigger computers than we have now. Some things once thought hard to implement in artificial minds, such as affective states and processes, including emotions, can be construed as aspects of the control mechanisms of minds. This view of mind is largely compatible in principle with psychoanalytic theory, though some details are very different. The therapeutic aspect of psychoanalysis is analogous to run-time debugging of a virtual machine. In order to do psychotherapy well we need to understand the architecture of the machine well enough to know what sorts of bugs can develop and which ones can be removed, or have their impact reduced, and how. Otherwise treatment will be a hit-and-miss affair.

Keywords: cosy; irlab

Some issues concerning requirements for architectures, mechanisms, ontologies and forms of representation in intelligent human-like or animal-like robots are discussed. The tautology that a robot that acts and perceives in the world must be embodied is often combined with false premises, such as the premiss that a particular type of body is a requirement for intelligence, or for human intelligence, or the premiss that all cognition is concerned with sensorimotor interactions, or the premiss that all cognition is implemented in dynamical systems closely coupled with sensors and effectors. It is time to step back and ask what robotic research in the past decade has been ignoring. I shall try to identify some ma jor research gaps by a combination of assembling requirements that have been largely ignored and design ideas that have not been investigated - partly because at present it is too difficult to make significant progress on those problems with physical robots, as too many different problems need to be solved simultaneously. In particular, the importance of studying some abstract features of the environment about which the animal or robot has to learn (extending ideas of J.J.Gibson) has not been widely appreciated.

Keywords: cosy; irlab

A child, or young human-like robot of the future, needs to develop an information-processing architecture, forms of representation, and mechanisms to support perceiving, manipulating, and thinking about the world, especially perceiving and thinking about actual and possible structures and processes in a 3-D environment. The mechanisms for extending those representations and mechanisms, are also the core mechanisms required for developing mathematical competences, especially geometric and topological reasoning competences. Understanding both the natural processes and the requirements for future human-like robots requires AI designers to develop new forms of representation and mechanisms for geometric and topological reasoning to explain a child's (or robot's) development of understanding of affordances, and the proto-affordances that underlie them. A suitable multi-functional self-extending architecture will enable those competences to be developed. Within such a machine, human-like mathematical learning will be possible. It is argued that this can support Kant's philosophy of mathematics, as against Humean philosophies. It also exposes serious limitations in studies of mathematical development by psychologists.

Keywords: cosy; irlab

This is a 'preprint' for the final chapter of a Handbook of Computational Psychology which is currently in press. The differences between this and the version to be published include British vs American spelling and punctuation. This version also has a few footnotes that had to be excluded. For some reason the publisher did not want abstracts for each chapter, so there is no official abstract. The preprint version also includes a table of contents for the chapter (copied below). Overview Instead of surveying achievements of AI and computational Cognitive Science as might be expected, this chapter complements the Editor's review of requirements for work on integrated systems in Chapter 1, by presenting a personal view of some of the major unsolved problems, and obstacles to solving them. It attempts to identify some major gaps, and to explain why progress has been much slower than many people expected. It also includes some recommendations for improving progress and for countering the fragmentation and factionalism of the research community. It it is relatively easy to identify long term ambitions in vague terms, e.g. the aim of modelling human flexibility, human learning, human cognitive development, human language understanding or human creativity; but taking steps to fulfil the ambitions is fraught with difficulties. So progress in modelling human and animal cognition is slow despite many impressive narrow-focus achievements, including those reported in earlier chapters. An attempt is made to explain why progress in producing realistic models of human and animal competences is slow, namely (a) the great difficulty of the problems, (b) failure to understand the breadth, depth and diversity of the problems, (c) the fragmentation of the research community and (d) social and institutional pressures against risky multi-disciplinary, long-term research. Advances in computing power, theory and techniques will not suffice to overcome these difficulties. Partial remedies are offered, namely identifying some of the unrecognised problems and suggesting how to plan research on the basis of `backward-chaining' from long term goals, in ways that may, perhaps, help warring factions to collaborate and provide new ways to select targets and assess progress.

Keywords: cosy; irlab

This paper complements McCarthy's ``The well designed child'', in part by putting it in a broader context, a space of sets of requirements and a space of designs, and in part by relating design features to development of mathematical competences. I moved into AI hoping to understand myself, especially hoping to understand how I could do mathematics. Over the ensuing four decades, my interactions with AI and other disciplines led to: design-based, cross-disciplinary investigations of requirements, especial those arising from interactions with a complex environment; a draft partial ontology for describing spaces of possible architectures, especially virtual machine architectures; investigations of how different forms of representation relate to different functions; analysis of biological nature/nurture tradeoffs and their relevance to machines; studies of control issues in a complex architecture; and showing how what can occur in such an architecture relates to our intuitive concepts of motivation, feeling, preferences, emotions, attitudes, values, moods, consciousness, etc. I conjecture that working models of human vision can lead to models of spatial reasoning that would help to support Kant's view of mathematics by showing that human mathematical abilities are a natural extension of abilities produced by biological evolution that are not yet properly understood, and have barely been noticed by psychologists and neuroscientists. Some requirements for such models, are described, including aspects of our ability to interact with complex 3-D structures and processes that extend Gibson's ideas concerning action affordances, to include proto-affordances, epistemic affordances and deliberative affordances. Some of what a child learns about structures and processes starts as empirical then, as a result of reflective processes, can be recognised as necessary (e.g., mathematical) truths. These processes normally develop unnoticed in young children, but provide the basis for much creativity in behaviour, as well as leading, in some, to development of an interest in mathematics. We still need to understand what sort of self-monitoring and self-extending architecture, and what forms of representation, are required to make this possible. This paper does not presuppose that all mathematical learners can do logic, though some fairly general form of reasoning seems to be required.

Keywords: cosy; irlab

Demonstration that humans can be presented with a collection of unpredictable photographs of natural, moderately complex scenes (e.g. about 10), at the rate of one a second, and can then answer somewhere between 30% and 70% of a set of unexpected questions about what was seen in the pictures. This demonstrates some constraints on possible mechanisms capable of supporting vision in humans and perhaps some other animals. The processing needs to go up several levels of abstraction (e.g. perhaps nine or ten levels) within a second. This almost certainly makes use of a great deal of prior knowledge about kinds of things that can be seen in our world, though most of that knowledge is dormant most of the time. Somehow the image data can wake up relevant subsets at various levels of abstraction, which can then collaborate in converging on an interpretation. If the image is removed after a short time not all the potential processing will have been completed, but a surprising amount has been achieved. There seems to be a lot of individual variation, though so far only informal tests have been done.

Keywords: cosy; irlab

This paper, combining the standpoints of philosophy and Artificial Intelligence with theoretical psychology, summarises several decades of investigation by the author of the variety of functions of vision in humans and other animals, pointing out that biological evolution has solved many more problems than are normally noticed. For example, the biological functions of human and animal vision are closely related to the ability of humans to do mathematics, including discovering and proving theorems in geometry, topology and arithmetic. Many of the phenomena discovered by psychologists and neuroscientists require sophisticated controlled laboratory settings and specialised measuring equipment, whereas the functions of vision reported here mostly require only careful attention to a wide range of everyday competences that easily go unnoticed. Currently available computer models and neural theories are very far from explaining those functions, so progress in explaining how vision works is more in need of new proposals for explanatory mechanisms than new laboratory data. Systematically formulating the requirements for such mechanisms is not easy. If we start by analysing familiar competences, that can suggest new experiments to clarify precise forms of these competences, how they develop within individuals, which other species have them, and how performance varies according to conditions. This will help to constrain requirements for models purporting to explain how the competences work. For example, Gibson's theory of affordances needs a number of extensions, including allowing affordances to be composed in several ways from lower level proto-affordances. The paper ends with speculations regarding the need for new kinds of information-processing machinery to account for the phenomena.

Keywords: cosy; irlab

Some AI researchers aim to make useful machines, including robots. Others aim to understand general principles of information-processing machines whether natural or artificial, often with special emphasis on humans and human-like systems: They primarily address scientific and philosophical questions rather than practical goals. However, the tasks required to pursue scientific and engineering goals overlap considerably, since both involve building working systems to test ideas and demonstrate results, and the conceptual frameworks and development tools needed for both overlap. This paper, partly based on requirements analysis in the CoSy robotics project, surveys varieties of meta-cognition and draws attention to some types that appear to play a role in intelligent biological individuals (e.g. humans) and which could also help with practical engineering goals, but seem not to have been noticed by most researchers in the field. There are important implications for architectures and representations.

Keywords: cosy; irlab

Presenting some of the requirements for a truly helpful, as opposed to merely engaging (or annoying) artificial companion, with arguments as to why meeting those requirements is way beyond the current state of the art in AI.

Keywords: cosy; irlab

The full variety of powerful information-processing mechanisms 'discovered' by evolution has not yet been re-discovered by scientists and engineers. By attending closely to the diversity of biological phenomena, we may gain new insights into (a) how evolution happens, (b) what sorts of mechanisms, forms of representation, types of learning and development and types of architectures have evolved, (c) how to explain ill-understood aspects of human and animal intelligence, and (d) new useful mechanisms for artificial systems. We analyse tradeoffs common to both biological evolution and engineering design, and propose a kind of architecture that grows itself, using, among other things, genetically determined meta-competences that deploy powerful symbolic mechanisms to achieve various kinds of discontinuous learning, often through play and exploration, including development of an 'exosomatic' ontology, referring to things in the environment - in contrast with learning systems that discover only sensorimotor contingencies or adaptive mechanisms that make only minor modifications within a fixed architecture.

Keywords: cosy; irlab

There is still much to learn about the variety of types of learning and development in nature and the genetic and epigenetic mechanisms responsible for that variety. This paper is one of a collection exploring ideas about how to characterise that variety and what AI researchers, including robot designers, can learn from it. This requires us to understand important features of the environment. Some robots and animals can be pre-programmed with all the competences they will ever need (apart from fine tuning), whereas others will need powerful learning mechanisms. Instead of using only completely general learning mechanisms, some robots, like humans, need to start with deep, but widely applicable, implicit assumptions about the nature of the 3-D environment, about how to investigate it, about the nature of other information users in the environment and about good ways to learn about that environment, e.g. using creative play and exploration. One feature of such learning could be learning more about how to learn in that sort of environment. What is learnt initially about the environment is expressible in terms of an innate ontology, using innately determined forms of representation, but some learning will require extending the forms of representation and the ontology used. Further progress requires close collaboration between AI researchers, biologists studying animal cognition and biologists studying genetics and epigenetic mechanisms.

Keywords: cosy; irlab

Discussion of some of the relationships between (a) predicting physical, topological and geometrical consequences of motions and (b) predicting the changes in affordances that result from such motions, including both (b.1.) changes in action affordances (changes in what the agent can do in the environment) and (b.2.) changes in epistemic affordances, i.e. changes in the information available to the agent or changes in the ease of planning or deciding. It is suggested that in some circumstances the predictions can be based on processes operating on selected fragments of a 2-D representation of a 3-D scene (or a 2.5-D representation when occlusion is involved) and reasoning by manipulating the representation. Moreover, where uncertainty is a problem for prediction it is often due to the existence of a ``phase boundary'' between configurations where the prediction definitely gives one result and configurations where the prediction definitely gives another result. One way of reducing uncertainty is move an object (or even the viewing position) away from such a phase boundary. This sometimes allows simple, deterministic, geometric reasoning to be used, instead of much more complex and unreliable reasoning with probability distributions and expected utilities.

Keywords: cosy; irlab

Investigating the evolution of cognition requires an understanding of how to design working cognitive systems since there is very little direct evidence (no fossilised behaviours or thoughts). That claim is illustrated in relation to theories about the evolution of language. Almost everyone seems to have got things badly wrong by assuming that language must have started as primitive communication between individuals that gradually got more complex, and then later somehow got absorbed into cognitive systems. An alternative theory is presented here, namely that generalised languages (GLs) supporting (a) structural variability, (b) compositional semantics (generalised to include both diagrammatic syntaxes and contextual influences on semantics at every level) and (c) manipulability for reasoning, evolved first for various kinds of 'thinking', i.e. internal information processing. This is incosistent with many theories of the evolution of language. It is also inconsistent with Dennett's account of the evolution of consciousness in Content and Consciousness (1969).

Keywords: cosy; irlab

This paper extends three decades of work arguing that instead of focusing only on (adult) human minds, we should study many kinds of minds, natural and artificial, and try to understand the space containing all of them, by studying what they do, how they do it, and how the natural ones can be emulated in synthetic minds. That requires: (a) understanding sets of requirements that are met by different sorts of minds, i.e. the niches that they occupy, (b) understanding the space of possible designs, and (c) understanding the complex and varied relationships between requirements and designs. Attempts to model or explain any particular phenomenon, such as vision, emotion, learning, language use, or consciousness lead to muddle and confusion unless they are placed in that broader context. in part because current ontologies for specifying and comparing designs are inconsistent and inadequate. A methodology for making progress is summarised and a novel requirement proposed for human-like philosophical robots, namely that a single generic design, in addition to meeting many other more familiar requirements, should be capable of developing different and opposed viewpoints regarding philosophical questions about consciousness, and the so-called hard problem. No designs proposed so far come close.

Keywords: cosy; irlab

Introduction to key ideas of semantic models, implicit definitions and symbol tethering through theory tethering, providing a criticism concept empiricism, including its recently revived version, ``symbol grounding theor''. The idea of an axiom system having some models is explained, showing how the structure of a theory can give some semantic content to undefined symbols in that theory, making it unnecessary for all meanings to be derived bottom up from (grounded in) sensory experience, or sensory-motor contingencies. Although symbols need not be grounded, since they are mostly defined by the theory in which they are used, the theory does need to be ``tethered'', if it is to be capable of being used for predicting and explaining things that happen, or making plans for acting in the real world. These ideas were quite well developed by 20th Century philosophers of science, and I now both attempt to generalise those ideas to be applicable to theories expressed using non-logical representations (e.g. maps, diagrams, working models, etc.) and begin to show how they can be used in explaining how a baby or a robot, can develop new concepts that have some semantic content but are not definable in terms of previously understood concepts. There is still much work to be done, but what needs to be done to explain how intelligent robots might work, and how humans and other intelligent animals learn about the environment, is very different from most of what is going on in robotics and in child and animal psychology. The addition of new explanatory hypotheses is abduction. Normally abduction uses pre-existing symbols. The simultaneous introduction of new symbols and new axioms (ontology-extending abduction) generates a very difficult problem of controlling search.

Keywords: cosy; irlab

J&L refer only implicitly to aspects of cognitive competence that preceded both evolution of human language and language learning in children. These are important for evolution and development but need to be understood using the 'design-stance', which the book adopts only for molecular and genetic processes, not for behavioural and symbolic processes. Design-based analyses reveal more routes from genome to behaviour than J&L seem to have considered. This both points to gaps in our understanding of evolution and epigenetic processes, and may lead to possible ways of filling the gaps.

Keywords: cosy; irlab

This is a contribution to discussions regarding the construction of a research roadmap for future cognitive systems, including intelligent robots, in the context of the euCognition network, and the UKCRC Grand Challenge 5: Architecture of Brain and Mind. I have argued that in the context of trying either (a) to produce working systems to elucidate scientific questions about intelligent systems, or (b) to advance long term engineering objectives through advancing science, the task of coming up with a set of requirements that is sufficiently detailed to provide a basis for developing milestones and evaluation criteria is itself a hard research problem. One aspect of the problem is to provide an analysis of words and phrases that are commonly used to specify objectives, but whose meanings are very abstract and unclear, in particular words like ``robust''. ``flexible'', ``creative'' and ''autonomous''. This document argues that the words all share a feature that could be described as expressing a ''meta-requirement''. What that means is that none of them is directly associated with a set of features which, if found in an object or process or system, would justify the application of the label, or which can be used to derive design features. In other words the words express concepts that do not specify criteria for their instances though they do express criteria for deriving criteria. To derive criteria from the concepts more information is required, from which the criteria can be derived, in a systematic way that differs for each of the meta-criteria. Analyses of the words based on this idea are proposed. This is an exercise in analysis of logical topography. Subsequent work will need to provided detailed examples of the use of the various meta-criteria.

This presentation elaborates on 'The substratum of this experience is the mastery of a technique' (Wittgenstein) I try to show, with illustrative videos, that many 'techniques' are implicitly involved in ordinary experiences - and that the complexities grow as a child develops, extending its ontology and therefore the variety of affordances it can experience and use. I point out that there are two interpretations of sensorimotor contingencies, one intrasomatic (relating only the contents of sensory and motor signals at various levels of abstraction) the other extrasomatic (amodal, objective), referring to an environment that exists independently of whether and how it is experienced or acted on, and that the latter provides computational advantages in some cases, supporting a Kantian rather than a Humean view of knowledge and concepts. This also suggests a re-interpretation of mirror neurons as 'abstraction neurons'. What we are conscious of in the environment depends on the ontology we have available. A child whose ontology does not include the notion of boundary, or the notion of alignment of boundaries may not be able to replace a cut-out wooden picture in its recess, even if he knows which recess it should go in. Careful observation of children at various stages shows transitions that involve extensions of the available ontology, which must go along with development of suitable forms of representation and mechanisms for manipulating them, and an architecture that combines them all. Thus the substratum of the more sophisticated child's experience is mastery of many 'techniques', not just one as implied by Wittgenstein (who probably did not intend that). It is suggested that there are considerable differences between precocial species whose competences and architecture are mostly genetically determined and altricial species that develop most of their own competences e.g. through playful exploration, driven by meta-level bootstrapping mechanisms. Only when I started working in detail on requirements for a human-like robot able to manipulate 3-D objects using vision and an arm with gripper did I notice what should have been obvious long before, namely that structured objects have 'multi-strand' relationships not expressible simply as R(x, y), because the relation between x and y involves many relations between parts of x and parts of y. For a more detailed presentation of the resulting theory see COSY-PR-0505: A (Possibly) New Theory of Vision (PDF) Hence, motion of such structured objects involves 'multi-strand' (concurrent) processes. That is, many relationships change in parallel - e.g. faces, edges, corners of one block may all be changing their relationships to faces edges and corners of another (and things get more complex when objects are flexible, e.g. your hand peeling a banana or a sweater being put on a child). Thus seeing what you are doing in such cases can have a kind of complexity that appears not to have been noticed previously because of too much focus on simpler visual tasks like recognition and tracking. I'll show why we need to postulate mechanisms in which concurrent processes at different levels of abstraction, in partial registration with the optic array (NOT the retina, since saccades, etc., occur frequently) are represented. Nothing in AI comes close to modelling this, and it seems likely that it will be hard to explain in terms of known neural mechanisms. If the opportunity arises I'll try to explain some of the implications for human development, understanding of causation, and computational modelling, and spell out requirements to be addressed in future interdisciplinary research, explaining deep connections with Gibson's notion of affordance, and its generalisation to 'vicarious affordance'. The evolution of grasping devices that move independently of eyes (i.e. hands instead of mouth or beak) had profound implications - undermining claims about sensory-motor contingencies - also suggesting that mirror neurons should have been called 'abstraction neurons'. Some of the ideas are also sketched here: COSY-DP-0601 'Orthogonal Competences Acquired by Altricial Species' A critique of common assumptions about 'sensorimotor contingencies' is presented, including making a distinction between somatic (internal) and exosomatic (external) ontologies. Too many people expect too much to come from the somatic (intrasomatic) variety - including knowledge of sensorimotor contingencies, a notion criticised in http://www.cs.bham.ac.uk/research/projects/cosy/papers/#dp0603 Requirements for 'fully deliberative' systems are analysed in http://www.cs.bham.ac.uk/research/projects/cosy/papers/#dp0604

Keywords: cosy; irlab

For some decades, researchers in AI and Cognitive Science have talked about animals or machines as having 'deliberative' capabilities. In my own work, I have, for 10 years or more, been contrasting 'reactive', 'deliberative' and 'meta-management' (sometimes referred to as 'reflective') capabilities (categories within which many further subdivisions are possible). The key feature of a deliberative system is the ability to represent and reason about, and to compare and evaluate, possible situations that do not exist, or are not known to exist, either because they are future possibilities, or because they are remote or hypothetical possibilities. That ability is analysed in more detail in the paper. In particular we see a need for a fully deliberative system to be able to construct representations of possible states of affairs of varying structure and varying complexity, using at least one formalism with compositional semantics, in mechanisms that allow two or more such structures to be constructed, analysed and compared, where the result of comparing them may be another complex structure describing the pros and cons. Additional related requirements are described. Much of this is a presentation of old ideas: going back to work by Minsky, Evans, Winston, and many others during the 1960s and early 1970s. Although I have taken all that for granted for many years, gradually I have come to realise that the ideas are not all widely understood and the word 'deliberative' is used in different ways, partly because people have not analysed the variety of cases in a deep way that is widely shared. I try to contrast 'fully deliberative' systems with much simpler kinds of 'proto-deliberative' systems, while allowing for many simpler cases in between (including intermediate states through which evolutionary trajectories have passed, and some through which developing individuals may pass). It is important that in a complex architecture with many components there are different kinds of subsets, and in my work I have characterised three (partly overlapping) main subsets, which differ in their evolutionary history, in their spread amongst other animals besides humans, and in their functionality (though they may overlap in the kinds of mechanisms they use).

Keywords: cosy; irlab

I have been trying, with limited success, to get people to understand the importance (for theories of mental processes including learning, perception, reasoning and communication), of a distinction between learning about sensorimotor contingencies (concerned with relations between states, events and processes within an animal or machine) and learning about objective condition-consequence contingencies (concerned with relations between states, events and processes in the environment). The distinction is important for theories of infant development, for the design of robots that act in and learn about their environment, and for philosophical and other theories of embodied cognition. The document is a discussion note listing some possible reasons why the different sorts of people fail to appreciate the distinction (e.g. they are concept empiricists, or they already use the phrase 'sensorimotor' so broadly as to cover both categories, not realising the importance of the subdivision they are not attending to). Various examples are presented that illustrate the distinction and its importance. This elaborates on some of the points made in the discussion document on 'Orthogonal Recombinable Competences'

Keywords: cosy; irlab

This symposium is inspired by UKCRC Research Grand Challenge 5: GC5: Architecture of Brain and Mind. The aim of GC5 is to provoke unified discussion of long term research goals in AI, Cognitive Science, and related disciplines, especially goals concerned with giving computers a useful and general subset of human capabilities, implemented in a biologically inspired fashion. The symposium can also be seen as part of a series of related events attempting to promote a high-level long-term vision of achievable scientific goals of AI/Cognitive Science, including The DAM (Designing an Mind) Symposium at AISB'00 (Davis, 2005), the Tutorial on Philosophical Foundations of AI at IJCAI'01 (Sloman and Scheutz, 2001), the St. Thomas symposium in 2002 (Minsky et al., 2004), and the IJCAI'05 Tutorial on Learning and Representation in Animals and Robots (Sloman and Schiele, 2005). It presents themes central to the EC-funded Cognitive Systems initiative including the CoSy project which is part of that initiative, whose members have helped to organise this symposium, and the euCognition project which is funding this meeting. A common feature is the focus on scientific goals rather than useful applications though implementation of working systems is central to the proposed methodology. This introduction to the symposium provides some background and highlights some of the major problems to be overcome.

Keywords: cosy; irlab

A child or baby robot that has to manipulate 3-D objects in its environment would face a combinatorial explosion if all possible situations have to be learnt about separately. This could take evolutionary time-scales. It is conjectured that humans and some other altricial species instead use innate mechanisms for decomposing situations into components that can be explicitly learnt about, and stored in such a way that the competence can be re-used in combination with other learnt competences, in perceiving novel situations and performing novel actions. That includes learning about kinds of surface fragments (e.g. varieties of curvature and surface discontinuities), kinds of surface properties (e.g. texture, hardness, etc.), kinds of material (rigid, flexible in different ways), kinds of objects composed of materials and shapes, kinds of relationships, kinds of changes in relationships, kinds of causal connections between changes. These need to be represented in a manner that is independent of precise sense-data when they are perceived, or sensorimotor contingencies, so that knowledge about them can be used in planning future actions, thinking about the past, and comparing actions using different hands, or hands or mouth in different positions. This implies a use of 'objective' representations (e.g. of 3-D structure) which can then also be used in perceiving 'vicarious' affordances (for others). An implication is that mirror neurons should have been called 'abstraction neurons'. There are many other implications, for robotics, psychology and neuroscience.

Keywords: cosy; irlab

Since the 1970s AI as a science has progressively fragmented into many activities that are very narrowly focused. It is not clear that work done within these fragments can be combined in the design of a human-like integrated system - long held as one of the goals of AI as science. A strategy is proposed for reintegrating AI based around a backward-chaining analysis to produce a roadmap with partially ordered milestones, based on detailed scenarios, that everyone can agree are worth achieving, even when they disagree about means. This is a summary of ideas being developed within the CoSy project about how to plan long term research using a partially ordered network of scenarios and a grid of requirements for competences.

Keywords: cosy; irlab

Test domains for AI can have a deep impact on research. The polyflap domain is proposed for testing complex AI theories about architectures, mechanisms and forms of representation involved in features of human and animal intelligence that evolved to enable perception, action, and learning in diverse environments containing things that we can perceive and manipulate, and many complex processes involving objects that differ in shape, materials, causal properties, and relations to one another. We need a test environment that is rich enough to provide some of that variety of structures, processes and affordances, yet simple enough to be within reach of robotics research in the not too distant future.

Keywords: cosy; irlab

This is part of an attempt to explain why our ability to perceive and produce processes involving 3-D objects of varying shape was so important for the evolution of the human mind, at the same time as pointing out what is wrong with most of the stuff that gets written about the importance of embodiment. I suspect this reinvents some of Piaget's ideas. It is consistent with some of the main themes of McCarthy's 'The Well-Designed Child'. I have recently (April 2006) discovered many connections with the book The Infant's World by Philippe Rochat (2001). The main idea is that children acquire types of information that are orthogonal insofar as they relate to aspects of things or situations in the environment that can vary (nearly) independently, e.g. kinds of stuff things are made of, kinds of local surface features, kinds of relations between things, kinds of whole objects (composed of stuff with specific surfaces and parts with multiple relations), kinds of processes, etc. The competences are also recombinable insofar as they can be used in perceiving or producing novel structures and processes. The requirement for re-use in novel combinations seems to impose strong requirements on the forms of representation used. The recombination is predictive: you can imagine many details of the process of trying to put on a shirt made of paper, or lead, or the process of sitting on a chair made of butter, even if you have never encountered such a thing. The competences can involve different levels of abstraction. E.g. grasping something with your teeth and with finger and thumb are extremely different as regards sensory input and motor signals. But an animal that can represent what is common to both has a powerful re-usable abstraction that can also be applied to grasping done by another person (e.g. a child who may need help) or grasping done by machines. I conclude that the so-called 'mirror neurones' should have been called 'abstraction neurones', and that might have prevented much confusion (e.g. about imitation). Powerful innate mechanisms are needed for acquiring such competences through play and exploration. Very few species can do it. As far as I can tell there is nothing in AI that accounts for this, and no known neural mechanisms. (Data-mining techniques can be viewed as deriving separate 'competences' from large amounts of data, but as far as I know those techniques and the forms of representation chosen are not designed to support creative recombination, like solving a problem by inventing something new involving previously known kinds of motion, of shape, and of physical stuff) I'd be interested to know if there's anything implemented by anyone in AI that models such learning. I have not yet found AI literature identifying the problem, though it's possible that I've read something in the past which I had forgotten. It's also likely that someone has used a different label for this notion of orthogonal competences, which is why I failed to find previous work on this.

Keywords: cosy; irlab

This paper discusses some of the long term objectives of cognitive robotics and some of the requirements for meeting those objectives that are still a very long way off. These include requirements for visual perception, for architectures, for kinds of learning, and for innate competences needed to drive learning and development in a variety of different environments. The work arises mainly out of research on requirements for forms of representation and architectures within the PlayMate scenario, which is a scenario concerned with a robot that perceives, interacts with and talks about 3-D objects on a tabletop, one of the scenarios in the EC-funded CoSy Robotics project.

Keywords: cosy; irlab

Several high level methodological debates among AI researchers, linguists, psychologists and philosophers, appear to be endless, e.g. about the need for and nature of representations, about the role of symbolic processes, about embodiment, about situatedness, about whether symbol-grounding is needed, and about whether a robot needs any knowledge at birth or can start simply with a powerful learning mechanism. Consideration of the variety of capabilities and development patterns on the precocial-altricial spectrum in biological organisms will help us to see these debates in a new light. It seems that after evolution discovered how to make physical bodies that grow themselves, it discovered how to make virtual machines that grow themselves. Researchers attempting to design human-like, chimp-like or crow-like intelligent robots will need to understand how. Whether computers as we know them can provide the infrastructure for such systems is a separate question.

Keywords: cosy; irlab

A two-day tutorial was held in The University of Edinburgh on 30th and 31st July 2005 at IJCAI 2005 on REPRESENTATION AND LEARNING IN ROBOTS AND ANIMALS.

Keywords: cosy; irlab

This discussion note suggests that some forms of expression that are apparently vague, inviting interpretations of their meaning in terms of probability distributions, would be better construed as having a different form of semantics, namely specifying an 'higher order' function from contexts to truth-conditions. So statements made using them have a two level semantics. The first level specifies the function, which has to be applied to arguments extracted from the context, which may be linguistic or non linguistic, including the purpose of the communication. Then when that function is applied to the arguments the result is a specification of truth-conditions. This can be extended to how questions and imperatives using those expressions also need to be interpreted. I first proposed this sort of interpretation for 'better' in 1969 in How to derive 'Better' from 'is', American Phil. Quarterly Vol 6, pp43-52, but I think the phenomenon is much more common than has been realised. I try to show how the use of such things can be predicted on the basis of Grice's theory of communication, and draw some conclusions regarding the evolution of language, and the relations between linguistic and non-linguistic mental functions. From this viewpoint, communication is collaborative problem-solving, not the transmission and decoding of some signal, and the ability to use a language is just a special case of a more general ability to solve problems by combining different kinds of competence. This is related to the amazing invention of a sign language by Nicaraguan deaf children and to arguments for the evolution of inner structured languages prior to the evolution of language for communication. This is a discussion paper and everything is still tentative.

Keywords: cosy; irlab

It is often thought that there is one key design principle or at best a small set of design principles, underlying the success of biological organisms. Candidates include neural nets, `swarm intelligence', evolutionary computation, dynamical systems, particular types of architecture or use of a powerful uniform learning mechanism, e.g. reinforcement learning. All of those support types of self-organising, self-modifying behaviours. But we are nowhere near understanding the full variety of powerful information-processing principles `discovered' by evolution. By attending closely to the diversity of biological phenomena we may gain key insights into (a) how evolution happens, (b) what sorts of mechanisms, forms of representation, types of learning and development and types of architectures have evolved, (c) how to explain ill-understood aspects of human and animal intelligence, and (d) new useful mechanisms for artificial systems.

Keywords: cosy; irlab

We report on some of the hard unsolved problems we have identified on the basis of detailed analysis of some of the processes that will have to occur when the PlayMate and Explorer robots perform their tasks. The analysis used our scenario-driven research methodology. We introduce some preliminary characterisations of the key problems and some preliminary ideas for dealing with them, inspired in part by studies of cognition in humans and other animals. We confirm the conjecture in the CoSy proposal that various kinds of representations are required for different sorts of sub-mechanisms (including for instance representations concerned with planning complex sequences of actions and representations used in producing and controlling fast and fluent movements). The different representations are in part related to different ontologies, since different sub-mechanisms acquire, manipulate and use information about different subject-matter. A substantial part of this report is therefore concerned with first draft, incomplete, ontologies that we expect our robots will need, some parts of which the robots will have to develop for themselves, especially ontologies concerned with objects and processes that have quite complex structures involving multi-strand relationships. A particularly important requirement for a robot with 3-D manipulation capabilities is the ability to perceive and understand what we have labelled 'multi-strand' relationships (where multiple parts of complex objects are related, e.g. edges, corners and faces of two cubes), which cause multi-strand processes to occur when objects are moved, with several different relationships changing in parallel. Perceiving such processes seems to require something like a simulation process to occur. Moreover, this needs to happen at different levels of abstraction concurrently (some continuous, with high or low resolution, and some discrete capturing 'qualitative' structural changes), for the same reason as many researchers have claimed that perception of static scenes involves multiple-levels of abstraction. So we conclude that our robot is likely to require an architecture and mechanisms that support several concurrent simulations at different levels of abstraction, in registration with one another and (where appropriate) with the sensory data. It seems that a mechanism like this can also implement some of what is often referred to as spatial or visual reasoning, and could be relevant to perception and understanding of affordances. We consider in particular requirements for a pre-linguistic robot that is capable of perceiving, acting in and to some extent reasoning about the world before being able to talk about it, and raise questions about how that might relate to learning that adds linguistic competence. We note that in animals there is wide variation between species that start with most of the ontology and representational competence they will ever need and those that somehow learn or develop what they need and suggest that further study of those cases may yield clues regarding options for robots of different kinds. Most of this work has not yet been published. This is work-in-progress and much of it remains to be expanded, clarified and polished.

Keywords: cosy; irlab

Illustration of some of the requirements for a vision system capable of being used in a robot that manipulates 3-D objects. The pictures displayed here are very easy for humans to understand not merely insofar as they recognise the objects depicted, in spite of poor quality and poor resolution, but also because humans easily see various ways in which the objects can and cannot be grasped, and can plan a sequence of moves to transform one of the configurations presented into another.