The PlayMate: An Object Manipulation Scenario

Videos

Year 1

• Visual Learning

  • "This is a green box." [01'50/4.2MB]
    Fusing vision and language to enable object name learning.

• Spatial Relationships

  • "The cow is to the left of the pink pig." (.mov) [03'49/13.1MB]
    Talking about projective spatial relations between objects in a visual scene.
  • "The black cow is near the brown horse." (.mov) [01'45/6MB]
    Talking about topological spatial relations between objects in a visual scene.

• Context Sensitive Saliency

  • "Do you see the blue mug?" (.mov) [02'37/5.4MB]
    Using context-sensitive visual saliency to find objects in a scene.

Year 2

• Cross Modal Ontology Learning

  • "This is a small blue thing." [02'03/31.9MB]
    Learning a mapping between the names of properties (colour, size) and the appearance of an object.

• Language driven manipulation

  • "Put the blue thing to the right of the red thing." [01'21/19.8MB]
    The robot interprets the instruction, plans and executes it.
  • "Put the blue things to the left of the red thing." [01'24/20.6MB]
    The robot realises it need only move one object to satisfy the request.

• Action learning and recognition

  • Action recognition (.avi). [01'10/52.4MB]
    Using a learned hierarchical graphical model of action to recognise push and pull actions with a variety of objects.

Year 3

• Continuous cross modal learning and unlearning
  • "These are elongated objects." [03'19/71.7MB]
    Continuous learning of associations between automatically extracted visual features and basic visual concepts (colour, shape) in the integrated system.
  • "This object is not blue." [00'10/2.15MB]
    Unlearning - updating/correcting the current representations with negative information (negation).
  • "Put the blue thing to the right of the red thing." [02'56/8.15MB]
    
    
• PlayMate, Year 3
  • Playmate Y3 system: The PlayMate can move objects around on request and can cope if things don't turn out as planned.

Year 4

• PlayMate, Year 4
  • Playmate Y4 system: The PlayMate can watch a human playing simple games and answer questions about what it sees.