Toward high-performance, memory-efficient, and fast reinforcement learning–Lessons from decision neuroscience

Recent insights from decision neuroscience raise hope for the development of intelligent brain-inspired solutions to robot learning in real dynamic environments full of noise and unpredictability.

Source: Sciencemag.org – Science Robotics Latest Content

Learning ambidextrous robot grasping policies

Universal picking (UP), or reliable robot grasping of a diverse range of novel objects from heaps, is a grand challenge for e-commerce order fulfillment, manufacturing, inspection, and home service robots. Optimizing the rate, reliability, and range of UP is difficult due to inherent uncertainty in sensing, control, and contact physics. This paper explores “ambidextrous” robot grasping, where two or more heterogeneous grippers are used. We present Dexterity Network (Dex-Net) 4.0, a substantial extension to previous versions of Dex-Net that learns policies for a given set of grippers by training on synthetic datasets using domain randomization with analytic models of physics and geometry. We train policies for a parallel-jaw and a vacuum-based suction cup gripper on 5 million synthetic depth images, grasps, and rewards generated from heaps of three-dimensional objects. On a physical robot with two grippers, the Dex-Net 4.0 policy consistently clears bins of up to 25 novel objects with reliability greater than 95% at a rate of more than 300 mean picks per hour.

Source: Sciencemag.org – Science Robotics Latest Content

Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programs

Humans can infer concepts from image pairs and apply those in the physical world in a completely different setting, enabling tasks like IKEA assembly from diagrams. If robots could represent and infer high-level concepts, then it would notably improve their ability to understand our intent and to transfer tasks between different environments. To that end, we introduce a computational framework that replicates aspects of human concept learning. Concepts are represented as programs on a computer architecture consisting of a visual perception system, working memory, and action controller. The instruction set of this cognitive computer has commands for parsing a visual scene, directing gaze and attention, imagining new objects, manipulating the contents of a visual working memory, and controlling arm movement. Inferring a concept corresponds to inducing a program that can transform the input to the output. Some concepts require the use of imagination and recursion. Previously learned concepts simplify the learning of subsequent, more elaborate concepts and create a hierarchy of abstractions. We demonstrate how a robot can use these abstractions to interpret novel concepts presented to it as schematic images and then apply those concepts in very different situations. By bringing cognitive science ideas on mental imagery, perceptual symbols, embodied cognition, and deictic mechanisms into the realm of machine learning, our work brings us closer to the goal of building robots that have interpretable representations and common sense.

Source: Sciencemag.org – Science Robotics Latest Content

Inverted and vertical climbing of a quadrupedal microrobot using electroadhesion

The ability to climb greatly increases the reachable workspace of terrestrial robots, improving their utility for inspection and exploration tasks. This is particularly desirable for small (millimeter-scale) legged robots operating in confined environments. This paper presents a 1.48-gram and 4.5-centimeter-long tethered quadrupedal microrobot, the Harvard Ambulatory MicroRobot with Electroadhesion (HAMR-E). The design of HAMR-E enables precise leg motions and voltage-controlled electroadhesion for repeatable and reliable climbing of inverted and vertical surfaces. The innovations that enable this behavior are an integrated leg structure with electroadhesive pads and passive alignment ankles and a parametric tripedal crawling gait. At a relatively low adhesion voltage of 250 volts, HAMR-E achieves speeds up to 1.2 (4.6) millimeters per second and can ambulate for a maximum of 215 (162) steps during vertical (inverted) locomotion. Furthermore, HAMR-E still retains the ability for high-speed locomotion at 140 millimeters per second on horizontal surfaces. As a demonstration of its potential for industrial applications, such as in situ inspection of high-value assets, we show that HAMR-E is capable of achieving open-loop, inverted locomotion inside a curved portion of a commercial jet engine.

Source: Sciencemag.org – Science Robotics Latest Content

An anthropomorphic soft skeleton hand exploiting conditional models for piano playing

The development of robotic manipulators and hands that show dexterity, adaptability, and subtle behavior comparable to human hands is an unsolved research challenge. In this article, we considered the passive dynamics of mechanically complex systems, such as a skeleton hand, as an approach to improving adaptability, dexterity, and richness of behavioral diversity of such robotic manipulators. With the use of state-of-the-art multimaterial three-dimensional printing technologies, it is possible to design and construct complex passive structures, namely, a complex anthropomorphic skeleton hand that shows anisotropic mechanical stiffness. We introduce a concept, termed the “conditional model,” that exploits the anisotropic stiffness of complex soft-rigid hybrid systems. In this approach, the physical configuration, environment conditions, and conditional actuation (applied actuation) resulted in an observable conditional model, allowing joint actuation through passivity-based dynamic interactions. The conditional model approach allowed the physical configuration and actuation to be altered, enabling a single skeleton hand to perform three different phrases of piano music with varying styles and forms and facilitating improved dynamic behaviors and interactions with the piano over those achievable with a rigid end effector.

Source: Sciencemag.org – Science Robotics Latest Content

Soft wall-climbing robots

Existing robots capable of climbing walls mostly rely on rigid actuators such as electric motors, but soft wall-climbing robots based on muscle-like actuators have not yet been achieved. Here, we report a tethered soft robot capable of climbing walls made of wood, paper, and glass at 90° with a speed of up to 0.75 body length per second and multimodal locomotion, including climbing, crawling, and turning. This soft wall-climbing robot is enabled by (i) dielectric-elastomer artificial muscles that generate fast periodic deformation of the soft robotic body, (ii) electroadhesive feet that give spatiotemporally controlled adhesion of different parts of the robot on the wall, and (iii) a control strategy that synchronizes the body deformation and feet electroadhesion for stable climbing. We further demonstrate that our soft robot could carry a camera to take videos in a vertical tunnel, change its body height to navigate through a confined space, and follow a labyrinth-like planar trajectory. Our soft robot mimicked the vertical climbing capability and the agile adaptive motions exhibited by soft organisms.

Source: Sciencemag.org – Science Robotics Latest Content

Morphogenesis in robot swarms

Morphogenesis allows millions of cells to self-organize into intricate structures with a wide variety of functional shapes during embryonic development. This process emerges from local interactions of cells under the control of gene circuits that are identical in every cell, robust to intrinsic noise, and adaptable to changing environments. Constructing human technology with these properties presents an important opportunity in swarm robotic applications ranging from construction to exploration. Morphogenesis in nature may use two different approaches: hierarchical, top-down control or spontaneously self-organizing dynamics such as reaction-diffusion Turing patterns. Here, we provide a demonstration of purely self-organizing behaviors to create emergent morphologies in large swarms of real robots. The robots achieve this collective organization without any self-localization and instead rely entirely on local interactions with neighbors. Results show swarms of 300 robots that self-construct organic and adaptable shapes that are robust to damage. This is a step toward the emergence of functional shape formation in robot swarms following principles of self-organized morphogenetic engineering.

Source: Sciencemag.org – Science Robotics Latest Content