The ideas for TACMAN were initiated during a retreat on grasping, which was organised by the consortium leader in August, 2012. During the last day of the retreat, the group of researchers tried to evaluate which parts of grasp, grip, and manipulate were understood or solved. We concluded that manipulation was one of the most challenging problems, and that robots have not even come near to solving this.
The retreat, to which selected researchers working on human and robot hand use had been invited, much facilitated the selection of the partners for the project; all partners but one were present at the retreat, and already then brainstormed on the contents of TACMAN. Consequently, we were able to set up the project in such a way that all expertise, required to tackle the tasks of TACMAN, is optimally represented in the consortium. Each partner has a rather well-defined position in the project, and there is very limited overlap between the partners. Furthermore, the chosen partners are internationally recognised leaders in their respective fields; in short, we were able to put together the best team for the job.
To summarise the assignments and expertise we need:
- expertise on human hand and tactile sensing physiology. Umea University is internationally recognised as leading the field;
- expertise on robot learning methodologies. The TU Darmstadt runs one of the very few labs in Europe focussing on robot learning, and leading on their application on real hardware;
- expertise on skin data processing and machine learning. TU München is a leading researcher on the integration of neuroscience and robotics, successfully demonstrated machine learning approaches on real-world robot data.
- expertise on integrated robotic systems. For this, IIT is clearly recognised as the leading European institution. Furthermore, IIT has extensive expertise on robotic grasping with their iCub platform, as well as with developing and exploiting its tactile sensors;
- expertise on tactile sensors, available at IIT and TU München.
Technische Universität München, Chair of Robotics and Embedded Systems
Partner TUM is one of the largest and in terms of third-party funding, one of the most successful universities in Germany. In 2006 it was appointed to be one of only three “top-level elite universities” by the German National Science Foundation (DFG). There are numerous interdepartmental, coordinated activities taking place in the fields of robotics, mechatronics, control theory, biology etc.
The Department of Informatics is one of the largest and most well-equipped computer science departments in Germany and has a long tradition of industry collaboration. The “Robotics and Embedded Systems” Chair in the Department of Informatics is headed by Prof. Alois Knoll.
The Biomimetic Robotics and Machine Learning (BRML) lab is headed by Patrick van der Smagt. His lab focuses on methods to map high-dimensional non-linear data within a control process. For this, parametric models are used which employ non-linear mappings. These are combined into deep and recurrent architectures which are subsequently optimised with classical and novel optimisation techniques on a wide variety of objectives.
Patrick van der Smagt
holds a professorship at TUM and a position at TUM-fortiss. He has been scientific coordinator of the following EC projects: SENSOPAC (FP6), The Hand Embodied (FP7); as well as coordinator of STIFF (FP7) and co-coordinator of VIACTORS (FP7). He has collaborated in various other European and national projects, and is active in the Human Brain Project. He is the author of a large number of scientific papers on robotics, machine learning, sensors, and neuroscience, and holds several tens of patens in these areas. In 2012, he received the prestigious Schrödinger Award from the Helmholtz Gesellschaft and in 2013 the Harvard Medical School/MGH Martin Research Prize.
is a Ph.D. student at TU München. His research focuses on unsupervised learning and variational inference. He is also interested in the application of information theory to stochastic control especially for intrinsically motivated movement generation.
is a Ph.D. student at TU München. His research focuses on the inference of lower-dimensional state representations for robust control mechanisms. Previously, he has successfully applied such methods for anomaly detection in robot time-series.
is a Ph.D. student at TU München. He is focus on machine learning of feature detection from high dimensional data and dynamical movement applied to human and robot. He is also interested in human grip force analyzation.
University of Southern California
Gerald E. Loeb
is an external advisory member of TU München's team located at the University of Southern California
. Professor Loeb started the Medical Device Development Facility when he arrived at USC in 1999. Prior to that he was Chief of the Biokinesiology Section at the National Institutes of Health and then Professor of Physiology and Director of the Biomedical Engineering Unit at Queens University in Canada. He was one of the inventors of the cochlear implant and served as Chief Scientist for Advanced Bionics Corp., one of the leading manufacturers. Dr. Loeb is an inventor on 54 issued US patents and author of over 200 scientific papers. Most of Dr. Loebs research team works on sensorimotor control of biological and mechatronic limbs. Their work on prosthetic limbs (a type of telerobot) led the development and commercialisation of the BioTac sensor. Dr. Loebs research team developed BION injectable neuromuscular stimulators and has been conducting several pilot clinical trials. His lab at USC is developing computer models of musculoskeletal mechanics and the interneuronal circuitry of the spinal cord, which facilitates control and learning of voluntary motor behaviours by the brain and robotic controllers. These projects build on Dr. Loebs long-standing basic research into the properties and natural activities of muscles, motoneurons, proprioceptors and spinal reflexes.
Technische Universität Darmstadt, Intelligent Autonomous Systems
The Technische Universität Darmstadt (TU Darmstadt) is one of Germany’s leading technical universities, and also Germany’s first fully autonomous university. TU Darmstadt) has a state funded budget of 270 million Euros (2010, incl. building funds) and currently participates in 68 FP7 projects.
The Intelligent Autonomous Systems (IAS) institute of TU Darmstadt) is considered one of the strongest robot learning groups in Europe with expertise ranging from the development of novel machine learning methods (e.g., novel reinforcement learning approaches, policy search, imitation learning, regression approaches, etc.) over autonomous robotics (e.g., robot learning architectures, motor skill representation, acquisition & refinement, grasping, manipulation, nonlinear control, operational space control, robot table tennis, legged locomotion) up to the design of biomimetic motor control systems and brain-robot interfaces. IAS members are well-known researchers both in the machine learning and the robotics community. IAS currently participates in the EU projects GeRT (2010–2013), CompLACS (2011–2015) and CoDyCo (2013–2017).
is a full professor of Computer Science, head of IAS as well as Senior Research Scientist at the Max Planck Institute for Intelligent Systems. Jan Peters holds four masters degrees in computer science, electrical and mechanical engineering from TUM, USC & FernUni Hagen as well as a Ph.D. from USC. He has received numerous awards, the most recent awards include the 2013 Early Career Award of the IEEE Robotics & Automation Society and the 2013 INNS Young Investigator Award.
is a postdoctoral researcher at IAS. He is the TACMAN team leader for TU Darmstadt. Elmar received his PhD in computer science in 2014 from the Graz University of Technology. During his Ph.D., he worked on reinforcement learning algorithms for motor planning using probabilistic inference, biologically inspired movement primitive representations based on muscle synergies, and investigated how networks of spiking neurons can solve motor control and motor planning problems. He is also the CoDyCo
team leader for TU Darmstadt. In CoDyCo he works on the modelling of human motor control and on robotic implementations. For more information visit his website here
is a Ph.D. Student at IAS under the supervision of Jan Peters. He has a M.Sc. on Electrical and Computer engineering from Instituto Superior Técnico. His general research interests are machine learning and control systems applied to robotics. More specifically he is also interested in feature learning and pattern recognition strategies while his work focuses on Bayesian Optimisation for robot grasping through the use of tactile and visual feedback.
Herke van Hoof
performs research towards his PhD at the IAS institute. His research interests are active learning and reinforcement learning, mainly applied to robots. Tactile feedback for grasping provides specific challenges, among others high dimensionality. He is interested in using Bayesian methods to deal with uncertainty in a robust manner. Another interest is exploration needed to find optimal policies, for example by finding stochastic policies that include optimal exploration noise, or actively selecting informative actions.
is a Ph.D. Student at IAS under the supervision of Jan Peters. He received his master degree in Computer Science (with honors) from the Technische Universitaet Darmstadt in 2015. His research interests are biologically-inspired machine learning concepts applied to robotics. His focus lies on tactile feature learning for grasping using probabilistic latent variable models. For more information visit his website here
Somatosensory and sensorimotor research have been pursued at the Physiology Section, Department of Integrative Medical Biology, Umea ̊ University since late 1960s. The research address fundamental issues pertaining to dexterous manipulation in humans from the level of single neurons to the level of brain circuitries involved in the control of the human hand (fMRI). To this end numerous techniques have been developed and adapted by highly competent engineers in collaboration with senior researchers: complex instrumented test objects with multi-axial force measurements, 3D position sensing, TMS and rTMS, standard neurophysiological procedures such as EMG and EEG, microneurography, servo mechanism, and more. Using experimental these experimental tools behavioural and neurophysiological questions are addressed and the results obtained are regularly published in high impact journals. This research has been well funded by the Swedish Research Council for decades, by various EC projects for more than 10 years but also other international funding agencies.
Benoni B. Edin
received his M.D. in 1983 and his Ph.D. in 1988. After a 2 year postdoc period at the Department of Neurology, Wisconsin University, Madison, WI he returned to Umea ̊ University and has since then been involved in neurophysiological research and teaching physiology. He was 2009 appointed professor of physiology. His main research interests have revolved around muscle spindle and hairy skin receptors in humans studied behaviourally and with microneurography. His work has been cited >1,200 times (h-index of 19; averaged citation per article 32): http://www.researcherid.com/rid/D-1057-2009
received a Ph.D. in Health Science from the Norwegian University of Science and Technology (Trondheim, Norway) and a Ph.D. in Human Movement Science from the University of Nice - Sophia Antipolis (Nice, France) in 2013. Aude-Clémence Doix currently holds a postdoctoral position at the Department of Integrative Medical Biology at the University of Umeå where she works on the sensorimotor control in humans.
Istituto Italiano di Tecnologia
The Istituto Italiano di Tecnologia
(Italian Institute of Technology, IIT) is a Foundation established jointly by the Italian Ministry of Education, University and Research and the Ministry of Economy and Finance to promote excellence in basic and applied research. The research plan of the institute focuses on Humanoid Technologies & Robotics, Neuroscience and Cognition, Nanotechnology and Materials. The Institute has a staff of about 1200 people and 10 sites, the central research lab being located in Genova. IIT has a large experience with the management of large research projects and has been involved in more than 70 EU funded projects in the last 6 years.
Participation to the project will be through the “iCub Facility”. The iCub Facility is a new IIT endeavour to support the integration and possibly the transfer to industry of the iCub and generally the humanoid-related technologies, defining in this way the road map for the iCub evolution. Specifically, its aim is to coordinate and agree all the technical aspects related to the integration, compatibility and quality of the new technologies incorporated in the iCub as well as to coordinate the incorporation of the new software produced by the iCub community at large. The Facility can count on a staff of about 50 people, more than half of them with technical roles in design and engineering.
Current activities of the Facility range from the mechatronics of the iCub to its software infrastructure. Particular focus has been devoted lately to whole body control, manipulation and tactile sensing through various EU projects (Xperience, RoboSKIN, Chris). Ongoing activities cover also vision and machine learning through various internal and international collaborations (e.g. MIT).
Prof. Giorgio Metta
is Senior Researcher at IIT and director of the iCub Facility. He is also Deputy Director at IIT, delegate to the international relations of the institute. Giorgio Metta is Professor of Cognitive Robotics at the University of Plymouth. He was one of the coordinators of the FP6 IST 004370 RobotCub project which will be providing one of the robotic platforms to the project (the iCub). Giorgio Metta has been active in the field of Cognitive Robotics for the past 15 years resulting in about 200 peer-reviewed publications. His main interests are at the intersection of robotics and neuroscience, in particular, with regards to the development of sensorimotor coordination. He has been PI in several PF7 projects. http://www.iit.it/giorgio-metta
received his degree in Electronic Engineering (with honours) in 2000 and Ph.D. in Robotics in 2004 from the University of Genoa. He was later postdoctoral researcher at the MIT CSAIL. Presently he is Team Leader at the IIT. In the past ten years Lorenzo Natale worked on various humanoid platforms. His research interests range from sensorimotor learning and perception to software archi- tectures for robotics. He has participated to several EU-funded projects (MIRROR, CogVis, ADAPT, RobotCub, RoboSKIN and CHRIS). http://www.iit.it/lorenzo-natale