Warehouse robotic automation is attracting increasing interest. Robotics shows potential to impact logistics and efficient distribution of products. Prior successful commercial ventures, such as Kiva Systems, which was acquired and adopted by Amazon, have shown how the industrial landscape can quickly change through the adoption of emergent automation technology. An example task in this domain, which requires further development for successful adoption by industry, corresponds to picking and placing items from storage units, including shelves, bins, and boxes. This task is the focus of a robotics competition, the Amazon Picking Challenge (APC), which brings together multiple academic and industrial teams from around the world.

The goal of this workshop is to bring together existing technologies and current needs for picking and transportation in warehouse automation. We provide industrial representatives the opportunity to describe their needs, and academic representatives to describe tools and capabilities that they can provide. More importantly, this workshop is an opportunity to get to know people interested in the are and foster collaboration to improve the state­-of-­the-­art in warehouse automation. We also give an opportunity to Amazon and Amazon Picking Challenge teams to describe learned lessons. We do this through a series of talks both from industrial and academic speakers, and a panel discussion to encourage interaction and discussion among participants.

Date: Sunday, Aug 21, 2016
Time: 8:30am - 6:00pm
Location: CASE 2016, Fort Worth, Texas, USA



Topics of Interest

  • Hardware platforms. What are appropriate hardware technologies to build efficient solutions for warehouse automation? We are interested in grippers, mobile bases, sensors, manipulators, etc.
  • Software architecture. What software components need to be integrated to achieve deployable systems in real warehouses? How robust and maintainable are different architectures?
  • Perception challenges. What are the specific challenges that arise in warehouse setups in terms of detecting, identifying and tracking products and related storage infrastrcuture? Are there emergent solutions in this domain?
  • Planning and control. How do we efficiently plan and control robotic motions that require contact for picking and transferring items? How do we ensure safety?
  • Manipulation challenges. How do robots reliably and autonomously grasp objects in clutter? What are the tradeoffs and benefits of different types of end­effectors and different methods for picking?

Summary Videos from the Amazon Picking Challenge 2016

Team Delft (summary video)
ACRV (summary video)
PFN (summary video)
Team NimbRo (summary video)
C^2M (summary video)
MIT-Princeton (summary video)

For APC teams: You can send us a short video describing your system to <awl2016.texas@gmail.com> and we will link it in the website.

Panel Discussion

Host: Kostas Bekris.
Participants: (left to right in video) Dinesh Manocha, Erik Nieves, Ulf Hartmann, Michael Ferguson, Joey Durham, Jianxiong Xiao, and Anton Milan.
Questions: (extended notes) (video) 

  • What frustrates in industry about the current state of research in manipulation? What would industry like to see more of? Is the Amazon Picking Challenge covering any gaps in current research areas?
  • Should academic research be focused on integration of existing components or should it be focused on developing new solutions?
  • Do vacuum based grasping strategies solve most manipulation challenges in warehouses?
  • What are the major roadblocks hindering robots from appearing in warehouses?
  • How will warehouses change as the capability of robots increase? E.g., Amazon Prime Air. How will this affect the development of new technologies?


8:40 Academic talk: Kostas BekrisRutgers University (talk video)
9:10 Industry talk: Ulf Hartmann, UniGripper (talk video)
9:40 Industry talk: Jianxiong Xiao, AutoX (talk video)
10:30 Academic talk: Peter Yu, MIT (talk video)
11:00 Industry talk: Remus Boca, ABB (talk video) 
11:30 Industry talk: Yasunori Kamiya, Preferred Networks (talk video) 
14:20 Industry talk: Erik Nieves, Plus One Robotics (talk video)
14:50 Academic talk: Dinesh Manocha, University of North Carolina Chapel Hill (talk video) 
15:20 Industry talk: Joey Durham, Amazon Robotics
16:10 Industry Talk: Michael Ferguson, Fetch Robotics (talk video)
16:40 Academic talk: Anton Milan, University of Bonn (talk video)


Speakers and Abstracts

From the Academic Community

  • Kostas Bekris, Associate Professor, Rutgers (talk video) 
    Lessons from the 1st Amazon Picking Challenge and Rutgers' Participation
    This talk will first summarize the results of a survey conducted among the 26 international teams that took place in the Amazon Picking Challenge, with the objective of retrieving items from warehouse shelves. This survey led into a T-ASE publication by many team organizers. The survey covered aspects such as each team's background, mechanism design, perception apparatus, planning and control approaches as well as software engineering practices. It also indicated trends on how to approach warehouse automation for logistic purposes in general.
    As part of this talk we will also highlight the architecture of Rutgers' participation in the competition and a public data set that we have made available in this context.
    Kostas Bekris is an Associate Professor of Computer Science at Rutgers University in New Jersey. He received his Ph.D in Computer Science from Rice University. He is working in robotics, specifically motion planning and coordination with applications in manipulation and novel platforms with complex dynamics. His research has been supported by NSF, DHS, DHS and NASA, including a NASA Early Career Faculty award.
  • Dinesh Manocha, Professor, University of North Carolina Chapel Hill (talk video) 
    Motion Planning for Industrial Robots and Warehouse Automation
    Algorithmic motion planning has been actively studied in robotics and related areas for more than three decades. In spite of considerable progress in terms of algorithmic techniques and applications, we need better planning systems that can deal with the challenges that arise in the context of industrial robots and warehouse automation. These include dealing with sensor data, environmental uncertainties, robustly finding a desired path between different poses, realtime computations, and the safe trajectory planning in the presence of humans.
    In this talk, we give a brief overview of our recent work to handle some of the problems. These include new optimization based methods that can compute smooth and collision-free trajectories for high DOF robots. We exploit the parallel capabilities of current CPUs and GPUs for realtime computation, and present new techniques for probabilities collision detection to handle environment uncertainties. We will demonstrate their application in developing a system, DoraPicker, an autonomous picking system. Finally, we will also present some preliminary results related to safe motion planning for robots working with or next to humans.
    Dinesh Manocha is currently the Phi Delta Theta/Mason Distinguished Professor of Computer Science at the University of North Carolina at Chapel Hill. He received his Ph.D. in Computer Science at the University of California at Berkeley 1992. Along with his students, Manocha has also received 14 best paper awards at the leading conferences. He has published more than 400 papers and some of the software systems related to collision detection, GPU-based algorithms and geometric computing developed by his group have been downloaded by more than 150,000 users and are widely used in the industry. He has supervised 30 Ph.D. dissertations and is a fellow of ACM, AAAS, and IEEE. He received Distinguished Alumni Award from Indian Institute of Technology, Delhi.
  • Anton Milan, Team Nimbro Member and Postdoc in the Autonomous Intelligent Systems group of University of Bonn (talk video) 
    Amazon Picking Challenge 2016: Team NimbRo of University of Bonn
    Automation in warehouses is becoming increasingly important in order to relieve humans from mundane and heavy tasks. This talk will present Team NimbRo's successful solution for this year's Amazon Picking Challenge. We will first give a broad overview of the entire system and then focus on two challenging aspects. First, motion generation using a highly flexible IK-based keyframe interpolation framework featuring null space cost optimization. Second, our approach to object perception, which includes online learning from deep features, semantic segmentation on GPUs using pre-trained models, as well as 6D object pose estimation for better grasp point selection. Finally we will point out the most difficult items for our setup and our approaches to handle them.
    Anton Milan is a Postdoc at the Autonomous Intelligent Systems group at the University of Bonn. He received his PhD (Dr.-Ing.) from the Technische Universitaet Darmstadt, Germany in 2013. He has worked as a software developer in the computer graphics industry and as a senior research fellow at the University of Adelaide in Australia. He served as a reviewer for various computer vision and robotic conferences and journals. His main research interests include object detection, semantic segmentation and multi-target tracking.
  • Peter Yu, PhD Student MCube Lab at MIT (talk video)
    Team MIT-Princeton at Amazon Picking Challenge: More than a suction game.
    Peter Yu is a PhD student of Electrical Engineering and Computer Science at MIT. He received the degrees of B.S. in Computer Science from National Chiao-Tung University, in 2010, Taiwan and M.S. in Computer Science from National Taiwan University, in 2012. He is now working with Alberto Rodriguez, and John Leonard in MCube lab. He studies physical interactions (e.g. pushing objects) to help robots handle daily items. He is a key member in MIT APC Team. 

From the Industrial Community

  • Remus Boca, Senior Principal Scientist, Mechatronics and Sensors, ABB (talk video)
    Warehouse & Logistics, Perspectives from a Robot Manufacturer
    During the collaboration with MIT for the last two years and with Princeton this year, technical challenges solved are addressed including the overall robot system for easy delivery and integration, robot options, cameras and calibration, and robotic perception for grasping and planning. Perspectives regarding future directions and needs for warehouse and logistic will be discussed.
    Remus Boca is a senior principal scientist at the ABB Corporate Research Center in Bloomfield, CT. He earned his PhD in Industrial Robotics (’01) from University Politehnica of Bucharest. He’s been involved in Robot Vision for 15 years and in every technical challenge from sensor layout, camera calibration, object recognition, object localization, tracking to scanning, bin-picking, easy programming for many and diverse robotic industrial applications.
  • Joey Durham, Manager of Research and Advanced Development, Amazon Robotics
    Assembling Orders in Amazon’s Robotic Warehouses
    Amazon Robotics builds the world’s largest mobile robotic fleet where many thousands of robots deliver inventory shelves to pick operators in e-commerce warehouses. Each Amazon warehouse holds millions of items of inventory, most customer orders represent a unique combination of several items, and many orders need to be shipped within a couple hours of being placed to meet delivery promises. This talk will describe how mobile robots and human operators collaborate to solve this challenging problem and enable Amazon to ship millions of orders every day. I will also discuss the results of the recent Amazon Picking Challenge and the next big frontier for robotics in warehousing.
    Joey Durham is Manager of Research and Advanced Development at Amazon Robotics. His team focuses on resource allocation algorithms, machine learning, and path planning for robotic warehouses. He also runs the Amazon Picking Challenge robotic manipulation contest. Joey joined Kiva Systems after completing his Ph.D. at the University of California at Santa Barbara in distributed coordination for teams of robots. He has been with the company through its acquisition and growth into Amazon Robotics. Previously he worked on path planning for autonomous vehicles at Stanford University for the DARPA Grand Challenge.
  • Michael Ferguson, CTO, Fetch Robotics (talk video) 
    Agile Robotic Solutions for Warehousing & Logistics
    Fetch Robotics is a robotics solution provided for logistics and material handling applications. Unlike traditional technologies that require significant time to deploy, our solutions can often be deployed in a day without interruption to warehouse operations. This talk will discuss our approach to building agile and versatile robotic solutions and some of the challenges faced in taking new technologies into the field.
    Michael is Chief Technology Officer at Fetch Robotics. He has extensive experience in designing and working with electronics, controls, sensors, perception and planning required to bring robotic hardware to life. Prior to joining Fetch Robotics, Michael was a co-founder and CTO at Unbounded Robotics, the Founder of Vanadium Labs, and served as a Software Engineer at Willow Garage.
  • Yasunori Kamiya, Team PFN Member, Preferred Networks (talk video) 
    Team PFN's Robotics System at Amazon Picking Challenge and Our Products Expansion in Industrial Field
    Preferred Networks Inc. (PFN) is a Tokyo-based startup focusing on applications of latest artificial intelligence technologies to emerging problems in the Internet of Things (IoT). PFN collaborates with many world-leading companies in industries, such as FANUC for intelligent robots and Toyota motors for autonomous driving. We joined Amazon Picking Challange 2016 because APC is advanced and related to our industrial buisiness, and a good oppotunity to show our tech. I will talk about our APC's system and our buisinesses related to industrial autonomons robotics.
    Yasunori Kamiya is a system developer/researcher in Preferred Networks, Inc. His specialty is around image recognition, machine learning, robotics and semiconductors. He received his PhD in Information Science from Nagoya University in Japan, in 2010. He developed/researched conflict avoidance products for automobiles, using image recognition methods with large-scale high-speed computing systems, and the many robotics systems for several robotics competitions such as the Robocup Soccer. He tackled APC2016 by using these experiences.
  • Ulf Hartmann, Sales & Marketing Director, UniGripper / Tepro Machine & Pac Systems (talk video)
    Robotic End of Arm Tooling for order picking and distribution centers
    What are the challenges of distribution centers and warehouses? Which options for gripping are on the market, and what are the challenges of each of these? We will look at what is already on the market, which areas are covered and tested, and where a solution has to be found. Last but not least we will analyze the findings of the APC and try to come to a recommendation of what to use for APC 2017 based on the past challenges.
    Ulf Hartmann is currently the sales & marketing director for UniGripper. Ulf has built on the innovative culture of UniGripper and developed the brand further to being a solution provider rather than selling components, giving him a unique insight in the different areas and possibilities of the automation sector. Currently he is working on his master’s thesis where he analyzes the robotic industry’s interrelation with sustainability and corporate social responsibility, in addition to establishing his own company which will merely focus on becoming a cross-technology solution provider.
  • Erik Nieves, Founder, Plus One Robotics (talk video)
    From Automotive to Logistics - perspectives from a robotics OEM
    The transformation of logistics from fully manual to highly automated operations presents challenges to the business of industrial robotics. In this non-academic talk, we will discuss the ramifications of bringing robot automation to warehousing, distribution, and fulfillment.
    Before founding PlusOne Robotics, Erik Nieves spent 25 years at Yaskawa Motoman developing robot technologies and systems across varied industries and applications.
  • Jianxiong Xiao, Founder & CEO, AutoX (talk video)
    3D Deep Learning for Robot Perception
    Jianxiong Xiao (a.k.a., Professor X) is the Founder and CEO of AutoX, Inc., a high-tech startup currently in stealth mode. Previously, he was an Assistant Professor in the Department of Computer Science at Princeton University and the founding director of the Princeton Computer Vision and Robotics Labs from 2013 to 2016. He received his Ph.D. from the Computer Science and Artificial Intelligence Laboratory (CSAIL) at the Massachusetts Institute of Technology (MIT) in 2013. Before that, he received a BEng. and MPhil. in Computer Science from the Hong Kong University of Science and Technology in 2009. His research focuses on bridging the gap between computer vision and robotics by building extremely robust and dependable computer vision systems for robot perception. In particular, he is a pioneer in the fields of 3D Deep Learning, Autonomous Driving, RGB-D Recognition and Mapping, Big Data, Large-scale Crowdsourcing, and Deep Learning for Robotics. His work has received the Best Student Paper Award at the European Conference on Computer Vision (ECCV) in 2012 and the Google Research Best Papers Award for 2012, and has appeared in the popular press. Jianxiong was awarded the Google U.S./Canada Fellowship in Computer Vision in 2012, the MIT CSW Best Research Award in 2011, and two Google Faculty Awards in 2014 and in 2015 respectively. He co-lead the MIT+Princeton joint team to participate in the Amazon Picking Challenge in 2016, and won the 3rd and 4th place worldwide.