Jacky Liang

Hello!

I’m a PhD candidate at Carnegie Mellon University’s Robotics Institute, and I’m co-advised by Prof. Oliver Kroemer and Prof. Maxim Likhachev.

My research focuses on enabling robust and generalizable robot manipulation by explicitly reasoning about contacts, efficiently collecting interaction data in both simulation and the real world, and incorporating structured manipulation priors into learning-based methods.

Before joining RI I interned at NVIDIA for the Isaac project and worked on using GPU-accelerated physics simulations for robot learning. Before that and during my time at UC Berkeley I did research in robot grasping and manipulation with Prof. Ken Goldberg.

In my spare time I write for and co-run Skynet Today and the Last Week in AI newsletter. Our goal is to demystify and disseminate AI news without the hype. Check us out and subscribe if you like what you see :)

Selected Articles

Why we don’t have Robot Butlers yet

DeepMind’s AlphaFold 2—An Impressive Advance With Hyperbolic Coverage

OpenAI’s dexterous robotic hand — separating progress from PR

Job loss due to AI - How bad is it going to be?

Research

Conference Papers

Search-Based Task Planning with Learned Skill Effect Models for Lifelong Robotic Manipulation

Jacky Liang*, Mohit Sharma*, Alex LaGrassa, Shivam Vats, Saumya Saxena, Oliver Kroemer. International Conference on Robotics and Automation (ICRA). May 2022. [arXiv] [Website]

Visual Identification of Articulated Object Parts

Vicky Zeng, Timothy E. Lee*, Jacky Liang*, Oliver Kroemer. International Conference on Intelligent Robots and Systems (IROS). September 2021. [arXiv] [Website]

Contact Localization for Robot Arms in Motion without Torque Sensing

Jacky Liang, Oliver Kroemer. International Conference on Robotics and Automation (ICRA). May 2021. [arXiv] [Website]

Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation

Mohit Sharma*, Jacky Liang*, Jialiang Zhao, Alex LaGrassa, Oliver Kroemer. Conference on Robot Learning (CoRL). November 2020. Plenary Presentation. [arXiv] [Website]

Learning Active Task-Oriented Exploration Policies for Bridging the Sim-to-Real Gap

Jacky Liang, Saumya Saxena, Oliver Kroemer. Robotics: Science and Systems (RSS). July 2020. [arXiv] [Website]

In-Hand Object Pose Tracking via Contact Feedback and GPU-Accelerated Robotic Simulation

Jacky Liang, Ankur Handa, Karl Van Wyk, Viktor Makoviychuk, Oliver Kroemer, Dieter Fox. International Conference on Robotics and Automation (ICRA). May 2020. [arXiv] [Website]

DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System

Ankur Handa, Karl Van Wyk, Wei Yang, Jacky Liang, Yu-Wei Chao, Qian Wan, Stan Birchfield, Nathan Ratliff, Dieter Fox. International Conference on Robotics and Automation (ICRA). May 2020. [arXiv] [Website]

Homography-Based Deep Visual Servoing Methods for Planar Grasps

Austin S. Wang, Wuming Zhang, Daniel Troniak, Jacky Liang, Oliver Kroemer. International Conference on Intelligent Robots and Systems (IROS). Macau, China, November 2019. [Paper]

Towards Precise Robotic Grasping by Probabilistic Post-grasp Displacement Estimation

Jialiang Zhao, Jacky Liang, Oliver Kroemer. Field and Service Robotics (FSR). Tokyo, Japan, August 2019. [arXiv]

GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learning

Jacky Liang*, Viktor Makoviychuk*, Ankur Handa*, Nuttapong Chentanez, Miles Macklin, Dieter Fox. Conference on Robot Learning (CoRL). Zurich, Switzerland, October 2018. [arXiv] [Website]

Using dVRK Teleoperation to Facilitate Deep Learning of Automation Tasks for an Industrial Robot

Jacky Liang, Jeffrey Mahler, Michael Laskey, Pusong Li, Ken Goldberg. Conference on Automation Science and Engineering (CASE). Xi’an, China, August 2017. Finalist, Best Student Paper Award. [PDF] [Website]

Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics

Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Xinyu Liu, Juan Aparicio Ojea, Ken Goldberg. Robotics: Science and Systems (RSS). Cambridge, MA. July 2017. [PDF] [Website]

Design of Parallel-Jaw Gripper Tip Surfaces for Robust Grasping

Menglong Guo, David V. Gealy, Jacky Liang, Jeffrey Mahler, Aimee Goncalves, Stephen McKinley, Ken Goldberg. International Conference on Robotics and Automation (ICRA). Singapore, May 2017. [PDF]

Undergraduate-Led Survey Class to Improve CS Education for New Students

Nathan Zhang*, Jacky Liang*, Amanda Tomlinson*, Frank Boensch, Anant Sahai. SIGCSE ‘20: Proceedings of the 51st ACM Technical Symposium on Computer Science Education. February 2020. [Paper]

Technical Reports

A Modular Robotic Arm Control Stack for Research: Franka-Interface and FrankaPy

Kevin Zhang*, Mohit Sharma*, Jacky Liang*, Oliver Kroemer. November 2020. [arXiv]

* Equal Contribution.

Teaching

16-350 Planning Techniques for Robotics

Spring 2021 Teaching Assistant

16-662 Robot Autonomy

Spring 2020 Teaching Assistant

CS188 Introduction to Artificial Intelligence

Spring 2017 Discussion Teaching Assistant

EE16A Design of Information Devices and Systems

Fall 2016, 2017 Discussion Teaching Assistant

Education

Carnegie Mellon University. Robotics Institute. PhD Student

University of California Berkeley. Electrical Engineering and Computer Science. Bachelor of Science, Class of 2018

Open-Source Libraries

FrankaPy - Python library for controlling the Franka Emika Panda robot arm.
IsaacGym-Utils - Python library that provides simpler APIs for the Nvidia Isaac Gym robot simulator
Simple ZMQ - Python library that uses zmq to send arbtirary objects over a network
Async Savers - Python library for asyncrhonously saving data in shards
Data Learning Boilerplate - Boilerplate Python code for collecting data and training neural networks using W&B and PyTorch Lightning