Dhiraj Gandhi

I am currently working as Lead Robotics AI Research Engineer at Nimble Robotics. Before that I was Research Engineer at Facebook AI Research(FAIR), Pittsburgh lab. I graduated from Masters program at The Robotics Institute, Carnegie Mellon University (CMU) advised by Prof.Abhinav Gupta in July 2019.

I have completed my bachelors in Mechanical engineering from the Indian Institute of Technology (IIT) Guwahati in 2015. Before joining CMU as a master student, I had spent around half a year as an engineer at The Hitech Robotic Systemz and two year & half as a research associate at CMU with Prof. Abhinav Gupta.


Press Coverage

Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias

Learning to Fly by Crashing

Curious Robot

Research and Selected Projects

Object Goal Navigation using Goal-Oriented Semantic Exploration
Devendra Chaplot , Dhiraj Gandhi, Abhinav Gupta, Ruslan Salakhutdinov
NeurIPS 2020
Paper, Project page

This work studies the problem of object goal navigation which involves navigating to an instance of the given object category in unseen environments. We propose a modular system called, 'Goal-Oriented Semantic Exploration' which builds an episodic semantic map and uses it to explore the environment efficiently based on the goal object category. Empirical results in visually realistic simulation environments show that the proposed model outperforms a wide range of baselines including end-to-end learning-based methods as well as modular map-based methods and led to the winning entry of the CVPR-2020 Habitat ObjectNav Challenge.


Visual Imitation Made Easy
Sarah Young , Dhiraj Gandhi, Shubham Tulsiani, Abhinav Gupta, Pieter Abbeel, Lerrel Pinto
CoRL 2020
Paper, Project page

Visual imitation learning provides a framework for learning complex manipulation behaviors by leveraging human demonstrations. However, current interfaces for imitation such as kinesthetic teaching or virtual reality based control prohibitively restricts our ability to collect large-scale data in the wild. Obtaining such diverse demonstration data is paramount for the generalization of learned skills to novel, previously unseen scenarios. In this work, we present an alternate interface for imitation that simplifies the data collection process, while allowing for easy transfer to robots.


Leaning to Explore using Active Neural SLAM
Devendra Chaplot , Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, Ruslan Salakhutdinov
ICLR 2020
Paper, Project page

This work presents a modular and hierarchical approach to learn policies for exploring 3D environments, called Active Neural SLAM. Our approach leverages the strengths of both classical and learning-based methods, by using analytical path planners with learned SLAM module, and global and local policies.The proposed model can also be easily transferred to the PointGoal task and was the winning entry of CVPR 2019 Habitat PointGoal Navigation Challenge.


Object-centric Forward Modeling for Model Predictive Control
Yufei Ye , Dhiraj Gandhi, Abhinav Gupta, Shubham Tulsiani
CoRL 2019
arXiv, project page

In this paper we present an approach to learn an object-centric forward model, and show that this allows us to plan for sequences of actions to achieve distant desired goals. We propose to model a scene as a collection of objects, each with an explicit spatial location and implicit visual feature, and learn to model the effects of actions using random interaction data.


PyRobot: An Open-source Robotics Framework for Research and Benchmarking
Adithya Murali*,Adithya Murali*, Kalyan Vasudev Alwala*, Dhiraj Gandhi*, Lerrel Pinto, Saurabh Gupta, Abhinav Gupta [* Equal contribution]
arXiv, project page, code, facebook AI blog

This paper introduces PyRobot, an open-source robotics framework for research and benchmarking. PyRobot is a light-weight, high-level interface on top of ROS that provides a consistent set of hardware independent mid-level APIs to control different robots. PyRobot abstracts away details about low-level controllers and inter-process communication, and allows non-robotics researchers (ML, CV researchers) to focus on building high-level AI applications.


Self-Supervised Exploration via Disagreement
Deepak Pathak*, Dhiraj Gandhi* ,Abhinav Gupta [* Equal contribution]
ICML 2019
arXiv, project page, code

In this paper, we propose a formulation for exploration inspired by the work in active learning literature. Specifically, we train an ensemble of dynamics models and incentivize the agent to explore such that the disagreement of those ensembles is maximized. This allows the agent to learn skills by exploring in a self-supervised manner without any external reward.


Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias
Abhinav Gupta*, Adithya Murali*, Dhiraj Gandhi* , Lerrel Pinto* [*Equal contribution]
arXiv, data-set

We present the first systematic effort in collecting a large dataset for robotic grasping in homes. The models trained with our home dataset showed a marked improvement of 43.7% over a baseline model trained with data collected in lab. Our architecture which explicitly models the latent noise in the dataset also performed 10% better than one that did not factor out the noise.


Learning to Grasp Without Seeing
Adithyavairavan Murali, Yin Li, Dhiraj Gandhi, Abhinav Gupta
arXiv, video, dataset, project page

Can a robot grasp an unknown object without seeing it? In this paper, we present a tactile-sensing based approach to this challenging problem of grasping novel objects without prior knowledge of their location or physical properties. Our key idea is to combine touch based object localization with tactile based re-grasping. To train our learning models, we created a large-scale grasping dataset, including more than 30 RGB frames and over 2.8 million tactile samples from 7800 grasp interactions of 52 objects.


CASSL: Curriculum Accelerated Self-Supervised Learning
Adithyavairavan Murali, Lerrel Pinto, Dhiraj Gandhi, Abhinav Gupta
ICRA, 2018
arXiv, video

Recent self-supervised learning approaches focus on using a few thousand data points to learn policies for high-level, low-dimensional action spaces. However, scaling this framework for high-dimensional control require either scaling up the data collection efforts or using a clever sampling strategy for training. We present a novel approach - Curriculum Accelerated Self-Supervised Learning (CASSL) - to train policies that map visual information to high-level, higher- dimensional action spaces.



Learning to Fly by Crashing
Dhiraj Gandhi, Lerrel Pinto , Abhinav Gupta
IROS, 2017
arXiv, video, dataset & trained model

How do you learn to navigate an Unmanned Aerial Vehicle (UAV) and avoid obstacles? In this project, we propose to bite the bullet and collect a dataset of crashes itself! We build a drone whose sole purpose is to crash into objects: it samples naive trajectories and crashes into random objects. We crash our drone around 11K times to create one of the biggest UAV crash dataset. We show that simple self-supervised models learnt on this data is quite effective in navigating the UAV even in extremely cluttered environments with dynamic obstacles including humans.


The Curious Robot: Learning Visual Representations via Physical Interactions
Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park and Abhinav Gupta
Spotlight Presentation at ECCV 2016

What is the right supervisory signal to train visual representations? In case of biological agents, visual representation learning does not require semantic labels. In fact, we argue that biological agents use active exploration and physical interactions with the world to learn visual representations unlike current vision systems which just use passive observations (images and videos downloaded from web). Towards this goal, we build one of the first systems on a Baxter platform that pushes, pokes, grasps and actively observes objects in a tabletop environment.

We use these physical interactions to collect more than 130K datapoints, with each datapoint providing backprops to a shared ConvNet architecture allowing us to learn visual representations. We show the quality of learned representations by observing neuron activations and performing nearest neighbor retrieval on this learned representation. Finally we evaluate our learned ConvNet on different image classification tasks.


Auto PID tuning of Automated Guided Vehicle(AGV)
Work experience at The Hi-Tech Robotic Systemz Ltd.

Engineers at company faced difficulties in tuning the AGVs PID controller using the prevalent Ziegler-Nichols technique. This tuning technique was also not robust enough to work on varying loading conditions (from 0-2000lb). To overcome this cumbersome process, I undertook an initiative to automate it. After conducting a literature review and running proof-of-concept experiments in the Gazebo simulator, I implemented a model free Iterative Feedback Tuning scheme. AGVs tuned with this method are running smoothly in production for the past couple of years at the Honda and Nokia plants in India.


Sampling based path planning algorithm
video, code

In this project I tried to study different sampling based algorithms like PRM, RRT, RRT* and implemented them in matlab. I tried to smooth out trajectory obtained using these algorithms using Bezier curve method. Furthermore I tried to exted these algoritms for car like non-holonomic object using Dubins curve and Reeds-Shepps curve.


Design & Development of hybrid legged and wheeled robot
Dhiraj Gandhi, Jagjeet Singh , A. Narayana Reddy
Proceedings of iNaCoMM2015
pdf, video

Wheeled locomotion is most widely used mechanism for mobile robots on even terrain whereas walking mechanism is suitable for mobility of robots on uneven terrain. To achieve advantages of wheeled as well as biped robot, we designed a robot with both mobility mechanisms. A 6-degree of freedom (DoF) biped robot has been constructed with 3 DoF on each leg with mechanism of transforming it into a wheeled robot.

Design of Parrlel Planar Manipulator (PPM)
2014 summer internship at Systemantics
pdf, video

Parallel manipulators have higher payload capacity, higher mechanical rigidity and better accuracy than their serial counter parts. However presence of singularities within the workspace is major problem with these manipulators. In this project I worked on optimizing the ratio of singularity free workspace to the total space occupied by the mechanism with respect to link lenght. Furthermore I extended the work to come up with design algorithm that engineers can use for its optimal design. On application side I used 2RR PPM to deposit glue evenly on engine gaskit in simulation


Design & Development of Spherical Robot
Dhiraj Gandhi, Akshay Khatri and Leena Vachhani
2013 Summer internship at SysCon Dept. IIT Bombay
pdf, video

Spherical Robots have several advantages over robots that use wheels or legs for locomotion, including inner component protection, low rolling resistance, and ability to move in any direction. At the same time, it brings number of challenging problems in modelling, stabilization, and path following. In this project we designed and manufactured novel two pendulum based Spherical robot within diameter of 6 inch and achieved remote controlled omni-directional movements. We also mathematically modeled the system to observe the effect of pendulum state on the motion of robot.

Source stolen from here