Zihan (Hanry) Ding

丁子涵

zhding96@gmail.com / zhding@mail.ustc.edu.cn / zihand@princeton.edu

Google Scholar / GitHub / CV / Research Gate / Previous Website

About Me

Zihan Ding is second-year Ph.D. at Electrical and Computer Engineering Department, Princeton University, supervised by Chi Jin. He obtained a MSc Machine Learning specialist degree with distinction from Imperial College London in Fall 2019. He previously worked with Dr. Edward Johns at Imperial Robot Learning Lab for his thesis project. Before Msc, he received two Bachelor degrees from University of Science and Technology of China in 2018, major in Photoelectric Information Science and Engineering (Physics Dept., Bachelor of Science) and dual in Computer Science and Technology (CS Dept., Bachelor of Engineering). His bachelor thesis is supervised by Dr. Jinming Cui and Prof. Yunfeng Huang.
His primary research interests are: deep reinforcement learning (RL), robot learning, simulation-to-reality transfer. He also has general interests in meta learning, transfer learning, imitation learning, quantum computation, quantum machine learning, neuroscience, etc. His previous research topics cover: robot learning, sim-to-real transfer, tactile sensory on robots, RL applications, exlainability of RL policy, generative models, multimodal policy representation, machine learning in quantum computation, robotic visuomotor control, etc.


"Research is about intellectual pilgrimage when climbing the mountain of persent knowledge, self-talking, acquiring deep and structral understanding, capturing the flash of inspiration, quick prototyping, persuing rigorism, simplicity and beauty."

Publications

For all publications please check my Google Scholar and Research Gate.

Books
Deep Reinforcement Learning: Fundamentals, Research and Applications
Hao Dong, Zihan Ding, Shanghang Zhang Eds.
Springer 2020 ISBN 978-981-15-4094-3, 1st ed.
[Homepage][eBook][Amazon][中文版] Buy at the Springer Shop
机器学习系统:设计与实现(Machine Learning System: Design and Implementation)
Luo Mai, Hao Dong et al

清华大学出版社 Tsinghua University Press 2022 ISBN Coming Soon.
Author of Chapter: Reinforcement Learning System
[Homepage]
Papers
A Deep Reinforcement Learning Approach for Finding Non-Exploitable Strategies in Two-Player Atari Games
Zihan Ding*, Dijia Su*, Qinghua Liu, Chi Jin
[Paper][Code1][Code2][Slide]
Learning Distributed and Fair Policies for Network Load Balancing as Markov Potentia Game
Zhiyuan Yao*, Zihan Ding*
36th Conference on Neural Information Processing Systems (NeurIPS) 2022
[Paper][Code]
Multi-Agent Reinforcement Learning for Network Load Balancing in Data Center
Zhiyuan Yao, Zihan Ding and Thomas Heide Clausen

[Paper][Code]
Reinforced Workload Distribution Fairness
Zhiyuan Yao, Zihan Ding and Thomas Heide Clausen
Machine Learning for Systems at 35th Conference on Neural Information Processing Systems (NeurIPS) 2021.
[Paper][Code]
Probabilistic Mixture-of-Experts for Efficient Deep Reinforcement Learning
Jie Ren, Yewen Li, Zihan Ding, Wei Pan, Hao Dong
[Paper][Code]
Bayesian Optimization for Wavefront Sensing and Error Correction
Qian Zhong-Hua, Ding Zi-Han, Ai Ming-Zhong, Zheng Yong-Xiang, Cui Jin-Ming, Huang Yun-Feng, Li Chuan-Feng, Guo Guang-Can
Chinese Physics Letters 2021.
[Paper]
Sim-to-Real Transfer for Robotic Manipulation with Tactile Sensory
Zihan Ding, Ya-Yen Tsai, Wang Wei Lee, Bidan Huang
International Conference on Intelligent Robots and Systems (IROS) 2021.
[Paper][Code]
DROID: Minimizing the Reality Gap using Single-Shot Human Demonstration
Ya-Yen Tsai, Hui Xu, Zihan Ding, Chong Zhang, Edward Johns, Bidan Huang
IEEE Robotics and Automation Letters (RA-L) 2021.
[Paper]
Robotic Visuomotor Control with Unsupervised Forward Model Learned from Videos
Haoqi Yuan, Ruihai Wu, Andrew Zhao, Haipeng Zhang, Zihan Ding, Hao Dong
International Conference on Intelligent Robots and Systems (IROS) 2021.
[Paper][Website]
CDT: Cascading Decision Trees for Explainable Reinforcement Learning
Zihan Ding, Pablo Hernandez-Leal, Gavin Weiguang Ding, Changjian Li, Ruitong Huang
[Paper][Code]
RLzoo: A Comprehensive and Adaptive Reinforcement Learning Library
Zihan Ding, Tianyang Yu, Yanhua Huang, Hongming Zhang, Guo Li, Quancheng Guo, Luo Mai and Hao Dong
ACM Multimedia Open Source Software Competition 2021.
[Paper][Repo]
Crossing The Gap: A Deep Dive into Zero-Shot Sim-to-Real Transfer for Dynamics
Eugene Valassakis, Zihan Ding and Edward Johns
International Conference on Intelligent Robots and Systems (IROS) 2020.
[Paper][Website][Video]
Sim-to-Real Transfer for Optical Tactile Sensing
Zihan Ding, Nathan F. Lepora and Edward Johns
International Conference on Robotics and Automation (ICRA) 2020.
[Paper][Website][Video]
Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence
Yuhang Song, Jianyi Wang, Thomas Lukasiewicz, Zhenghua Xu, Mai Xu, Zihan Ding, and Lianlong Wu
The Thirty-Fourth AAAI Conference on Artificial Intelligence 2020.
[Paper][Code]
Fast and High-Fidelity Readout of Single Trapped-Ion Qubit via Machine-Learning Methods
Zihan Ding, Jinming Cui, Yunfeng Huang, Chuanfeng Li, Tao Tu, Guangcan Guo
Physical Review Applied 2019.
[Paper] [Code]
Tensor Super-Resolution with Generative Adversarial Nets: A Large Image Generation Approach
Zihan Ding, Xiao-Yang Liu, Miao Yin
International Joint Conference on Artificial Intelligence, Human Brain Artificial Intelligence 2019.
[Paper] [Code]
Deep Reinforcement Learning for Intelligent Transportation Systems
Xiao-Yang Liu, Zihan Ding, Sem Borst, Anwar Walid
NeurIPS Workshop on Machine Learning for Intelligent Transportation Systems 2018.
[Paper] [Code]
Accelerated Exhaustive Eye Glints Localization Method for Infrared Video Oculography
Zihan Ding, Jiayi Luo, Hongping Deng
Proceedings of the 33rd Annual ACM Symposium on Applied Computing, SAC ‘18.
[Paper] [Code]

Open Source Projects

TensorLayer Reinforcement Learning Tutorials: principal developer.

TensorLayer Reinforcement Learning Tutorials are hands-on examples of implementing DRL algorithms with TensorLayer, each example is self-contained with simple structure, particularly suitable for novices.

RLzoo: principal developer.

RLzoo is a collection of the most practical reinforcement learning algorithms, frameworks and applications. It is implemented with Tensorflow 2.0 and API of neural network layers in TensorLayer 2, to provide a hands-on fast-developing approach for reinforcement learning practices and benchmarks.

MARS: principal developer.

MARS is a library for multi-agent RL on Markov games, like PettingZoo Atari, SlimeVolleyBall, etc.

Robot Learning Library: principal developer.

Robot Learning Library is a collorative, open-source library for robot learning, mainly using techniques like deep reinforcement learning, robotics simulation, computer vision.

AI4Finance-Foundation: FinRL: contributor.

AI community has accumulated an open-source code ocean over the past decade. Applying these intellectual and engineering properties to finance will initiate a paradigm shift from the conventional trading routine to an automated machine learning approach, even RLOps in finance.

Academic Service

Conference Reviewer: AAAI'23, NeurIPS'22, AISTATS'22, ICML'22, CISS'22, NeurIPS'21, IEEE/ASME AIM'21, IROS'21, NeurIPS'20 QTNML workshop, NeurIPS'19 AD workshop.

Journal Reviewer: RA-L'21.