Peter Karkus

Hi, I'm Peter Karkus.

Research Scientist @ NVIDIA Research
pkarkus@nvidia.com | Google Scholar | LinkedIn

Previously:

PhD Candidate, Research Assistant @ National University of Singapore

Research Intern @ DeepMind,

Research Intern @ Google Brain

Visiting Researcher @ MIT CSAIL

Fulbright Visiting Researcher @ CMU Robotics Institute

Intern @ NTT Basic Research Lab

I am a researcher in machine learning and robotics with a long-term vision of building human-level robot intelligence. My research focuses on autonomous vehicles and embodied AI that combines learning with structure and reasoning. My recent works are on neural networks that encode classic robot algorithms in order to learn partially observable planning, visual navigation, localization, and mapping tasks.

Interests: reinforcement learning, neural networks, embodied AI, autonomous vehicles, planning, robot control, visual navigation.

Selected papers

DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehicles [pdf] [website] [code]
P. Karkus, B. Ivanovic, S. Mannor, M. Pavone
Conference on Robot Learning (CoRL), 2022

Differentiable SLAM-net: Learning Particle SLAM for Visual Navigation [pdf] [website] [talk]
P. Karkus, S. Cai, D. Hsu
Conference on Computer Vision and Pattern Recognition (CVPR), 2021

Beyond Tabula-Rasa: a Modular Reinforcement Learning Approach for Physically Embedded 3D Sokoban [pdf] [website]
P. Karkus, M. Mirza, A. Guez, A. Jaegle, T. Lillicrap, L. Buesing, N. Heess, T. Weber.
arXiv preprint arXiv:2010.01298, 2020

Differentiable Mapping Networks: Learning Structured Map Representations for Sparse Visual Localization [pdf] [website] [talk]
P. Karkus, A. Angelova, V. Vanhoucke, R. Jonschkowski.
International Conference on Robotics and Automation (ICRA), 2020

Discriminative Particle Filter Reinforcement Learning for Complex Partial observations [pdf] [code]
X. Ma, P. Karkus, N. Ye, D. Hsu, W.S. Lee.
International Conference on Learning Representations (ICLR), 2020

Particle Filter Recurrent Neural Networks [pdf] [code]
X. Ma*, P. Karkus*, D. Hsu, W.S. Lee. *equal contribution
AAAI Conference on Artificial Intelligence (AAAI), 2020

Differentiable Algorithm Networks for Composable Robot Learning [pdf] [talk]
P. Karkus, X. Ma, D. Hsu, L.P. Kaelbling, W.S. Lee, T. Lozano-Perez.
Robotics: Science and Systems (RSS), 2019
Best Student Paper Nomination. Best System Paper Nomination.

Factored Contextual Policy Search with Bayesian Optimization [pdf]
R. Pinsler*, P. Karkus*, A. Kupcsik, D. Hsu, W.S. Lee. *equal contribution
International Conference on Robotics and Automation (ICRA), 2019

Particle Filter Networks with Application to Visual Localization [pdf] [code]
P. Karkus, D. Hsu, W.S. Lee.
Conference on Robot Learning (CoRL), 2018

QMDP-Net: Deep Learning for Planning under Partial Observability [pdf] [code]
P. Karkus, D. Hsu, W.S. Lee.
Advances in Neural Information Processing Systems (NeurIPS), 2017

Recursive Constraint Manifold Subsearch for Multirobot Path Planning with Cooperative Tasks [pdf]
P. Karkus, G. Wagner, H. Choset.
Annual Symposium on Combinatorial Search (SOCS), 2016

Generation of entangled photons using an arrayed waveguide grating [pdf]
N. Matsuda, H. Nishi, P. Karkus, T. Tsuchizawa, K. Yamada, W.J. Munro, K. Shimizu, H. Takesue.
Journal of Optics 19(12), 2017

On-Chip Generation and Demultiplexing of Quantum Correlated Photons Using a Silicon-Silica Monolithic Photonic Integration Platform [pdf]
N. Matsuda, P. Karkus, H. Nishi, T. Tsuchizawa, W.J. Munro, H. Takesue, and K. Yamada.
Optics Express 22(19), 2014

Contact: pkarkus@nvidia.com