PPO Control with the Standard Open ARM101
Implementation of the Proximal Policy Optimization (PPO) algorithm, tested with the Standard Open ARM101 in IsaacLab and deployed in the real world
Posts by Collection
portfolio
Proximal Policy Optimization within IsaacLab
Implementation of the Proximal Policy Optimization (PPO) algorithm, tested with multiple environments within IsaacLab
SO-ARM101 SmolVLA Finetuning
Finetuning SmolVLA for the SO-ARM101 in the real world.
PokerBot
A Texas Hold’em playing Robot
Visual Odometry Pipeline and Visual Slam Pipeline with the KITTI dataset in Python
Visual odometry pipeline, and a visual SLAM pipeline. Both tested with the KITTI dataset 
RRT* with differential-drive dynamics implementation in Python
Implementation of the RRT* algorithm with and without differential-drive dynamics within jupyter notebooks
Proximal Policy Optimization, Deep Reinforcement Learning
Implementation of the Proximal Policy Optimization (PPO) algorithm, tested with the Atari Pong Gymnasium environment
Capstone
Review and Update of Electrical Infrastructure for an Existing Underwater Robot
Advent of Code 2025 Solutions
My solutions in Python to the Advent of Code 2025 challenges.
Proximal Policy Optimization, Deep Reinforcement Learning
Implementation of the Proximal Policy Optimization (PPO) algorithm, tested with the Lunar Lander Gymnasium environment
Animated Julia and Fatou set on an FPGA
Implementation of a VGA driver, and an animated Julia and Fatou set
Deep Q-Network
Implementation of the Deep Q-Network algorithm, tested with the Cart Pole Gymnasium environment
Reinforcment Learning Implementations
Implementations for various reinforcement learning algrotihms for both discrete and continuous state and action spaces
publications
Methods for combining and representing non-contextual autonomy scores for unmanned aerial systems
Published in 2022 8th International Conference on Automation, Robotics and Applications (ICARA), 2022
Measuring an overall autonomy score for a robotic system requires the combination of a set of relevant aspects and features of the system that might be measured in different units, qualitative, and/or discordant. In this paper, we build upon an existing non-contextual autonomy framework that measures and combines the Autonomy Level and the Component Performance of a system as overall autonomy score. We examine several methods of combining features, showing how some methods find different rankings of the same data, and we employ the weighted product method to resolve this issue. Furthermore, we introduce the non-contextual autonomy coordinate and represent the overall autonomy of a system with an autonomy distance. We apply our method to a set of seven Unmanned Aerial Systems (UAS) and obtain their absolute autonomy score as well as their relative score with respect to the best system.
Recommended citation: Brendan Hertel, Ryan Donald, Christian Dumas, S Reza Ahmadzadeh (2022). "Methods for combining and representing non-contextual autonomy scores for unmanned aerial systems" 2022 8th International Conference on Automation, Robotics and Applications (ICARA).
Download Paper
Contextual Autonomy Evaluation of Unmanned Aerial Vehicles in Subterranean Environments
Published in 2023 9th International Conference on Automation, Robotics and Applications (ICARA), 2023
In this paper we focus on the evaluation of contextual autonomy for robots. More specifically, we propose a fuzzy framework for calculating the autonomy score for a small Unmanned Aerial Systems (sUAS) for performing a task while considering task complexity and environmental factors. Our framework is a cascaded Fuzzy Inference System (cFIS) composed of combination of three FIS which represent different contextual autonomy capabilities. We performed several experiments to test our framework in various contexts, such as endurance time, navigation, take off/land, and room clearing, with seven different sUAS. We introduce a predictive measure which improves upon previous predictive measures, allowing for previous real-world task performance to be used in predicting future mission performance.
Recommended citation: Ryan Donald, Peter Gavriel, Adam Norton, S Reza Ahmadzadeh. (2023). "Contextual Autonomy Evaluation of Unmanned Aerial Vehicles in Subterranean Environments" 2023 9th International Conference on Automation, Robotics and Applications (ICARA).
Download Paper
An Adaptive Framework for Manipulator Skill Reproduction in Dynamic Environments
Published in 2024 21st International Conference on Ubiquitous Robots (UR), 2024
Robot skill learning and execution in uncertain and dynamic environments is a challenging task. This paper proposes an adaptive framework that combines Learning from Demonstration (LfD), environment state prediction, and highlevel decision making. Proactive adaptation prevents the need for reactive adaptation, which lags behind changes in the environment rather than anticipating them. We propose a novel LfD representation, Elastic-Laplacian Trajectory Editing (ELTE), which continuously adapts the trajectory shape to predictions of future states. Then, a high-level reactive system using an Unscented Kalman Filter (UKF) and Hidden Markov Model (HMM) prevents unsafe execution in the current state of the dynamic environment based on a discrete set of decisions. We first validate our LfD representation in simulation, then experimentally assess the entire framework using a legged mobile manipulator in 36 real-world scenarios. We show the effectiveness of the proposed framework under different dynamic changes in the environment. Our results show that the proposed framework produces robust and stable adaptive behaviors.
Recommended citation: Ryan Donald, Brendan Hertel, Stephen Misenti, G Yan, Reza Azadeh. (2024). "An Adaptive Framework for Manipulator Skill Reproduction in Dynamic Environments" 2024 21st International Conference on Ubiquitous Robots (UR).
Download Paper
talks
Talk 1 on Relevant Topic in Your Field
Published:
This is a description of your talk, which is a markdown file that can be all markdown-ified like any other post. Yay markdown!
Tutorial 1 on Relevant Topic in Your Field
Published:
This is a description of your tutorial, note the different field in type. This is a markdown files that can be all markdown-ified like any other post. Yay markdown!
Talk 2 on Relevant Topic in Your Field
Published:
This is a description of your talk, which is a markdown files that can be all markdown-ified like any other post. Yay markdown!
Conference Proceeding talk 3 on Relevant Topic in Your Field
Published:
This is a description of your conference proceedings talk, note the different field in type. You can put anything in this field.
teaching
Teaching experience 1
Undergraduate course, University 1, Department, 2014
This is a description of a teaching experience. You can use markdown like any other post.
Heading 1
Heading 2
Heading 3
Teaching experience 2
Workshop, University 1, Department, 2015
This is a description of a teaching experience. You can use markdown like any other post.