Designing strategic interactions between dynamical systems
Correlated Equilibria
Normally, in a game, each player picks a strategy independently, aiming to do as well as possible given the others’ choices, but in a correlated equilibrium, players can coordinate their actions using a shared signal, which can lead to better overall outcomes.
The problem? There’s no neat optimization-based method for finding these correlated equilibria, which makes it hard to apply learning algorithms such as policy gradient or optimal control. Read this paper to fnd out how to formulate a correlated strategy optimization problem.
Multi-agent Path Coordination Game
In a bustling warehouse filled with robots, or a fleet of self-driving taxis navigating city streets. Each vehicle (player) has its own way of moving, but they all share the same roads. The tricky part? Their paths will be delayed if too many try to use the same route at once. Read further to understand how to model multi-agent path planning with congestion via Markov Decision Processes (MDPs).
Multi-Player Reachability
Imagine multiple robots moving through a maze. Each robot needs to reach its destination by a certain time and they must avoid bumping into each other. Read this paper to understand how find reachability-maximizing paths using decentralized policies.
Decision-making under forecast and environment uncertainty
MDP for guiding satellite collision avoidance maneuvers
Satellites often need to dodge each other to prevent collisions, but deciding when to maneuver is tricky—move too early and you waste fuel, wait too long and you risk disaster. This paper tackles that problem by modeling satellite collision avoidance as a Markov Decision Process (MDP) and training policies with reinforcement learning, the same AI ideas used in video games. The result is a decision-making system that learns to balance safety and fuel efficiency better than fixed “one-size-fits-all” rules, making satellites smarter, safer, and longer-lived in an increasingly crowded space environment.
Set-based operators for stationary and uncertain MDPs
How do you play a video game where the rules of the world keep changing? Sometimes the enemy gets stronger, sometimes the terrain shifts, sometimes the controls feel different. In this sequence of papers, we treat outcomes as sets as opposed to single answers. Using lattice theory and fix-point analysis on robust Markov decision processes, we show that the outcomes will not spiral out of control, even if the environment never settles.
Bullwhip minimization for forecast-driven supply chains
Supply chain like a line of people passing buckets of water. If the person at the front makes a tiny mistake, the splashes get bigger and bigger as they pass down the line. That’s the Bullwhip effect, and it causes huge problems in real supply chains. This project looks at the worst-case spikes that happen when demand forecasts are wrong. Instead of just measuring average errors, we approximate the $H_\infty$ norm to capture these spikes. Using a combination of data-driven modeling and positive semi-definite programming, we bound and reduce these spikes, making supply chains more stable and less wasteful.
Trajectory Planning in Uncontrolled Airspace
At the majority of airports in the U.S., pilots coordinate directly with each other over radio using loosely structured natural language, without air traffic control. The central challenge addressed in this work is: How can an autonomous aircraft interpret and integrate radio communications into its motion planning to safely and effectively coordinate with other aircraft in an untowered airport environment, like human pilots do.
MPC for on-orbit servicing
Satellites spin freely in zero gravity, and future repair robots will need to dock and grab them without crashing. This project shows how to use smart predictive control algorithms to make a servicing satellite align its spin with a tumbling target, then carefully move its robot arm to make contact without any sudden jolt. By combining reaction wheels with the robot arm in a coordinated way, the system can smoothly match motion and “catch” the satellite safely.
Modeling + shaping trends in population games
Tolling for MDP congestion games
This project shows how to learn the smallest “just enough” toll to keep traffic within limits over time, even if we can’t directly model each driver’s private objectives. It treats toll computation as an interactive process with a crowd of drivers: set a toll, see how the crowd responds, then adjust. Leveraging theoretical guarantees of inexact gradient descent, this feedback tolling mechanism converges and can enforce constraints when people are only approximately optimal. Using NYC Uber data, we demonstrate reducing congestion while minimizing the reduction in the average driver earnings.
Alpha fair routing
In urban airspaces, aerial vehicles will share a small network of airspaces under a range of flight priorities (emergency helicopters, air taxis). This project designs a routing mechanism to fairly split limited air space capacity across neighborhoods while remaining safe when weather or disruptions shrink the available capacity. Specifically, we guarantee α-fair split of service among communities while keeping traffic within risk-aware capacity limits. The problem boils down to a convex optimization problem (efficient to solve) and, on an Austin case study, the solution delivers a more even distribution of service than simply maximizing total trips.
Sensitivity Analysis
At Nash equilibrium, how much does the steady-state behavior change if the costs change a little? We use sensitivity analysis to study this, and use the results to define and demonstrate a stochastic Braess paradox (adding a new “shortcut” in a network worsens the average efficiency), which fundamental relies on the sensitivity of Nash equilibrium to route cost changes.