Vision Aided Rocket Landing GNC
A rocket that has to land itself, flying the whole descent on a state it can only estimate.
A simulated guidance, navigation and control stack for a small rocket flying a powered landing. IMU, optical flow, radar altimeter and GPS are fused in an error state Kalman filter, an LQG controller closes the loop, and the full descent is visualised in Foxglove with live thrust vectoring and truth against estimate tracking.
- Estimator
- Error state Kalman filter
- Control
- LQG
- Sensing
- IMU, optical flow, radar altimeter, GPS
- Actuation
- Thrust vectoring
- Telemetry
- MCAP, replayed in Foxglove
- Language
- Python
- Role
- Simulation, estimation and control
- Stack
- Python, Error state Kalman filter, LQG control, Foxglove
The loop
The simulation flies a small rocket through a powered descent to touchdown, with the full guidance, navigation and control stack in the loop. Nothing that flies the vehicle is allowed to see the truth: the sensors are what the filter gets, the filter's estimate is what the controller acts on, and the true trajectory exists only so the two can be drawn against each other after the fact.
That separation is the whole point of building it this way. An estimator tested on its own looks good in isolation, and a controller given perfect state always flies well. Run them together and the errors compound in the way they would on real hardware: estimator lag becomes control effort, and control effort moves the vehicle the estimator is trying to track.
Navigation
Four sensors are fused, chosen because they fail in different ways. The IMU carries the state forward at high rate and drifts. The radar altimeter fixes height above the ground directly. Optical flow measures lateral motion once the ground is close enough to resolve texture. GPS bounds absolute position at a slower rate and with coarser precision. Any one of them alone is insufficient for a landing; together they cover each other.
The fusion runs as an error state Kalman filter, which tracks the small correction to a nominal trajectory rather than the trajectory itself. Attitude is the reason this matters: keeping the error in a minimal three parameter form, and folding it back into the nominal quaternion after every update, avoids the singularity and normalisation problems that a filter carrying attitude directly runs into.
Guidance and control
The loop is closed with an LQG controller, the pairing of a linear quadratic regulator with a Kalman estimator, commanding throttle and thrust vector angles. In the recording the thrust trace sits around 250 N through most of the descent and steps up for the flare, and both gimbal axes hold near zero with small corrections either side, which is what a vehicle tracking a descent profile rather than fighting one looks like.
Reading the descent
Everything the run produces is logged to MCAP and replayed in Foxglove. The 3D panel carries the vehicle, its gimbal and the landing plane, tracked from the outside so the attitude is legible; the plots beside it carry thrust, descent rate, translation and rotation. Translation is drawn twice, estimate against truth, so the navigation error is a visible gap on the same axes rather than a number in a log.
Building the visualisation as part of the simulation rather than after it changed how the work went. A filter that has diverged looks obviously wrong on a 3D view seconds before it shows up in a residual plot, and a controller chattering on the gimbal is something you see immediately.
Want the detail behind this one? Just ask.