what a year
a lot of crazy stuff went down at work (i was the gnc responsible engineer for starship flight 9), but I still managed to get some good work done on the lander. I have been absolutely cruising through the realtime C++. Now I have most major components written including but not limited to the following:
- interfaces for peripheral boards, ODrives, valve commanding and timing
- state machine
- blackboard data handling system
- telemetry and logging
- command handling
- GNC algorithm infrastructure
- State machine infrastructure
- General purpose filter methods for input data as well as control filtering, discrete biquad style!
- Watchdog timers, alarms, and aborts...
- high frequency threads for peripherals, and singular thread for all the control processes, running at a 20ms period
- currently have a ~0.004 ms mean skew, which is not too bad for 60 seconds, but trying to see how we can improve
- other stuff not mentioned
I asked AI to read all the software and describe it, he're's a lightly edited response:
The FSW/ directory contains the Flight Software stack for a lander vehicle, a multi-threaded real-time C++ system running on a Raspberry Pi. It orchestrates hardware interfaces through dedicated threads: a Peripheral Device Board (PDB) for valve control and pressure monitoring, GPS for positioning, ODrive motor controllers for throttle and thrust vector control (TVC), and an IMU for inertial measurements. The system uses a blackboard pattern for thread-safe data sharing, implements a state machine for vehicle control with configurable priorities, and includes telemetry to send binary data to a ground station over UDP. Control algorithms in the algorithms/ subdirectory include second-order filters for signal processing, and the codebase features comprehensive configuration management through config.h for hardware mappings, thread priorities, safety limits, and communication parameters. The system is designed for real-time operation with SCHED_FIFO scheduling, includes graceful shutdown handling, and provides test utilities for validating individual hardware components.
- live telemetry plotting and logging
- multiple dashboards for a P&ID, general plots, and a telemetry searching box (for realtime debugging and such, which has already been very useful)
- Manual Valve commanding direct from the interface with execute + confirm. This will primarily be for loading ops and such.
- It's all based on React and Flask
I bought a clockwork uconsole so that while I'm literally in the field, working on or loading the vehicle with propellants, I can view telemetry, pressures, etc and command valve states as I need to. I plan on having folks help me out here, but sure helps to see a real time telemetry plot. (i posit)
AI's thoughts on Nexus:
The ground_scripts/Nexus/ directory contains a real-time telemetry monitoring and control system for the Lander Flight Software. It includes a Python Flask backend that receives UDP telemetry packets (binary and text) from the FSW on port ****, parses them using a binary telemetry parser, and serves the data via a REST API and WebSocket streaming. The React frontend provides a web dashboard with three main tabs: a Control interface for sending commands to the FSW (including valve controls with safety logging), a Charts tab with time-series visualizations using Plotly.js, and a Raw Data tab displaying telemetry metrics grouped by category (PDB, IMU, GPS, Control, ODrive). The system supports optional telemetry logging to JSON files, includes a startup script for launching both backend and frontend servers, and provides a REST API endpoint for sending commands back to the FSW, enabling bidirectional communication between the ground station and the vehicle.
Of course, while I think I have a reasonably good handle on the infrastructural things, I am still very much a noob when it comes to a lot of the intense low level software dev stuff. I also am really not a pro at the web-app flask/React stuff. But to be honest, AI tools are just insanely good at kickstarting me and getting me in the right direction. Obviously, just test along the way and spend more time running and testing the infra than writing it...
Remaining work for a static fire operation is the following:
- writing out the static fire state machine and event sequences
- flow characterizing the pintle and annulus, and the throttle valve flow map itself
- tune the pressure control schema, aborts etc
- going to rebuild the throttle valve itself, there's an asymmetry that is annoying me
- hook up my static fire thrust structure rig to the avionics, and calibrate that shit
- oh man then also fix some of the hardware here because im shit at designing things
- find a lox dewar and a solid set of friends willing to perform dubious tasks in the desert with me
- build out nexus as I test the system
I think my primary issue here is not just focusing and finishing my work; context switching penalties have got the best of me. I'd really like to fire this thing, put together a gnc stack, hover test it, and then full send it on some wack trajectory. Then i'm going to pivot to an entirely different field for projects, and maybe focus on applied mathematics.
some pics:
Some of the Nexus pages. It's quite ugly in my opinion at the moment, but I'll make it look nicer later, just need it to be functional to start. And as I dev more of the systems I'll change what plots I care about.
in action on megadesk
Control period plot over time. probably good enough for now, but keeping a close eye on this as I start implementing more gnc algorithms for the stack. it's pretty minimal right now
.png)
I ran a tank pressurization test (plot here is kinda lame but at least shows something). I pressed up the N2 tanks to like 500psi, tuned the regulators, and then commanded the press valves to press the individual propellant tanks to target pressures around 300psi. It all worked as expected, which was very shocking to me.
.png)
.png)
uconsole for field access to telemetry and commanding (I ordered in March and it came in October... so if you order one, don't expect it anytime soon).
Testing limit switches on all the linear actuators. I had to write a homing sequence because im too poor for absolute position sensors :) works fine tho. reboots work well too (afaict)
Had to do some tolerance cleanup passes on my friend Zach's lathe. Thanks Zach!
pintle fusion welding went off better than expected
I welded all the feed bungs to injector head. also i hate warping so much, even if you leave one screw off, you'll get very slight warpaccino on the injector plate. fafo
Jank static fire stand that needs to be rethought. Initial goal was to have multiple adaptor plates to support TVCs. But without the TVC add plate, there's zero clearance for tube routing lmao. Nobody said i was ever good at mechanical design.. to be fair i just freeball threw shit together in my garage for this one
A collection of my shitty hand made devices, embodying "perfect is the enemy of good enough"
Oh not sure if I mentioned, I replaced all the relay and wiring routing web of mess with a single mosfet valve driver board, and pared it down to 10 valves and 10 ADC inputs. The teensy samples and packets everything, and shoves it off to the flight computer, where all the real processing happens. currently sending packets over serial at 50hz. this is a far more elegant solution, and the design has held up under all sorts of heinous unrealistic stress testing. don't mind the fugly hand made battery lol
side shot
So much more to do! but it's gonna be frickin great!
I'm thinking about next steps post-lander. I think the framework I've built here is pretty sweet, and should be extensible to many real time systems. Call me crazy, but it'd be fun to make a fleet of autonomous things, all running on the same core software stack. or maybe i chill out and just do some math stuff or camera stuff
