Real-Time Nonlinear MPC in the Browser

Real-Time Computation Headaches

Getting the NMPC code to run two hundred times a second is where I struggled a bit. Part I's offline solver took about 2 seconds to compute a single full path. Doing that inside a 5ms budget, in JavaScript no less, requires some careful coding.

But I have a few tricks up my sleeve. For starters, I don't need to compute the entire trajectory every time, I only need to compute a segment of it which is determined by the horizon. Second, from the previous solve I already have an answer that, unless something crazy happened, will be a pretty good guess for the current moment as well. So instead of just throwing away ALL that old data, I just throw away the part that I already used (the first step) and I start the new optimization from the previous one's solution. This warm start strategy gets a "good enough" answer in a single iteration so there is no need to iterate to full convergence. This is called Real-Time Iteration: Each tick gets a single solver step instead of a full solve, and the answer keeps improving as the ticks roll by. A little sloppy per tick but very fast overall. .

Here's a live demo of NMPC in action compared to the TVLQR tracker from Part I. In this example the NMPC is also tracking the pre-computed trajectory so its not fully independent yet. If you add a disturbance to the system you can see how they vary in their response. TVLQR tries to erase the whole error immediately and it doesn't work very well. The NMPC plans a longer way back with less corrective force. NMPC doesn't always catch either, but it wins more often than the TVLQR tracker does. The horizon and rate dials connect to the NMPC behavior directly. The zero-order hold: Holding the motor command steady between updates, like a staircase instead of a smooth curve. Every digital controller does it, so the slower the loop, the bigger each step. starts to cause problems around 30Hz as the correction force is too out of sync with reality.

Swinging Up From Scratch

Tracking Part I's plan works great, but the promise of this article is that we do everything live. If we keep using the pre-computed trajectory it kinda sucks the fun out of everything.

So let's take the training wheels off. All the controller will be given is a goal of trying to get the energy of the pendulum equal to that of the energy in the pendulum when it's standing upright at equilibrium.

That may seem strange at first. I don't tell it anything about where or how it should do that. It works out on its own that pumping back and forth is the only thing it can do to add the energy it needs as it searches over sequences of forces and finds which actions score best.

However, I now have a problem since I ditched the offline plan to warm-start from. The controller has to discover the whole maneuver on its own, needing 15 to 20 iterations every tick instead of just one. Getting that to actually work turned into a series of headaches.

Getting It to Move at All

When I first tested the from-scratch controller, it didn't move at all. The optimizer was not throwing errors, but the cart just sat there hanging. The problem was how the angle was measured at the hanging starting position. I made a mistake with my wrapping: Adding or subtracting full turns until an angle lands in one standard range, so the same pose always gets the same number. function.

In Part I I had a bug where a link would spin around 360° because I didn't wrap the angle values so the error was showing as greater than it actually was. So, in Part I I wrapped the values between −π and π and that solved my problem. For NMPC, the terminal cost has a catch bowl: A cost that grows like a bowl around the upright pose, so the optimizer always has a downhill direction to follow toward it. and I'd implemented it by wrapping the joint angle into (−π, π] like I did before. Worked before, so should work this time, right? Well... no. The simulation always starts hanging where θ = π, which is where the (−π, π] wrapping function has a discontinuity.

The solver builds its derivatives by finite differencing (nudge left, nudge right, subtract the value). Doing that at the start when θ = π meant that one side of the equation was near +π and the other was near −π. This caused the second derivative to be very very negative, which the solver treats as something going terribly wrong. To protect itself, it injects heavy damping to force the curvature back positive. Because motor commands are computed by dividing the correction by that damped curvature, dividing by such a massive damping term made the commanded force effectively zero.

The fix is dead simple, just use a different wrapping function. 1 − cos θ has the same bowl I wanted, without the discontinuity right at the resting position. Lesson learned. Picking a wrapping function with a break in it was the wrong call for something that touches every angle, since sooner or later the system lands exactly on the break, and here "sooner" was literally frame one.

Getting It Fast Enough

With the solver no longer blowing up, the next problem was speed. A from-scratch swing-up needs around twenty iterations a tick. My initial solve took around 25ms per tick, which blew way past my 5ms budget. The solver was struggling with heavy math and constant garbage collection.

My first implementation calculated the Jacobian: A table of how much every output changes when you nudge each input. For this machine, that's how next tick's state shifts if you poke the current state or the force a little. the lazy (as in easier to implement) way by running fourteen separate physics evaluations at every trajectory knot. I tried replacing the lazy approach with a standard textbook formula. Accelerations come from $a=M^{-1}r$, where $r$ is the vector of applied forces. Differentiating that requires differentiating the inverse mass matrix:

Funny enough, this benchmarked even slower than my original code. Profiling showed that creating the 3×3 matrices in JavaScript cost more than the fourteen physics rollouts it removed.

Luckily I can simplify the math by reusing work from earlier. Multiplying the textbook formula by the force vector $r$ lets us substitute the acceleration vector $a=M^{-1}r$ directly into the right side:

With this approach every intermediate calculation is now just a plain scalar number. The Jacobian dropped from 5.5 microseconds down to about 1 microsecond per knot. I wrote a verification script testing this formula against my old finite-difference code across 500 random states to confirm no degradation happened. All came out clean.

The rest of the time savings came from reducing memory allocations. The backward pass calculates several matrices and vectors at every knot. My naive code built all of them out of fresh nested arrays of numbers (number[][]). A single 6×6 matrix created an outer array holding six inner arrays on the heap. Vectors were fresh number[] arrays. The line search also allocated new arrays of state vectors (State[]) for each candidate step size. That created over 100,000 short-lived array objects in a single solve and triggered repeated garbage collection pauses.

// 6x6 matrix: 7 heap arrays, slow nested dereference Vxx[r][c]

const Vxx: number[][] = Array.from(

  { length: 6 },

  () => new Array(6).fill(0)

);

Vxx[r][c] = cost;

// Physics step: allocates fresh 6-element State array every call

function step(s: State): State {

  return [s[0] + dt * s[1], s[1] + dt * a[0], ...];

}

// Line search: allocates 50 new trajectory state arrays per step

const xsNew: State[] = new Array(50);

// 6x6 matrix: 1 flat buffer using row-major indexing: r * 6 + c

const Vxx = new Float64Array(36);

Vxx[r * 6 + c] = cost;

// Physics step: writes 6 states [x, v, θ1, ω1, θ2, ω2] into scratch

function stepInto(out: Float64Array, s: State): void {

  out[0] = s[0] + dt * s[1]; // cart position

  out[1] = s[1] + dt * a[0]; // cart velocity

  // ... links 1 & 2 (indices 2 through 5)

}

// Line search: zero-allocation pointer swap between iterations

let temp = xs;

xs = xsNew;

xsNew = temp;

Replacing those nested arrays with preallocated Float64Array scratch buffers dropped each physics evaluation from 87 nanoseconds down to 58 nanoseconds. More importantly, it killed garbage collection pauses entirely.

I originally considered rewriting the solver in WebAssembly (WASM). It turns out I did not need a faster language. I just needed to "get good". The entire solve dropped from over 25ms down to just 1.2ms per tick. That comfortably fits inside my 5ms budget.

Getting It to Actually Stay Up

With the solver now fast and correct, the pendulum could finally be caught. However, it still didn't know how to behave once it was up there.

The energy target on its own turned out to be a bad objective. It was just too simple. A pendulum whipping past at high speed can carry the exact same total energy as one balanced calmly upright. They are treated as equal so nothing encouraged a controlled catch. I also didn't add anything to control the cart, which wandered off the end of the rail while the links kept flailing.

I spent some time wrestling with the cost weights to fix this. I cranked up the penalty on motor effort so the cart chilled out a bit. I added a centering term to pull it back toward the middle of the track. I also penalized joint velocities so there was something to stop the links form whipping around out of control. Eventually I got a repeatable swing-up. Tuning was still pure whack-a-mole though. Nudging any single value past its sweet spot tended to immediately break something else.

Even "tuned", it was still failing around 25% of the time with a runaway cart. This is where I ran into the short-sightedness issue with an MPC controller. Controllers that use pre-computed plans can afford to be short-sighted, because the plan already did the long-range thinking. One without a plan cannot.

With a short time horizon it simply commits to whatever move looks best for the time that it has. With a time horizon of 0.5 seconds the optimizer can't see far enough ahead to know it'll need to come back, so it yeets itself off to the side. Counterintuitively, a harder rail limit made that worse. But if I just give it a marginally larger 1 second horizon the controller can find the pump to catch maneuver. It never even leaves the rail this way so no extra constraints required. With a long enough horizon the catch rate rose back to nearly 100%.

One problem remained after all those fixes — the cart kept drifting after a catch, even with the pendulum balanced well. Energy is the right objective for swinging up but its a pretty useless one for balancing. I needed to split up the behavior into two parts, like in Part I, and Part I's Riccati solution already gives me the number needed here. $P$, the total future cost of being in any given state near upright.

Blending that in as a terminal cost once it nears the linearization region gives the planner a way to control the balance. With this the cart settles and the drift disappears.

The Result

All of those speedbumps lead to this. The from-scratch controller, no precomputed anything, replanning its whole future from wherever it is. The faint links fanning out ahead of it are the solver's own predictions. They are for the horizon it just solved, redrawn from scratch every tick. Kick it and watch that prediction reroute in real time, in pure JS baby.

What I find funny about this write up is how little of the actual work for me in making this was control theory. The controller that finally works is about fifteen lines of cost function and it is basically straight out of a textbook. What took the most time was silly mistakes I kept making like having a wrapping function with a discontinuity sitting on the start state, lazy programming practices, and tweaking the terminal cost. As with most things though, the ordinary business of making something actually run as intended is most of the job.

One cool thing about debugging this controller is that all of these issues are findable because I could set a breakpoint and see what the heck it is doing. A trained policy does not come with this luxury, so debugging it is a lot more like guessing.