Building a Robot Brain

How Robots Move

Most industrial robots move by blindly following pre-recorded coordinate paths. That works well in factories with rigid jigs, but if an object shifts by even a couple centimeters, the robot fails. Modern learned policies solve this by using neural networks to process camera pixels and joint sensors in real time, constantly adjusting their movement as the environment changes.

My benchmark task is a tabletop pick-and-place with a 7-DOF Franka Emika Panda: A 7-joint research robot arm with torque sensing in every joint. arm in the MuJoCo: A fast physics engine used heavily in robotics and RL research. Short for Multi-Joint dynamics with Contact. physics engine. A central workspace spawns either a red or green cube at a randomized position. The robot must look through its cameras, infer the color, descend, achieve a centered pinch grasp, transport the cube across the table to the corresponding landing pad (red to starboard/right at +0.2 m, green to port/left at -0.2 m), deposit it gently, and return to a ready pose.

Trained using Proximal Policy Optimization (PPO) paired with a demonstration-seeded reverse curriculum to bridge early exploration, the policy learns to coordinate vision, recurrent memory, and continuous velocity control. Privileged simulation state is used strictly during training to supervise an asymmetric critic and an auxiliary geometric grounding head. Both are discarded after training. At runtime, the deployed agent makes decisions purely from onboard sensory inputs, achieving a 98.7% task completion rate across cold-start evaluation rollouts.

To achieve this in deployment without hardcoding coordinates or using privileged state, the policy relies strictly on realistic onboard sensors:

• A global camera (128 × 128 RGB) gives an overhead view of the table and the landing pads.
• A wrist camera (128 × 128 RGB) gives an eye-in-hand view with local depth, alignment, and contact cues near the fingers.
• Proprioception is 18 numbers: 7 joint positions, 7 joint velocities, 2 gripper finger positions, and 2 finger velocities.

Why Mamba + LNN?

Robot learning leans heavily on Transformers, and I wanted to see how small a policy could get while still doing closed-loop pick-and-place from pixels. Pairing a Mamba State Space Model with a Liquid Neural Network gives a policy with a fixed-size memory and about 250,000 parameters. I haven't compared it against a plain GRU or MLP of the same size, so I can't tell you the architecture is what makes it work.

When I began researching recent work in adaptive robotics, I was looking for architectures designed specifically for continuous physical interaction. That search led directly to Liquid Neural Networks: Recurrent networks built on continuous-time equations, where the input changes how fast the internal state evolves. (Hasani et al., 2022) and their ability to approximately solve continuous neural differential equations in closed form. Around the same time, exciting work was emerging applying State Space Models to physical robot control, notably research papers like LocoMamba (vision-driven quadruped locomotion with Mamba) and StratMamba (a dual-memory Mamba for LiDAR obstacle avoidance on a quadruped).

I was also heavily inspired by recent work in real-time edge intelligence, such as "UAV Cybersecurity with Mamba-KAN-Liquid Hybrid Model: Deep Learning-Based Real-Time Anomaly Detection" (MDPI, 2025). That paper suggests that a hybrid architecture uniting Mamba's linear sequence memory with Liquid continuous dynamics can operate in strict real-time with high fidelity on physical edge systems.

If a hybrid model could process high-frequency sensor telemetry on an autonomous UAV, could a similar dual-rate hierarchy control the fine contact dynamics of a 7-DOF robotic arm? I wanted to test this in practice and see if I could contribute meaningful empirical findings to this emerging area of research.

Real robots are edge devices with hard power, thermal, latency, and memory budgets, and you can't mount a server rack of H100s to a robotic arm. I wanted a policy small enough that those budgets stop being a problem.

Even in simulation, a Transformer policy has to keep keys and values for every past step, which eats VRAM and limits how many environments a single GPU can run in parallel. A fixed-size recurrent state avoids that.

I believe we can engineer our way out of brute-force compute reliance by building smarter inductive biases: Assumptions baked into a model's structure so it needs less data to learn the right thing. directly into the network architecture.

Liquid networks use continuous-time cells, so they can model smooth velocity and torque curves with very few parameters. Mamba keeps a fixed-size recurrent memory that updates in constant $O(1)$ time per step and decides for itself which visual events to remember.

Mamba and Liquid Neural Networks may not end up being the best answer to the compute problem. Implementing them from scratch, finding where they break, and iterating on those failures is the point of this project.

A 250k-Parameter Robot Brain

Together, these inductive biases give a complete perception, memory, and motor control system totaling just 250,547 parameters. The Cortex (vision encoders plus Mamba) holds 161,444 of them and the Brainstem (CfC, actor, and critic) holds 89,103. The critic and the grounding head are only used in training, so the deployed policy is about 227k parameters, roughly 450 KB in FP16.

For scale, RT-1, a Transformer-based manipulation policy, has about 35 million parameters, roughly 140 times more.

Parameter count alone does not tell the full compute story. The two camera encoders only have about 68,000 weights, which is tiny, but because convolutional filters slide across every single pixel in an image, they require roughly 71 million MACs: A MAC is one multiply plus one add. Convolutions need a lot of them. (multiply-accumulate math steps) for every visual frame. At 10 frames a second, that is about 1.4 billion calculations per second (1.4 GFLOP/s).

This is why the 5:1 multi-rate split is such an advantage. By restricting the heavy image processing to just 10 times a second, the fast 50 Hz motor loop only has to crunch 18 joint numbers on each tick. That keeps the high-frequency control loop lightweight.

The deployed weights are about 450 KB in FP16 (227 KB in INT8), so on paper they fit in the 1 MB of SRAM on a Teensy 4.1. I haven't run any of this on real hardware, and the vision encoder's activations would need to fit too, so treat that as a back-of-envelope result.

The Multi-Rate Sensorimotor Dilemma

Building an end-to-end visual policy presents an immediate physical paradox: vision is slow and computationally heavy, but robot dynamics must be fast.

Take highway driving. You don't re-evaluate your lane 100 times a second. You glance down the road once or twice a second (my 10 Hz Cortex) while your hands make constant small corrections on the wheel (my 50 Hz Brainstem).

I borrowed the cortex and brainstem names from biology, where vision takes tens of milliseconds to process and spinal reflexes close the loop much faster. In physical simulation, camera rendering is the dominant bottleneck—rendering at 50 Hz consumes 95% of total environment rollout time. But if you slow the whole robot down to 10 Hz to match vision, motor control becomes jerky, contact dynamics bounce unpredictably, and the arm frequently triggers simulator collision guardrails.

I solve this by decoupling the agent into a 5:1 multi-rate hierarchy:

1. The Mamba Cortex runs at 10 Hz (every 100 ms). It processes both camera images, tracks visual history across the episode, and compresses the scene into a 64-dimensional context vector $C_t$.
2. The Liquid Brainstem runs at 50 Hz (every 20 ms), five ticks per visual update. It takes the latched context vector $C_t$ alongside fresh, high-frequency joint proprioception to command smooth Cartesian velocity deltas.

The Mamba Cortex

A pick-and-place episode lasts up to 900 control steps (18 seconds at 50 Hz, or 180 cortex updates at 10 Hz). Throughout that sequence, the visual environment undergoes drastic transitions: the cube is spotted, approached, occluded by the gripper during the reach, lifted into the air, and moved to the landing zone. The agent needs a sequence model to maintain temporal continuity.

Why Mamba Over Transformers or Standard RNNs?

A Transformer keeps a key and a value for every past step and attends over all of them, so memory grows with episode length and attention compute grows quadratically. An LSTM or GRU keeps a fixed-size state and gates how much to overwrite, but it can't be trained in parallel over time. Mamba also keeps a fixed-size state, and its input-dependent parameters decide how much of each new frame gets written. That's the property I wanted.

Beyond constant $O(1)$ step-time inference, Mamba's linear hidden state update allows entire trajectory sequences to be trained in parallel on GPUs via an associative prefix scan: A way to compute every running total of a sequence in parallel instead of one step at a time. It works because the update is linear. , which avoids the slow sequential backpropagation through time that traditional RNNs need.

Fixed Memory with Dynamic Input Valves

In classical State Space Models (SSMs), the system maps continuous input streams into a hidden memory state using static, unchanging parameters:

The block I implemented is Mamba-3, which swaps the rule below for an exponential-trapezoidal discretization that also mixes in the previous input, and adds complex-valued state. I'm showing the original zero-order-hold form because it's the simplest way to see what the timescale $\Delta$ does.

Discretized under a timescale step size $\Delta$ using Zero-Order Hold (ZOH): Treating the input as constant over each time step. It is how you turn a continuous equation into discrete steps. :

In older linear models like S4, $\Delta ,\mathbf{A},\mathbf{B},$ and $\mathbf{C}$ are fixed constants. The key change in Mamba is making $\Delta (x_t),\mathbf{B}(x_t),$ and $\mathbf{C}(x_t)$ dynamic functions of the current input $x_t$.

This selective mechanism allows the cortex to regulate how strongly incoming observations update the stored 64-dimensional context vector $C_t$.

In robot control, when key visual milestones occur, such as spotting the target landing pad after lift, the learned weights evaluate the sensory frame and drive $\Delta$ open to commit the new destination coordinates to memory. When observations provide steady or redundant feedback, $\Delta$ contracts, causing $\overline{\mathbf{A}}\to \mathbf{I}$ and $\overline{\mathbf{B}}\to 0$, stabilizing the stored trajectory plan ($h_t\approx h_{t-1}$) against state drift.

SpatialSoftmax & Perception

Before visual frames can enter the Mamba sequence model, they must be processed by convolutional layers. But visual RL has a common problem: flattening into a linear layer throws away spatial structure.

The usual alternative flattens the 2D feature maps into one long vector and feeds it to a linear layer. That throws away where things are in the image and ties the layer size to the image resolution.

I replace the linear bottleneck with SpatialSoftmax: A layer that turns each feature map into one (x, y) point, the average location of its activation. (Levine et al., 2016). Instead of passing raw pixels, it computes the center of mass of feature activations, outputting explicit normalized coordinate landmarks:

What the Robot Actually Sees

Profiling in MuJoCo confirmed that GPU rendering costs are largely a fixed per-call overhead regardless of image resolution (64 px vs 160 px). What saves compute is rendering frequency (10 Hz vs 50 Hz). Keeping the camera resolution at a clean 128 × 128 keeps the convolutional feature extractor lightweight, avoids overfitting to micro-surface textures, and minimizes GPU VRAM consumption across thousands of parallel environments.

In the Attention Overlay (Tab 3 above) you can see a division of labor I didn't design. The overhead camera's red keypoints stay anchored along the table's center line and the landing pads, so those filters act as a fixed world reference. The wrist camera's cyan keypoints do the opposite: they cluster on the cube's top face and follow it as the gripper descends, grasps, and lifts.

The Proprioception Suppression Problem, FiLM & Auxiliary Grounding

When training dual-stream policies (vision + joint proprioception), gradient descent can discover a damaging shortcut.

In neural networks, joint sensors produce clean, immediate feedback, while camera pixels produce noisy, delayed signals. If you just glue vision and joint data together in a simple layer, the training algorithm gets lazy. It starts ignoring the camera and tries to steer the arm using joint momentum alone.

To force the agent to keep its eyes open, I apply two complementary architectural constraints:

1. Multiplicative FiLM conditioning (Perez et al., 2018). Instead of simple concatenation, visual keypoint embeddings generate affine parameters $(\gamma ,\beta )$ that scale and shift the proprioceptive embedding. Vision is a volume knob for the motor stream: collapsing the visual weights collapses the dynamic scaling $(1+\gamma )$, which makes it harder for the optimizer to ignore pixels:

2. Auxiliary keypoint grounding loss. FiLM alone isn't enough: if the visual generator weights stay near zero, $\gamma \approx 0$ and $\beta \approx 0$ leave proprioception virtually unmodulated. What keeps vis_emb honest is an auxiliary regression head trained via Mean Squared Error to predict the true 3D gripper-to-cube offset vector ($p_{\text{cube}}-p_{\text{gripper}}$). This forces the convolutional and SpatialSoftmax layers to extract accurate spatial geometry directly from pixels regardless of policy reward noise.

The Liquid Brainstem

Next is the motor half of the architecture, and the main reason I chose a Liquid Neural Network (CfC): A liquid network cell that uses an approximate closed-form solution, so no ODE solver is needed at run time. .

Operating on Stale Visual Context

Under the 5:1 multi-rate execution ratio, the visual cortex updates at 10 Hz (every 100 ms) while the motor brainstem updates at 50 Hz (every 20 ms). This means that for 4 out of every 5 ticks, the controller must command actions using an identical, latched visual context vector $C_t$ while receiving fresh 50 Hz joint feedback.

A memoryless MLP sees the same stale context on every tick between visual updates, so any noise in the joint readings goes straight into the action. A CfC cell carries a hidden state forward, so between updates it keeps moving along the trajectory it was already on and averages out sensor noise.

The brainstem achieves this continuous smoothing using a CfCCell: One step of a CfC network, written in closed form. , which computes ODE-derived state updates in closed form:

Note on sampling rate: In a fixed-rate controller ($\Delta t=0.020\text{ s}$), the time-dependent head $W_{a}x\cdot \Delta t$ is algebraically redundant with $W_{b}x$. Setting fixed_rate=True in the cell merges these heads, saving ~10.6k parameters while preserving the core benefit of the Liquid network: its exponential blending recurrence.

Kinematics & Null-Space Control

If an RL policy outputs raw joint position deltas ($\Delta q\in {\mathbb{R}}^7$), independent exploration noise causes chaotic whipping: a tiny angular twitch at the base shoulder compounds down the kinematic chain, whipping the hand across the workspace and into the table.

Instead, the policy commands intuitive Task-Space Cartesian Deltas at the Tool Center Point (TCP):

I resolve these Cartesian velocities $v_e$ into 7 joint velocities $\dot{q}$ using Damped Least Squares (DLS) Inverse Kinematics:

Put your palm flat on your desk and hold it still. You can still swing your elbow up, down, left, and right without moving your palm a millimeter, and that's null-space redundancy. The Panda has 7 joints but the task only constrains 6 (3 position, 3 orientation), so one degree of freedom is left over. Without something to pin it down, the elbow drifts into awkward poses. The projection matrix $(\mathbf{I}-J^{†}J)$ adds a gentle pull toward a natural resting posture without disturbing the gripper's path.

The Traps I Walked Into

Reinforcement learning has an uncanny talent for finding mathematical and physical loopholes in whatever environment you hand it. This project found several of its own.

Reward shaping turned out to be way harder to get right than I expected. Small shaping terms kept interacting with each other in ways I didn't predict, and more than once the policy found some shortcut that technically satisfied the reward without actually doing the task.

I had an issue where the arm scored well for just getting close to the cube, but it wasn't motivated enough to actually close the gap and grasp it. So the policy just learned to stop short, hovering the gripper a few centimeters above the cube and pinching at empty air over and over, farming the approach bonuses without ever committing to the grasp. Part II will cover how I get that fixed.

As you may have noticed in the video above, the table texture I was using looks a little wonky, kind of broken. Well, in Part II I tried to fix that and swapped in a properly textured table that actually looked table-shaped, and it broke the model. I'll get into how I fixed that and how it ties into training best practices. Part II also covers training at scale on cloud GPUs and modeling the Franka's real grip physics to improve its success rate.