Building a Robot Brain

How Robots Move

Most industrial robots move by blindly following pre-recorded coordinate paths. That works well in factories with rigid jigs, but if an object shifts by even a couple centimeters, the robot fails. Modern learned policies solve this by using neural networks to process camera pixels and joint sensors in real time, constantly adjusting their movement as the environment changes.

My benchmark task is a tabletop pick-and-place with a 7-DOF Franka Emika Panda: A popular 7-degree-of-freedom collaborative robotic manipulator known for torque control and human-like dexterity. arm in the MuJoCo: Multi-Joint dynamics with Contact: a fast physics engine designed for robotics and reinforcement learning. physics engine. A central workspace spawns either a red or green cube at a randomized position. The robot must look through its cameras, infer the color, descend, achieve a centered pinch grasp, transport the cube across the table to the corresponding landing pad (red to starboard/right at +0.2 m, green to port/left at -0.2 m), deposit it gently, and return to a ready pose.

Trained using Proximal Policy Optimization (PPO) paired with a demonstration-seeded reverse curriculum to bridge early exploration, the policy learns to coordinate vision, recurrent memory, and continuous velocity control. Privileged simulation state is used strictly during training to supervise an asymmetric critic and an auxiliary geometric grounding head—both discarded after training. At runtime, the deployed agent makes decisions purely from onboard sensory inputs, achieving a 98.7% task completion rate across cold-start evaluation rollouts.

To achieve this in deployment without hardcoding coordinates or using privileged state, the policy relies strictly on realistic onboard sensors:

• Global Camera (128 × 128 × 3 RGB): Overhead overview framing the table workspace and target landing pads.
• Wrist Camera (128 × 128 × 3 RGB): Eye-in-hand viewpoint providing local depth, alignment, and contact cues near the fingers.
• Proprioception (18-dimensional): 7 joint positions ($q_{\text{pos}}$), 7 joint velocities ($q_{\text{vel}}$), 2 gripper finger positions, and 2 finger velocities.

Why Mamba + LNN?

Standard robot learning relies heavily on Transformers, but Transformers are huge and power hungry. Pairing a Mamba State Space Model with a Liquid Neural Network creates a policy that runs with constant memory, adapts smoothly over time, and uses only about 250,000 parameters. That means you get intelligent closed-loop control that can easily run on small edge computers without needing a rack of hot GPUs.

When I began researching recent breakthroughs in adaptive robotics, I was looking for architectures designed specifically for continuous physical interaction. That search led directly to Liquid Neural Networks: Liquid Neural Networks: A class of bio-inspired neural models derived from continuous dynamical systems whose parameters adapt smoothly over time. (Hasani et al., 2022) and their ability to solve continuous neural differential equations in closed form. Around the same time, exciting work was emerging applying State Space Models to physical robot control—notably research papers like LocoMamba (using Mamba for agile quadruped locomotion) and StratMamba (applying structured SSMs to hierarchical decision-making).

I was also heavily inspired by recent work in real-time edge intelligence, such as "UAV Cybersecurity with Mamba-KAN-Liquid Hybrid Model: Deep Learning-Based Real-Time Anomaly Detection" (MDPI, 2025). That paper proved that a hybrid architecture uniting Mamba's linear sequence memory with Liquid continuous dynamics can operate in strict real-time with high fidelity on physical edge systems. If a hybrid model could process high-frequency sensor telemetry on an autonomous UAV, could a similar dual-rate hierarchy control the fine contact dynamics of a 7-DOF robotic arm? I wanted to test this in practice and see if I could contribute meaningful empirical findings to this emerging area of research.

Beyond the literature, I had a fundamental philosophical and engineering motivation: the assumption that we can infinitely scale Transformer compute is economically and physically unsustainable.

The modern AI playbook has largely converged on a single brute-force heuristic: make the Transformer bigger, rent thousands more datacenter GPUs, and let massive VRAM swallow the problem. In robotics, this breaks down. Real robots are edge devices with hard power, thermal, latency, and memory budgets. You cannot mount a server rack of H100s to a robotic arm.

Even in simulation, running parallel reinforcement learning rollouts with quadratic Key-Value attention caches ($O(T^2)$) over 900-step horizons causes extreme VRAM bloat, crippling the number of concurrent environments a single GPU can simulate.

I believe we can engineer our way out of brute-force compute reliance by building smarter inductive biases: The set of structural assumptions, mathematical constraints, and architectural priors embedded into a machine learning model to help it generalize effectively to unseen data without needing infinite training examples. directly into the network architecture:

• Continuous Dynamical Stability (Liquid Networks): Instead of treating time as artificial discrete hops, continuous-time ODE cells naturally model physical velocity and torque curves with lightweight parameter footprints.
• Selective Linear Memory (Mamba SSM): Instead of caching hundreds of past visual frames in quadratic memory, Mamba maintains a compact, fixed-size recurrent memory that updates in constant $O(1)$ step-time while dynamically deciding which visual events to remember and which to discard.

Mamba and Liquid Neural Networks may not end up being the ultimate, final answer to the Transformer compute bottleneck. But by implementing them from scratch, evaluating them in simulation, identifying their failure modes, and iterating on their shortcomings, we move robotics engineering in the right direction: toward lighter, faster, and physically grounded intelligent control.

A 250k-Parameter Robot Brain

The culmination of these inductive biases is a complete perception, memory, and motor control system totaling just 250,547 parameters (~227k active during robot deployment):

To see what this means in practice, compare this footprint against established robot manipulation policies and on-device models:

Parameter count alone does not tell the full compute story. The two camera encoders only have about 68,000 weights, which is tiny, but because convolutional filters slide across every single pixel in an image, they require roughly 71 million MACs: Multiply-Accumulate operations: a standard measure of AI compute. One MAC is one multiplication plus one addition (a × b + c). 71 million MACs means the computer does 71 million math steps for each camera frame. (multiply-accumulate math steps) for every visual frame. At 10 frames a second, that is about 1.4 billion calculations per second (1.4 GFLOP/s).

This is why the 5:1 multi-rate split is such an advantage. By restricting the heavy image processing to just 10 times a second, the fast 50 Hz motor loop only has to crunch 18 joint numbers on each tick. That keeps the high-frequency control loop fast and lightweight so it never overheats a small onboard robot computer.

To see what this means on physical silicon, compare how this policy executes across common embedded hardware platforms:

Memory-wise, the active deployment weights take up under 500 KB in FP16 (and just 227 KB in INT8), meaning the entire network fits directly inside the 1 MB internal SRAM of a $30 Teensy 4.1 without any external RAM chips. Computing a 50 Hz motor step takes under 0.25 milliseconds (barely 1.2% of its 20 ms cycle).

However, running naive, unoptimized 128×128 image convolutions on that same single core takes ~118 ms, slightly overrunning the 100 ms frame window. Stepping up to a $15 quad-core Raspberry Pi Zero 2 W easily absorbs that vision load at just 18% of a single core. Alternatively, on the Teensy itself, a few image processing and MAC-reducing optimizations bring the vision latency well within budget—which I will cover in a separate write-up.

The Multi-Rate Sensorimotor Dilemma

Building an end-to-end visual policy presents an immediate physical paradox: vision is slow and computationally heavy, but robot dynamics must be fast.

Think of driving a car down the highway. Your brain does not process 100 high-resolution snapshots every second to re-evaluate your route and lane choices. You glance down the road once or twice a second to check your lane and spot distant traffic (10 Hz Cortex), while your hands and wrists make continuous, subconscious micro-adjustments on the steering wheel to keep the car tracking smoothly (50 Hz Brainstem).

In biological systems, the optic pathway in the mammalian cortex takes tens of milliseconds to extract visual meaning, while reflex arcs in the brainstem and spinal cord fire at hundreds of Hertz using instant proprioceptive feedback. In physical simulation, camera rendering is the dominant bottleneck—rendering at 50 Hz consumes 95% of total environment rollout time. But if you slow the whole robot down to 10 Hz to match vision, motor control becomes jerky, contact dynamics bounce unpredictably, and the arm frequently triggers simulator collision guardrails.

I solve this by decoupling the agent into a 5:1 multi-rate hierarchy:

1. The Mamba Cortex (10 Hz / 100 ms period): Processes global and wrist camera images, tracks sequential visual history across the episode, and compresses the visual scene into a 64-dimensional context vector $C_t$.
2. The Liquid Brainstem (50 Hz / 20 ms period): Ticks five times for every single visual update. It consumes the latched context vector $C_t$ alongside fresh, high-frequency joint proprioception to command smooth Cartesian velocity deltas.

The Mamba Cortex

A pick-and-place episode lasts up to 900 control steps. Throughout that sequence, the visual environment undergoes drastic transitions: the cube is spotted, approached, occluded by the gripper during the reach, lifted into the air, and moved to the landing zone. The agent needs a sequence model to maintain temporal continuity.

Why Mamba Over Transformers or Standard RNNs?

To see why Mamba is uniquely suited for robotics, compare how different architectures handle memory:

• A Transformer is like a student who video-records every single second of a 3-hour lecture. They have perfect recall, but to answer a question on an exam, they have to re-watch the entire recording from start to finish. Over a 900-step episode, this quadratic Key-Value caching ($O(T^2)$) exhausts GPU VRAM and cripples parallel simulation throughput.
• A Standard RNN (LSTM/GRU) is like a student writing notes on an Etch-a-Sketch that fades at a fixed, indiscriminate rate every minute—eventually blurring critical early events regardless of whether new information arrived.
• Mamba (Selective SSM) is like a student taking notes with a pen and highlighter. When critical visual cues or stage transitions occur, they write furiously ($\Delta$ opens). When sensory frames are steady or redundant, the student lifts the pen, keeping their notes intact without overwriting them with meaningless noise.

Beyond constant $O(1)$ step-time inference, Mamba's linear hidden state update allows entire trajectory sequences to be trained in parallel on GPUs via an associative prefix scan: A parallel GPU algorithm (like a running cumulative sum) that computes every step's hidden state simultaneously in parallel using binary trees, rather than stepping through the sequence one tick at a time. , eliminating the slow sequential backpropagation through time required by traditional RNNs.

Fixed Memory with Dynamic Input Valves

In classical State Space Models (SSMs), the system maps continuous input streams into a hidden memory state using static, unchanging parameters:

Discretized under a timescale step size $\Delta$ using Zero-Order Hold (ZOH): A standard method for converting continuous physics/differential equations into discrete time steps by assuming the input signal stays constant (held flat) over each time step interval Δ. :

In older linear models like S4, $\Delta ,\mathbf{A},\mathbf{B},$ and $\mathbf{C}$ are fixed constants. Mamba's breakthrough is making $\Delta (x_t),\mathbf{B}(x_t),$ and $\mathbf{C}(x_t)$ dynamic functions of the current input $x_t$.

This selective mechanism allows the cortex to regulate how strongly incoming observations update the stored 64-dimensional context vector $C_t$. To see how each mathematical piece operates, map the equations directly to a student taking notes during a lecture:

A student on Day 1 (untrained weights) either writes down every single cough and sneeze or writes nothing at all. Through practice and studying past tests (gradient descent during RL training), the projection weights learn to recognize the professor's subtle cues so that $\Delta$ triggers at the exact right moments.

In robot control, when key visual milestones occur—such as spotting the target landing pad after lift—the learned weights evaluate the sensory frame and drive $\Delta$ open to commit the new destination coordinates to memory. When observations provide steady or redundant feedback, $\Delta$ contracts, causing $\overline{\mathbf{A}}\to \mathbf{I}$ and $\overline{\mathbf{B}}\to 0$, stabilizing the stored trajectory plan ($h_t\approx h_{t-1}$) against state drift.

SpatialSoftmax & Perception

Before visual frames can enter the Mamba sequence model, they must be processed by convolutional layers. But standard visual RL suffers from a notorious failure mode: flatten-and-linear layers destroy spatial structure.

If you ask a friend where your car keys are on a cluttered table, they don't describe the color, brightness, and texture of every square millimeter of wood grain across the entire surface. They just point their finger directly at the keys: "they are right there."

Standard neural networks do the opposite: they flatten 2D feature maps into thousands of unconstrained weights. If you change your camera resolution from 64 × 64 to 128 × 128, the flattened vector quadruples in size, completely invalidating trained network weights.

I replace the linear bottleneck with SpatialSoftmax: A differentiable layer that computes the spatial expected value of feature activations, outputting explicit (x, y) coordinate landmarks per feature channel. (Levine et al., 2016). Instead of passing raw pixels, it computes the center of mass of feature activations, outputting explicit normalized coordinate landmarks:

What the Robot Actually Sees

Profiling in MuJoCo confirmed that GPU rendering costs are largely a fixed per-call overhead regardless of image resolution (64 px vs 160 px). What saves compute is rendering frequency (10 Hz vs 50 Hz). Keeping the camera resolution at a clean 128 × 128 keeps the convolutional feature extractor lightweight, avoids overfitting to micro-surface textures, and minimizes GPU VRAM consumption across thousands of parallel environments.

Watching the SpatialSoftmax Attention Overlay (Tab 3 above) reveals a natural division of labor discovered during training. On the left, the overhead camera's red keypoints stay tightly anchored along the table's center dividing line and landing pads. Because the table is stationary in the global frame, these filters learn to act as a world coordinate anchor, giving the policy a constant reference datum for where the sorting boundary lies.

On the right, the eye-in-hand camera's cyan keypoints do the opposite: they dynamically cluster, stretch, and track the cube's top face and centroid as the gripper descends, grasps, and lifts. The wrist camera handles fine-grained local tracking relative to the fingers, while the global camera anchors the workspace frame of reference.

The Proprioception Suppression Problem, FiLM & Auxiliary Grounding

When training dual-stream policies (vision + joint proprioception), gradient descent often discovers a damaging shortcut.

Think of walking across your bedroom in the middle of the night. Straining your eyes in the dark to make out faint shapes is slow and difficult. Counting your steps and feeling the wall with your hand (joint proprioception) is effortless and immediate. You quickly learn to navigate purely by muscle memory with your eyes closed—which works fine until someone leaves a laundry basket in the middle of the room.

In neural networks, joint sensors produce clean, immediate feedback, while camera pixels produce noisy, delayed signals. If you just glue vision and joint data together in a simple layer, the training algorithm gets lazy. It starts ignoring the camera completely and tries to steer the arm using joint momentum alone, like trying to navigate a room with your eyes closed.

To force the agent to keep its eyes open, I apply two complementary architectural constraints:

1. Multiplicative FiLM Conditioning (Perez et al., 2018): Instead of simple concatenation, visual keypoint embeddings generate affine parameters $(\gamma ,\beta )$ that scale and shift the proprioceptive embedding. Vision acts as a mandatory volume knob for the motor stream: collapsing the visual weights collapses the dynamic scaling $(1+\gamma )$, preventing the optimizer from ignoring pixels:

2. Auxiliary Keypoint Grounding Loss: FiLM alone is not a guarantee—if visual generator weights stay near zero, $\gamma \approx 0$ and $\beta \approx 0$ leave proprioception virtually unmodulated. What keeps vis_emb honest is an auxiliary regression head trained via Mean Squared Error to predict the true 3D gripper-to-cube offset vector ($p_{\text{cube}}-p_{\text{gripper}}$). This forces the convolutional and SpatialSoftmax layers to extract accurate spatial geometry directly from pixels regardless of policy reward noise.

The Liquid Brainstem

Next is the motor half of the architecture—and the primary reason for choosing a Liquid Neural Network (CfC): Closed-form Continuous-time Neural Network: An ODE-derived neural network model that computes continuous-time dynamical state trajectories in closed form. .

Operating on Stale Visual Context

Under the 5:1 multi-rate execution ratio, the visual cortex updates at 10 Hz (every 100 ms) while the motor brainstem updates at 50 Hz (every 20 ms). This means that for 4 out of every 5 ticks, the controller must command actions using an identical, latched visual context vector $C_t$ while receiving fresh 50 Hz joint feedback.

Imagine walking through a dark room with a strobe light flashing once every second:

• A memoryless MLP panics between flashes. Every time the room goes dark, it has zero memory of where it was heading half a second ago. Any minor noise spike in sensor readings causes it to twitch and jerk violently.
• A Liquid CfC Network acts like physical momentum. It saw the doorway during the flash, so between flashes it smoothly glides forward along its continuous dynamical trajectory, absorbing high-frequency sensor noise without chattering.

The brainstem achieves this continuous smoothing using a CfCCell: CfCCell: Closed-form Continuous-time recurrent cell derived by Hasani et al. (Nature Machine Intelligence, 2022). , which computes ODE-derived state updates in closed form:

Note on sampling rate: In a fixed-rate controller ($\Delta t=0.020\text{ s}$), the time-dependent head $W_{a}x\cdot \Delta t$ is algebraically redundant with $W_{b}x$. Setting fixed_rate=True in the cell merges these heads, saving ~10.6k parameters while preserving the core benefit of the Liquid network: its exponential blending recurrence.

Kinematics & Null-Space Control

What action space should the policy output? When you reach for a coffee mug, you don't calculate individual joint angles: "rotate shoulder 14°, flex elbow 22°, twist wrist 5°".

If an RL policy outputs raw joint position deltas ($\Delta q\in {\mathbb{R}}^7$), independent exploration noise causes chaotic whipping: a tiny angular twitch at the base shoulder compounds down the kinematic chain, whipping the hand across the workspace and smashing the table.

Instead, the policy commands intuitive Task-Space Cartesian Deltas at the Tool Center Point (TCP):

I resolve these Cartesian velocities $v_e$ into 7 joint velocities $\dot{q}$ using Damped Least Squares (DLS) Inverse Kinematics:

Place your palm flat on your desk and keep it completely still. Notice how you can swing your elbow up, down, left, and right without moving your palm a single millimeter? That is null-space redundancy. Because the Panda has 7 joints but task space requires only 6 constraints (3 position + 3 orientation), there is 1 redundant degree of freedom. Without null-space stabilization, the robot's elbow will drift awkwardly into extreme contortions. The projection matrix $(\mathbf{I}-J^{†}J)$ injects a gentle bias pulling the elbow toward a natural resting posture without disturbing the gripper's path.

The Traps I Walked Into

Continuous-time dynamical control is a principled, lightweight alternative to brute-force discrete sequence models. Reinforcement learning has an uncanny talent for finding every mathematical and physical loophole in whatever environment you hand it. This project found several of its own.

Reward shaping turned out to be way harder to get right than I expected. Small shaping terms kept interacting with each other in ways I didn't predict, and more than once the policy found some shortcut that technically satisfied the reward without actually doing the task. I had an issue where the arm scored well for just getting close to the cube, but it wasn't motivated enough to actually close the gap and grasp it. So the policy just learned to stop short, hovering the gripper a few centimeters above the cube and pinching at empty air over and over, farming the approach bonuses without ever committing to the grasp. Part II will cover how I get that fixed.

As you may have noticed in Part I, the table texture I was using looks a little wonky, kind of broken. Well, in Part II I tried to fix that and swapped in a properly textured table that actually looked table-shaped, and it broke the model. I'll get into how I fixed that and how it ties into training best practices. Part II also covers training at scale on cloud GPUs and modeling the Franka's real grip physics to improve its success rate.