TatuEngine — updates and next steps

Context

TatuEngine has undergone a radical transformation in the last 30 days. From the traversal kernel with hundreds-of-times speedup to the complete Teacher-Student pipeline, what started as “what if RT Cores ran neural networks?” became a functional inference engine with an autopoietic agent, security sandbox, and 128 passing Python tests (not counting the 40+ C++ tests).

This post is an X-ray of the current state — what works, what broke, what we learned, and where the ship is heading.

What happened

SFT Pipeline: Teacher LoRA trained, Student ready

The biggest milestone was the Supervised Fine-Tuning pipeline. The Teacher LoRA (Qwen2.5-3B-Instruct in GGUF) finished training, and 381 examples were distilled into datasets. The Student (BitMamba-1B PyTorch) is ready, differentiable, forward+backward validated on the RTX 3060 — but still not trained.

The Teacher became a 2GB .gguf. The full SFT run is the next step — the Student hasn’t learned anything yet.

Security Sandbox: isolation levels

The sandbox went into production with permission levels: SAFE, PROMPT, RESTRICTED, DENIED. Blocks filesystem writes, dangerous calls, malicious argv. 51/51 tests in the sandbox module alone.

Each tool declares its sandbox level. Tool execution validates before running — if a SAFE tool tries to access a forbidden path, the sandbox blocks it.

Depth Pruning: Mamba2 48 → 32 layers

The pruning was an interesting experiment: take the original Mamba2-1.3B (48 layers) and surgically remove the last 16 abstraction layers. The rationale? Not all depth translates to quality for specific tasks — and 32 layers take up 33% less VRAM.

The pruning preserves embedding, norm_f, lm_head, and the first 32 SSM layers.

C++ Bindings: 124 errors (and why it’s not blocking)

The pybind11 bindings build accumulated 124 errors. Sounds serious, but it’s not — they’re structural warnings from cmake + pybind11 with C++23, not logic errors. The .so compiles and the Python tests pass (128/128). It’s on the todo list but doesn’t block delivery.

What we actually learned

Some lessons worth recording:

1. The AutoTrainer converges too fast. With spacing 3.5 (default), the lenses barely overlap. The system reaches high quality in a few steps and the AutoTrainer sits idle — not because it’s broken, but because the system already solved the problem. I need more complex inputs to test real topological evolution.

2. Adjacent Method won. Testing the 3 η options (-1, 0, +1) by finite differences is faster, more accurate, and free of the false gradients that the original proportional allocation had.

3. The SSM hidden state doesn’t carry history. Δh ≈ 0.000 between consecutive tokens. The real memory is in the SSM state (the kv_cache). The Energy Observer v1 measured the wrong thing — we fixed it.

4. Softplus overflows in float32 past x > 89. expf(89.0) = inf. A 1-line patch with the numerically stable formulation (log1pf(expf(-|x|))) saved 2 days of NaN debugging.

Lessons learned

Looking back, the biggest lesson isn’t technical — it’s strategic:

• The hybrid approach (CPU SSM + GPU matmul) was discovered in 3 days and solved what 2 weeks of OptiX couldn’t. The right problem isn’t “how to use RT Cores”, it’s “how to maximize throughput with available resources”.
• Native ternary models > post-training. Ternarizing a float32 model after training destroys coherence. BitMamba-2 works because it was trained from scratch with ternary weights.
• The SSM is not a transformer. Don’t try to parallelize what is sequential by nature. Accepting that the step is CPU-bound and focusing on accelerating matmuls was the right move.
• Documenting while building pays dividends. Every HISTORY.md entry prevented me from repeating past sprint mistakes. The document became the source of truth the roadmap should have been.

Next steps

The queue is clear:

1. Student SFT full training — run 3 epochs with 381 examples. That’s the current bottleneck.
2. Dataset expansion — batch distillation with varied prompts.
3. Systematic evaluation — Student vs Teacher reasoning benchmark.
4. Student to production — replace the legacy C++ API with the PyTorch Server.
5. Self-improving loop — Teacher → Student → generation → distillation → Teacher.

TatuEngine doesn’t replace Arachne — it’s the inference backend it consumes to deliver infinite context, low latency, and energy efficiency.

---

Timeline: 30 days ago it was an experiment with RT Cores that didn’t work. Today it’s 128 passing Python tests, dozens of C++ tests, an autopoietic agent with 8 subsystems, a security sandbox, and an SFT pipeline ready to run. The project doesn’t accelerate — it accumulates.