The story of TatuEngine — from dream RT Cores to an autopoietic agent

The engine that started with an absurd question

“Modern GPUs have RT Cores idle 99% of the time. What if we used them to process neural networks?”

That provocation gave birth to TatuEngine in June 2026. The vision was mesmerizing: each weight becomes a refracting lens inside a BVH, the input token is a ray fired at the geometry, and inference is light passing through the system — the answer emerges from coherent wave interference. Learning? Sculpting the path the light travels.

Thirty-odd days later, the project has 302 commits, a BitMamba-2 1B model running on GPU orders of magnitude faster than CPU, 400+ tests and an autopoietic agent. But the road here started with the dream dying on day one.

Context — the OptiX dream

The planned stack was ambitious:

• OptiX 7 for ray tracing acceleration
• Mamba 2.8B as the base model
• GGUF as the model format
• MCP as the API protocol

Sprint 1 built everything around that vision: src/core with lock-free pipeline and triple buffering, src/rt with OptiX 9.1 headers compiling PTX, src/mamba with a CUDA SSM kernel, src/field_theory with the Phase Field prototype. The docs from that era — PLANO_MESTRE.md and CARVEKNOWLEDGE.md — both reflected a vision that would never work in the real environment.

The struggle — the dream that died on WSL2

The blow: RT Cores don’t exist on WSL2

WSL2 has no RT Core support. The libnvoptix.so.1 inside WSL is a 14KB DXCore proxy that doesn’t export OptiX’s function table:

optixInit() → OPTIX_ERROR_ENTRY_SYMBOL_NOT_FOUND

Two weeks trying to work around it. What saved us: CUDA works natively in WSL (direct NVIDIA ioctl). RT Cores don’t — but CUDA Cores do. That was the first lesson: don’t force a technology where it doesn’t fit.

The second wall: 27 GB of lenses

Converting the 1B-parameter network to BVH as individual lenses would cost 27 GB — impossible for 12 GB of VRAM. The solution was the hybrid pipeline: the heavy matmuls (in_proj, out_proj, lm_head) run in CUDA reading the original tensors, while the SSM state (conv1d, RMSNorm, step) stays on CPU:

CPU: embedding → RMSNorm → conv1d+SiLU → SSM step → residual
GPU (bridge_matmul): in_proj → out_proj → lm_head

And here came another discovery: the Mamba SSM is inherently sequential — h(t) = f(h(t-1), x(t)). Unlike transformers (parallelizable KV cache), only the matmuls can be parallelized; the step is CPU-bound. The bottleneck became CPU↔GPU communication between tokens.

The NaN that almost became a legend

Softplus with float32 overflows: expf(89.0) = inf. One line saved two days of debugging: the numerically stable formulation (log1pf(expf(-abs(x)))).

The milestone: 50257/50257 bit-identical logits

The validation that proved the hybrid worked: 50257 identical logits between bridge_matmul and the CPU ground truth, bit for bit.

Block Codec: radical compression

The discovery that changed the game: instead of 1 lens per weight, compact ternary blocks — 245 MB total vs 27.26 GB float32 — over 100× compression, with ULP tolerance.

GPU Block-Traversal

The GPU traversal kernel processes entire blocks in parallel, with warp reduction and a shared-memory accumulator — without atomics:

Pipeline	CPU	GPU	Speedup
1 token inference	10.552 ms	41.9 µs	252×
16-ray traversal	74.2 ms	0.52 ms	146×
Bake 100K cells	~70 ms	4.8 ms	15×

Resolution — the autopoietic agent

With inference solved, TatuEngine became an agent. The GenerationContext gathers the subsystems that let the model self-evaluate and use tools: fixed-attractor detection, associative memory in the SSM latent space, tool execution via sandbox, recursive reasoning with checkpoint/restore, safety invariance and a semantic goal supervisor.

And the Hybrid Sandbox to protect file tools — path validation, size/type enforcement, isolation levels. Tests passing: 51 passed.

Metrics

Metric	June (birth)	Today
Commits	1	302
Tests	0	400+ C++ (30 suites) + 51 sandbox
GPU speedup	(OptiX didn’t work)	252× block-traversal
Compression	27.26 GB (float32)	245 MB (block codec, 114×)
Pipeline	Conceptual	Hybrid CPU/GPU bit-exact
Agent	0 subsystems	8 subsystems
Model	Mamba 2.8B (planned)	BitMamba-2 1B (functional)
Sandbox	didn’t exist	3 levels, 51/51 tests

Lessons

1. Don’t force a technology where it doesn’t fit — OptiX on WSL cost ~2 weeks; the hybrid approach was discovered in 3 days and solved it.
2. Native ternary models vs post-training — ternarizing a float32 after training destroys coherence. BitMamba-2 works because it was born with ternary weights.
3. The SSM is not a transformer — don’t try to parallelize what is sequential by nature. The real gain is in accelerating matmuls and accepting the step is CPU-bound.
4. AutoTrainer converges too fast — the system reaches 99.5% quality in 14 steps and the AutoTrainer sits idle, not because it’s broken, but because the system is already solved.