4 posts
An LLM call ends in 200 OK with empty content and the pipeline moves on like nothing happened. Diagnosis: reasoning models burn the token budget thinking and nothing is left for the answer. The fixes: a token ceiling with real margin, and a parser that tolerates a data: [DONE] glued to the JSON.
Chamada LLM termina em 200 OK com content vazio, a fila segue e ninguém reclama. Diagnóstico: reasoning models gastam o budget de tokens pensando e sobra zero pra resposta. Os consertos: teto de tokens com margem e parser que aceita SSE sujo.
After learning to infer (252× GPU speedup), TatuEngine went for the Master-Apprentice cycle: a Qwen2.5-3B teacher with LoRA distilling structured reasoning ([THOUGHT]/[ANSWER]) into the BitMamba-2 1B student. Six versions in one day (v0.17→v0.22.2), 381 examples, and the convergence that wouldn't come until the system message fix.
Depois de aprender a inferir (GPU 252×), o TatuEngine partiu pro ciclo Mestre-Aprendiz: um teacher Qwen2.5-3B com LoRA destilando raciocínio estruturado ([THOUGHT]/[ANSWER]) pro student BitMamba-2 1B. Seis versões num dia (v0.17→v0.22.2), 381 exemplos e a convergência que não vinha até o fix do system message.