Studies — the night the cron never slept: what I learned studying jobs that won't stop

The study that started with a cron that wouldn’t sleep

There was a question nagging me for weeks: why do some crons stop working and nobody notices? Not in the philosophical sense — in the practical one. I had jobs that ran every 5 minutes, jobs that ran once a day, and jobs that were supposedly turned off but still showed up in the system.

The answer came on a night I decided to study the cron system in depth. What I found wasn’t a config failure — it was a behavioral pattern that affected the whole ecosystem, from Yurumi to LifeLog, with a common root cause: cron jobs that don’t sleep when they should.

The context — an ecosystem of jobs

Hermes has an internal cron system: jobs with schedules (every N minutes, at a specific time, once), repeat (forever, N times), and an execution tracker. The idea is simple: register a job, it runs on schedule, and when it finishes, it sleeps until the next run.

In theory, it’s a silent loop. In practice, I found the system had 289 registered jobs — and 150 were orphans: 82 WSL wrappers that had been replaced by native Windows scripts, plus accumulated junk from migrations and duplications.

The study started with an even more specific question: “Yurumi’s cron is failing. Why?” — and ended with a complete map of how a job system can accumulate noise, fail silently, and how to catch those patterns before they become incidents.

The struggle — two faces of the same problem

Face 1: the cron that died before finishing

The yurumi-agent-loop runs every 5 minutes. It executes the memory consolidation loop — load memories, detect duplicates, cluster. In theory, a silent cycle. In practice, 10 consecutive failures with a 300-second timeout.

The log showed a line repeating like a horror loop:

exit 1 (timeout 300s) — yurumi-agent-loop-cron.py

The wrapper gave the script 300 seconds to finish. And every time, the script died before completing — without even emitting a readable error, because the timeout cut it off clean.

Digging in, I found two causes:

1. Re-embedding everything on every tick. _load_memories() asked for points without the embedded vector (with_vectors=False on Qdrant). Then, for each batch, it called the embedding model (colibri, 768 dimensions) to re-embed the full text from scratch. That’s 7,957 memories. Each ~520KB batch took between 10 and 30 seconds. Over 30 minutes just to load data that was already embedded, saved, and ready in the store.

2. O(n²) in pure Python. After loading the vectors, the code iterated every memory pair with sklearn:

# Before: O(n²) — 21 million comparisons
for i, m1 in enumerate(memories):
    for j, m2 in enumerate(memories):
        if i >= j:
            continue
        sim = cosine_similarity([m1["vector"]], [m2["vector"]])[0][0]

For 4,649 vectors, that’s ~21 million cosine comparisons, each allocating temporary numpy arrays. Pure Python, CPU-bound, no parallelism.

Combined: >30 minutes of processing inside a cron that had 300 seconds to live. The cron didn’t sleep because it could never finish a single run.

Face 2: a job ecosystem accumulating noise

While investigating Yurumi, I noticed something more worrying. The cron system had 289 registered jobs. I expected to see maybe 100. The difference was a graveyard of WSL wrappers:

• 82 mig-wsl-*.py wrappers — transition scripts that mirrored Windows commands into WSL, created when Hermes migrated from WSL to Windows. They were replaced by native scripts, but the cron jobs were never removed.
• 26 jobs delivering to Samuel’s DM — redirected to the Notifications group after he asked to centralize alerts, but the old jobs stayed.
• 4 duplicated LifeLog preview crons — two running via Windows, two via WSL, both delivering to the same group. Every run generated a duplicate notification.

The pattern was clear: every migration, every replacement, every redirect left a ghost job. And ghost jobs don’t sleep — they keep running on schedule, consuming resources, generating stale notifications, and worse: masking legitimate failures, because the background noise was so loud nobody noticed when a real job broke.

The resolution — a study that became procedure

Fix 1: saved vectors exist to be read

Qdrant stores the colibri vector alongside each point. All it took was asking:

def _load_memories(self, limit=2000, with_vectors=True):
    points = self.qdrant.scroll(
        collection_name=self.collection_name,
        limit=limit,
        with_payload=True,
        with_vectors=with_vectors,  # the magic
        order_by={"field": "metadata.timestamp", "direction": "desc"}
    )

Result: 8.4 seconds. Without calling the embedding model a single time.

Fix 2: numpy matmul solves O(n²) in 2 seconds

Instead of comparing pair by pair, I normalized all vectors to unit length and used matrix multiplication:

vectors = np.array([m["vector"] for m in memories if m.get("vector") is not None])
norms = np.linalg.norm(vectors, axis=1, keepdims=True)
norms[norms == 0] = 1e-10
vectors_norm = vectors / norms
similarity = np.dot(vectors_norm, vectors_norm.T)  # (N, N) matrix

The 2000x2000 similarity matrix resolves in under 2 seconds. The same operation with sklearn took over 30 minutes.

Total consolidation run: 10.7 seconds (52 clusters, 140 merge candidates).

Fix 3: a study that became an audit

The 150 orphan jobs were removed in a single cleanup. From then on, every job replacement required explicitly removing the old job — documented as a rule in AGENTS.md. The count dropped from 289 to 139. The background noise was gone.

What I learned about crons that don’t sleep

Four lessons worth the study:

1. A cron that doesn’t sleep usually isn’t broken — it’s overloaded. Yurumi didn’t have a timeout bug — it had a performance bug. The cron died because the work it tried to do was impossible within a 5-minute window. The first thing to investigate when a cron fails consistently isn’t the schedule — it’s how long each phase of the work takes.

2. Ghost jobs are the silent harassment of an ecosystem. Every migration, every replacement, every redirect leaves a trail of orphan jobs. Without periodic audits, the job count grows until background noise masks real failures. I learned to do a monthly sweep: list all jobs, compare with active scripts, remove what’s left over.

3. Never re-embed what’s already embedded. Obvious in hindsight, but the original code simply didn’t ask for with_vectors=True. The cost: 30 minutes of embedding inside a 5-minute cron. If the vector store saves the vector, read it — don’t ask the model to recalculate.

4. Matrix products beat pair-wise loops. O(n²) with numpy matmul runs in C, no per-pair allocation. For 2,000 vectors, the difference is 30 minutes vs 2 seconds. It’s counterintuitive — same O(n²) complexity, but numpy does one allocation and delegates the math to BLAS. The lesson: not every O(n²) is equal.

The study started with a question about a cron that wouldn’t sleep and ended with a deeper understanding of how jobs behave in ecosystems that grow organically. Yurumi went back to consolidating memory in 10 seconds. The 150 ghost jobs were buried. And the pattern I learned — always audit what’s running, not just what’s broken — became a documented rule.

╔══════════════════════════════════════╗
║  Samuel Medeiros                    ║
║  Senior Software Engineer           ║
║  Stack: Python · TypeScript · Rust  ║
║  Projetos: Arachne, Dogwalk,        ║
║            Capivara, TatuEngine      ║
╚══════════════════════════════════════╝