scraping

8 posts

Multi-engine pipeline in Arachne — 4 fallback layers

The architecture that actually solves web data extraction: Trafilatura, Crawl4AI SDK, Docker Sidecar, and Camoufox — how each layer works, the fallback criteria, and what I learned along the way.

arachne pipeline multi-engine

Continue reading →

Pipeline multi-engine no Arachne — 4 camadas de fallback

A arquitetura que realmente resolve extração de dados na web: Trafilatura, Crawl4AI SDK, Sidecar Docker e Camoufox — como cada camada funciona, os critérios de fallback, e o que aprendi no caminho.

arachne pipeline multi-engine

Continue reading →

First tests with Crawl4AI — Sidecar and Docker

Experimenting with Crawl4AI in Arachne: Docker Sidecar, Python SDK, comparison with Trafilatura, and the decision to use automatic fallback between engines.

arachne crawl4ai docker

Continue reading →

Primeiros testes com Crawl4AI — Sidecar e Docker

Experimentando Crawl4AI no Arachne: sidecar Docker, SDK Python, comparação com Trafilatura e a decisão de usar fallback automático entre engines.

arachne crawl4ai docker

Continue reading →

O Arachne nasceu de uma frustração — scrapers quebrados e dados perdidos

Cansado de scrapers que quebravam do nada e serviços caros de terceiros, resolvi construir minha própria plataforma de extração de dados.

arachne scraping crawl4ai

Continue reading →

Arachne Was Born from Frustration — Broken Scrapers and Lost Data

Tired of scrapers that broke out of nowhere and expensive third-party services, I decided to build my own data extraction platform.

arachne scraping crawl4ai

Continue reading →

Arachne — o scraper que virou plataforma

A história real de como um scraper Python de 200 linhas virou uma plataforma com 740+ arquivos, RAG, pipelines visuais e 36 ferramentas ReAct.

arachne scraping crawl4ai

Continue reading →

Arachne — the Scraper That Became a Platform

The real story of how a 200-line Python scraper became a platform with 740+ files, RAG, visual pipelines and 36 ReAct tools.

arachne scraping crawl4ai

Continue reading →