Every meaningful change ships with a version bump and an entry here.
The version next to the DisPindable logo top-left always links here.
— view planned work →
v0.15.238
2026-06-02
CURRENT
- Gap-fill now fetches only the missing pins, not every row. It used to walk every ID from 1 to the ceiling and skip the ones already cataloged — so for a nearly-complete source it churned through ~111k present rows to reach ~9k holes, and the progress bar read “1.8% (2,174 / 120,862)”. It now computes the missing IDs up front and iterates only those; the progress bar reads in real work (“59 / 9,612 gaps”, with the already-had count shown separately). Same fetch path and rate limits, just no busywork.
- Gap-fill is now automated. The 12-hour catalog task, after refreshing ceilings and scanning new pins above the ceiling, now also fills a bounded batch of the holes below it (CATALOG_AUTO_GAP_FILL_BATCH, default 300 per source per cycle). Gaps are recomputed from the DB each cycle, so filled IDs drop out and the next cycle continues with the next holes — it converges on full coverage on its own, no babysitting. It runs the sources one at a time and SKIPS entirely while a forward crawl is running, so it never doubles the request rate against a source the crawler is already hitting. A large initial catch-up is still faster via a manual full gap-fill; this is steady background top-up. Controlled by CATALOG_AUTO_GAP_FILL (on by default).