Programmatic SEO at 40,000 pages: 4 sites, 4 CMS, one tool

“Programmatic SEO” in agency decks usually means one thing: “we'll generate 500 pages overnight.” On a B2B catalogue that doesn't work — you get 500 near-duplicate thin pages that the search engine swallows and uses to demote the whole site. The real difficulty of programmatic isn't spinning up pages fast; it's keeping tens of thousands of pages across several sites and different engines under control — without drowning.

Here's a real project (no names, under NDA): a manufacturer of industrial heat-exchange equipment, a group of four corporate sites. Four different engines — two Bitrix systems, Bitrix24 and a static generator. Different admins, no shared source of data. The brief: bring it all into one loop and run promotion at industrial scale.

What “scale” means in practice

As of writing, 40,000+ pages across the group sit under one tool: the flagship site grew past 12,000 pages, two others run 14,000–17,000 pages each. 33,800+ pages are indexed in Yandex across the group. This isn't “we created 40,000 pages overnight”: most of the existing catalogue we put in order at the data level, plus added thousands of new pages on shared templates with structured data.

The key number here isn't “how many we generated” but “how many we keep under control.” 40,000 pages across four engines are four potential sources of duplicates, broken links, wrong canonicals and indexation regressions. Without one loop each lives its own life, and the problems surface once traffic has already dropped.

One tool across four different CMS

Each site plugs into the shared tool as a separate module — with its own adapter for its CMS. From there all the work runs in one dashboard: one model-and-brand database (52 brands and 1,768 models in one line), shared structured-data rules, a shared change log and a daily live audit of each site. The client sees the state of all four sites in one window, in real time — not four separate monthly reports.

That's the engineering approach instead of the manual one: edits go through a direct connection to the site, with a preview and the ability to undo any change. At 120 pages you can edit by hand. At 40,000 you can't: one person physically can't hold consistency, and a “universal template” without checks produces exactly those thin duplicates.

The full crawl: finding what you can't see

Once per pass the tool crawls every page of a site in full — 14,000–17,000 pages at a time. Not a sample, not the sitemap, but everything the server returns. At that volume there's always something the eye misses: pages with duplicate titles, orphaned URLs with no internal links, broken links, pages accidentally blocked from indexing, drifting canonicals.

What's found is fixed straight through the site connection, not handed over as “here are 400 tasks, go fix them yourself.” At scale that's critical: working such a list by hand would stretch over months, by which point it would be stale.

Indexation as its own metric

When you have tens of thousands of pages, the main question isn't “how many did we create” but “how many are actually indexed.” Across the group, 33,800+ pages are in Yandex search. And that number has to be defended: on one site, after a server outage, the index started shedding — the search engine excluded 1,740 pages. We restored availability and re-indexing, and excluded pages dropped to 165 — a tenfold cut. At scale that doesn't fix itself: you only see it on a full audit, and you have to fix it fast, before the dip reaches traffic.

Where it breaks

First — you can't do it without a reference. Before any bulk creation there must be one genuinely written reference page you want thousands of: with verified specs, markup, FAQ and internal links. Without it, scale just multiplies the mistake.

Second — infrastructure. Forty thousand pages, a daily audit and full crawls are a real load on the server. Heavy jobs have to be bounded in time and memory, or one bloated process takes everything down. That's the engineering, not the “content”, part of the work — and at scale it's mandatory.

Third — honesty with the numbers. Scale is not an instant traffic spike. Forty thousand pages under management and 33,800 indexed are reach and foundation; sustained growth in positions and traffic shows up later, after re-indexing, and has to be confirmed over time, not promised in month one. So in the case study we show scale, not an invented percentage.