Programmatic SEO: The Playbook, and the Indexing Problem Nobody Plans For

By on

A schematic flowchart showing the difference between traditional SEO (manual, slow) and programmatic SEO (automated, scalable).

Programmatic SEO is building many pages from one template and one dataset, so that a pattern like "{city} plumbers" or "{app A} to {app B} integration" gets a page for every combination that matters.

Most guides to it stop at the moment the pages publish. That is the easy half. The hard half starts afterwards, when Google indexes a fraction of them and you have to work out which fraction and why.

This is the full playbook: how to qualify a pattern before you build anything, how to launch in phases with criteria for stopping, and what to do about indexing, which is where these programs actually die.

If what you are looking for is which software to use, that is a different page: we compared them in programmatic SEO tools and how to pick one. This page is the method.

The qualifying test, before you build anything

Three questions. If any answer is no, a programmatic build will produce pages nobody wants and Google will not keep.

Is there a real pattern in how people search? Not a pattern you can imagine, one you can see. Take your template phrase, substitute ten different values, and check that all ten are searched. If only two of the ten have any volume, you have two pages, not a program.

Do you have something per page that is not the template? This is the one that decides whether the program survives. If every page differs only in the name substituted into the sentences, the pages are duplicates with a find and replace applied. Real differentiators are prices, counts, availability, reviews, photographs, specifications, or your own data.

Would a person who landed on one of these pages get their answer? Read one generated page as though you searched for it. If the answer to the query is not on the page, generating a thousand of them makes the problem a thousand times bigger.

The patterns that work, and roughly how many pages each yields

PatternExampleTypical page countWhat makes each page unique
Location"{service} in {city}"50 to 5,000Local providers, prices, travel times, regulations
Integration"{tool A} to {tool B}"500 to 50,000The actual field mapping, setup steps, limits
Comparison"{product A} vs {product B}"100 to 2,000Feature and price differences, pulled from data
Attribute"{material} {product}"100 to 3,000Specifications, stock, images
Use case"{tool} for {role}"50 to 500The workflow that role actually runs
Data lookup"{term} converted to {unit}"100 to 10,000The calculation itself, which is the value

The two on the ends behave differently from the rest. Location pages are the most competitive because everyone builds them. Data lookup pages are the easiest to justify because the tool on the page is the reason the page exists.

Step 1: find the modifier set, not the keyword

You are not looking for a keyword. You are looking for the list of values that goes in the variable slot, and the evidence that the list is searched.

Start with the head term, then pull every long-tail variant that contains it and look at what varies. That varying part is your modifier set. Cluster them: if "plumbers in leeds" and "leeds plumber" return the same results, they are one page, not two.

Then cut the list. Every modifier with no measurable demand is a page that will sit unindexed and dilute the ones that matter. It is better to launch 200 pages that are all wanted than 4,000 where 3,600 are filler. This is the most common mistake in the whole discipline and it comes from the fact that generating 4,000 costs the same as generating 200.

Step 2: the data, and where the unique part comes from

The dataset is the product. Everything downstream is plumbing.

Sources, in rough order of how defensible they are: your own operational data, licensed or purchased datasets, public APIs and open data, and content you assemble by hand for the top slice. The first is the only one your competitors cannot also buy.

Build it in a spreadsheet or a database with one row per page and one column per field the template needs. Include the fields you are not sure you will use, because adding a column later means regenerating everything.

Google Sheets

Two rules that save a rebuild. Every row needs a stable unique identifier, which becomes the URL slug, and every row needs a completeness score, so you can publish only the rows above a threshold. That second one is how you avoid publishing 900 pages with three empty sections.

Step 3: the template, built once

A programmatic template needs the same things as any page, with one addition: it has to degrade gracefully when a field is empty.

Include a title pattern, a meta description pattern, a first paragraph that answers the query using the row's own values, the data itself in a table or list, a section of genuinely varying content, an internal links block, and schema markup appropriate to the page type.

The degradation part matters more than it sounds. Write conditional blocks so that a missing field removes its section rather than rendering "Price: null" or an empty heading. Every large programmatic site has shipped that bug at least once.

Test the template against your five best rows and your five worst. The worst rows are the ones that tell you whether the completeness threshold is set correctly.

Step 4: generate and publish

This is the step people expect to be hard, and it is the most solved part of the process. A no-code sync, a bulk importer, or a static site generator will all move rows into pages.

What to get right here is less about the tool and more about three settings. Publish in batches rather than all at once. Keep the URL structure flat and readable, because you will be reading these URLs in reports for years. And generate the sitemap from the same source as the pages, so the two never disagree.

WP All Import

Where Distribb fits, and where it does not. Distribb writes and publishes individual researched articles on a schedule and handles the internal linking between them. It is not a template multiplier and will not turn a 5,000 row spreadsheet into 5,000 pages, so for a true programmatic build you want one of the tools on the comparison page above. Where it does help is the layer most programmatic programs are missing, which is the supporting editorial content that gives the generated pages something to be linked from.

Step 5: internal linking, which decides whether this works

Generated pages are orphans by default. They exist in the sitemap and nothing on the site points at them, which is the fastest way to have Google ignore them.

Every generated page needs three kinds of link pointing in. A hub page that lists its category, sibling links from related generated pages, and at least one link from an editorial page that was written by a person. That last one is the strongest signal and the one almost nobody builds.

Structure the hierarchy as category, then subcategory, then page, and use breadcrumbs so the structure is visible to crawlers as well as readers. There is a good breakdown of the enterprise version of this problem in these advanced internal linking strategies, and the mechanics of doing it automatically are in how to automate internal linking for blog posts.

Audit for orphans monthly. On a programmatic site the orphan count grows every time the dataset does.

The part nobody plans for: indexing

Here is the number that changes how you plan. On a large programmatic launch it is normal for Google to index somewhere between a fifth and two thirds of the pages you publish, and the rest sit in "Discovered, currently not indexed" or "Crawled, currently not indexed" indefinitely.

That is not a bug you can fix after the fact by resubmitting the sitemap. It is Google deciding the page is not worth the storage. What moves the number:

  • Uniqueness per page. The single strongest factor. Pages whose only variation is a substituted noun are the ones that get dropped.
  • Internal links. Pages with links pointing at them get crawled; orphans get discovered and skipped.
  • Publishing rate. Dropping 10,000 URLs into a sitemap at once looks different from adding 500 a week, and the second gets treated better.
  • Site-level authority. The same page on a site with 3,000 referring domains gets indexed where it would not on a site with 20.

Plan the program around an indexing rate rather than a page count. If you need 500 indexed pages and you expect a third to make it, that is the real scope of the build, and it is a very different conversation from "we will generate 5,000".

Google Search Console

Measure it weekly in Search Console with a filter on the URL prefix. Indexed page count against published page count is the only health metric this kind of program has.

Launch in phases, with the criteria written down first

Decide the criteria before you see the results, because after you see them you will find reasons to continue.

Phase 1, 50 pages, weeks 1 to 6. Publish the 50 best rows. Success is 60 percent or more indexed and any impressions at all by week six.

Phase 2, 250 pages, weeks 6 to 14. Only if phase 1 passed. Success is the indexing rate holding and the first page reaching page two of the results.

Phase 3, the rest, from week 14. Only if phase 2 held. Add in batches, and stop adding whenever the indexing rate drops below the phase 1 number.

The kill criteria. If phase 1 indexes under 30 percent, the problem is the pages, not the volume, and generating more of them will not fix it. Go back to the uniqueness question. If phase 2 indexes fine but produces no impressions after two months, the modifier set has no demand and the research step was wrong.

This structure exists because the failure mode of programmatic SEO is not a bad page, it is 4,000 bad pages published before anyone checked.

What the well-known examples actually teach

Zapier's integration directory is the case everyone cites, and the lesson usually gets stated backwards. The pages work because each one contains the actual field mapping between two specific tools, which is information a person needs and cannot get elsewhere. The template is the least interesting part of it.

NerdWallet is the more useful case because it includes a setback. After a core update cut its traffic sharply, the recovery involved pruning pages aggressively rather than adding more, and the traffic went up after the deletions. The detail is worth reading in Ahrefs' write-up of the NerdWallet case. The non-programmatic version of the same recovery, meaning growth from pages written one at a time, is covered in our guide to increasing website traffic organically.

The pattern across both: programmatic pages that carry real information survive core updates, and pruning the ones that do not carry any is a growth lever rather than a loss. Plan to delete a share of what you publish. We collected more of these in programmatic SEO examples and how to replicate them.

Two changes are worth building for now rather than retrofitting.

Answer engines quote pages that state a specific fact in a self-contained sentence. On a programmatic page that means the first paragraph should contain the actual value, not a promise that the value is below. "The average cost of a new boiler in Leeds is between X and Y" gets quoted. "Looking for boiler prices in Leeds? You are in the right place" does not.

Structured data matters more than it did, because it makes the fact machine-readable rather than inferred. Generate it from the same row that generates the page, so it can never drift out of sync with what the page says.

The pages most exposed to AI search are the ones whose entire value was aggregating something a model can now assemble itself. If your differentiator is your own data, this is a smaller problem than it looks.

The stack, briefly

You need five functions covered: keyword and modifier research, a place to hold the dataset, a template layer, something that pushes rows into the CMS, and monitoring. Most teams assemble these from a spreadsheet or database, their existing CMS, an import plugin or sync tool, and Search Console.

The full comparison with prices is on the tools page, so it is not repeated here. The short version is that the tool choice is the least consequential decision in this document, and the dataset is the most.

Questions people ask

Is programmatic SEO against Google's guidelines? No. Generating pages at scale is fine. Generating pages with no value is what the spam policies address, and the distinction is whether each page tells the reader something the others do not.

How many pages should I start with? Fifty. Enough to measure an indexing rate, small enough that being wrong costs a fortnight.

How long before it works? Expect three to six months before the pattern is clearly working or clearly not. Indexing signal arrives in weeks, ranking signal in months.

Can I use AI to write the content on the pages? For the assembled parts, yes, and for the differentiating part, no. If a language model can write the unique section of your page without your dataset, then your dataset was not the differentiator and the program has no defence.

What if only a third of my pages get indexed? That is within the normal range. Look at which third, because the pattern in what got indexed is the specification for what to do with the rest.

Where to start

Take one pattern, list the modifiers, and check that ten of them have real search demand. Then find the field that will differ on every page and confirm you can actually get it for every row. That is the whole qualification step, and it takes an afternoon.

If the honest answer is that you do not have a dataset and what you need is a steady stream of researched articles rather than generated pages, that is a different machine, and Distribb is built for that one.