Keyword clustering by search results: how to group keywords without cannibalization

What keyword clustering is and why you need it

A cluster in SEO: a group of queries for one page

A cluster is a group of queries that people search with one task in mind, and that a single page can answer. In English-language practice this is called keyword clustering: target queries are split into groups, each tied to its own page on the site.

A hypothetical example. “How to choose a camping tent” and “which tent to take backpacking” are most likely one cluster: in both cases the person is choosing a tent. “How to repair a tent yourself” is a different cluster, even though the word is the same: the person’s task is different.

What clustering decides: which queries one page should target, and which need separate pages

A query list on its own doesn’t say how many pages a site needs. Clustering answers that question: which phrasings one page should target, and which ones need a separate page.

You can get this wrong in either direction:

• merging different tasks into one group — the page tries to answer everything at once and doesn’t fully answer anything;
• splitting one task across several groups — you end up with pages competing for the same queries and getting in each other’s way. This is what’s called cannibalization.

Clustering by search results vs. by meaning

By meaning, by words, and by hand — fast but subjective

• By meaning. Queries that look like synonyms or the same topic are grouped together. Fast, but similar phrasings don’t always mean the same task.
• By words. Queries with shared words — including different word forms — are grouped together. The method is purely formal: the word “tent” shows up in queries about choosing one, repairing one, and renting one alike.
• By hand. A specialist reads the list and sorts the queries themselves. This works on small lists, but the result depends on who’s doing the grouping.

The shared weakness of all three methods: the groups reflect a person’s assumptions or rules about words, not which pages search actually shows for the query.

By search results: why shared URLs in the top results point to a shared task

This is how tool documentation describes the method: for each query, the top search results are collected and the page URLs in them are compared. The more overlap there is between the results for two queries, the more likely they are to end up in the same group.

The logic is simple. If the same pages rank for two queries, one page can probably answer both of them. This is a working hypothesis, not a search engine rule: neither Google nor Yandex has an official standard for this kind of grouping — it’s an industry method.

Collecting and comparing the top results for hundreds of queries by hand takes too long, so clustering by search results is done with specialized tools and software.

When each method fits, and how to combine them

Method	What’s compared	Advantages	Limitations
By search results	The page URLs in the top results for each query	Relies on what search actually shows for the query, not a guess; works for large lists	The result depends on the search engine, region and settings; disputed groups still need a manual check
By meaning or by words	The wording of the queries	Fast, no need to collect search results	Subjective; shared words don’t mean a shared task
By hand	A specialist’s judgment	Takes business and product knowledge into account	Slow on large lists; depends on the person

In practice the methods are combined. Clustering by search results provides the base groups, while the meaning of the queries and knowledge of the business are used to review disputed groups and decide on them.

Settings for clustering by search results

The settings determine what groups you get from the same list. Here’s what each parameter controls:

Parameter	What it sets	How it affects the groups
Search engine and region	Which search engine’s results are compared, and for which region they’re collected	If the Yandex and Google results differ, the groups differ too
Top depth	How many of the first result URLs to compare	Sets the range of the results where matches are looked for
Threshold (degree, strength)	How many URLs must overlap	A lower threshold means larger, fewer groups; a higher one means more, smaller groups
Linking method (soft, moderate, hard)	What a query’s results are compared against: the group’s main query, another query pairwise, or every query in the group	The soft method produces larger groups, the hard method produces tighter ones
Excluded domains	Which sites are ignored in the comparison	Matches on sites that rank for almost any query don’t merge unrelated groups together

Search engine, region and top depth

Only the results of the search engines selected in the settings are compared. Choose the engine whose traffic matters to you. If both matter, it’s worth comparing the Yandex and Google groups against each other: the differences will point to disputed queries.

Collect the results under the same conditions your users search in: for the region your site actually serves.

Top depth is how many of the first result URLs take part in the comparison. In tool documentation it’s either fixed or set as a separate option, in which case matches are only looked for among the first N URLs.

The matching-URL threshold: why there’s no universal number

The threshold is the minimum number of URLs that must overlap in the results for queries to end up in one group. Tool documentation calls it by different names, but the idea is the same:

• clustering degree — the number of URLs in the results that must match for queries to land in one group;
• grouping strength — the minimum match threshold between any two phrases.

The lower the threshold, the fuller the groups and the fewer of them there are. The higher the threshold, the more groups there are and the fewer queries in each.

There’s no universal “correct” threshold. How much the results for related queries overlap depends on the topic. A threshold that works well for one set of keywords can merge different tasks or fragment a single one on another. So you choose the threshold on your own data and check it against a sample of groups — more on that in step 2.

Soft, moderate and hard clustering

The linking method determines which queries in a group a new query’s results are compared against. The industry uses the terms soft, moderate and hard for these methods. Here’s how they work:

• Soft. The highest-volume query is taken as the group’s base, and the rest are compared against it. If there are enough matching URLs, the query joins the group. Whether the rest of the queries’ results match each other isn’t checked.
• Moderate. Queries are compared pairwise: if two queries’ results share enough URLs, they’re placed in the same group.
• Hard. The same URLs must appear in the top results of every query in the group at the same time.

These definitions lead to a practical conclusion. Soft clustering produces larger groups, but the queries in them are linked through the main query and may not match each other’s results. Hard clustering produces tighter groups where the results overlap for every query, but you end up with more groups. Neither method is better overall: the choice depends on which risk is worse for you — fragmenting one topic or merging different tasks.

Which domains to exclude from the comparison

Some sites rank in the top results for all sorts of queries. If you include them in the comparison, queries can appear to match just because that site shows up, not because of a shared task.

That’s why matches on certain popular sites can be excluded — for example, auction sites and the search engines’ own services. By the same logic, check whether your topic’s results include large aggregators and directories that show up for almost any query.

How to cluster your keyword set, step by step

Step 1. Prepare a cleaned-up list

You cluster a list that’s already been cleaned up: no irrelevant queries, other brands’ names, or duplicates. Junk in the source list turns into extra groups that you’ll have to sort out by hand anyway.

Step 2. Choose the settings and run a trial pass

Set the parameters:

1. The search engine and region — whichever you want traffic from.
2. Top depth.
3. The linking method — soft, moderate or hard.
4. The matching-URL threshold.
5. The domains to exclude from the comparison.

Run a trial pass and check a sample of groups by hand: a few of the largest, a few small ones, and a few random ones. If the groups mix different tasks, raise the threshold or pick a harder linking method. If one task is fragmented into small groups, lower the threshold or pick a softer method. Change one parameter at a time so you can see how it affects the result.

Step 3. Check the groups by hand

A tool’s output is a draft of groups, not a finished map. Check it against these signs:

Sign of a problem group	What it means	What to do
The group is too large and has queries with different tasks	The threshold is too low, or the soft method linked unrelated queries through the main one	Split the group by hand, or rerun with a higher threshold
Closely related phrasings ended up in different groups	The threshold is too high, or the hard method split one task apart	Compare the results; if the task is shared, merge the groups
A query didn’t land in any group	Its results share few URLs with other queries	Open the results: if it’s the same task as a nearby group, add it there; if it’s a separate task, keep it as its own group; if it’s off-topic, remove it
The group’s top results show different page types, for example articles and catalog pages	The group may combine different tasks or have mixed intent	Recheck the group: split it if the queries have different goals; keep it together if the intent is mixed (see step 4)
The Yandex and Google groups don’t match	The two engines’ results for these queries differ	Decide which engine takes priority; sort out disputed queries by hand using both engines’ results

Step 4. Check the group’s intent

Intent is the task a person is searching with: to find out, to choose, to buy, to find a specific site. Groups with different intents shouldn’t share one page. If the group’s top results show different page types, that’s a signal to recheck the group: it might need splitting, or the queries might have mixed intent and one page is enough. A separate article covers how to read intent from the search results.

Step 5. Record the result: the group’s main phrasing and its variants

For each group, record:

• the main phrasing — the query that best describes the group’s task; it’s convenient to name the group after it;
• variants — the group’s remaining queries;
• a comment — what you adjusted by hand and why.

Separately, save the run’s settings: search engine, region, depth, linking method, threshold and excluded domains. When new queries are added to the keyword set, you can check them with the same settings and compare them against decisions you’ve already made. The group’s variants also come in handy when working on the page — a separate article covers how to use a group’s queries in the copy.

From clusters to a page map

One cluster, one target URL

Every group needs exactly one target URL — existing or planned. If a group is assigned to two URLs, the pages will compete for the same queries. If it’s assigned to none, the demand is left without a page.

The outcome of this step is a map: group → target URL. It shows which pages are missing and which pages are competing for the same group.

Checking against existing pages

Before planning a new page, check whether an existing one already answers the group. There are four possible decisions: a new page, updating an existing one, merging several pages, or concluding that no separate page is needed. The page-decision table in the article on the keyword map shows how to choose between them.

When two clusters can share one page, and when one cluster can’t be split into two

Two clusters, one page. This is acceptable when the smaller cluster is a subtopic of the larger one: its top results show the same page types, and sections of broader material answer it. In that case the smaller cluster becomes a section of the page, not a separate URL.

One cluster, two pages. Don’t do this. If a group’s queries share the same results, two pages built for different phrasings will be answering one task and competing with each other. If it looks like a group needs splitting, recheck the results first: it might actually be two clusters.

More on how we build a page map from keyword research.

How clustering prevents cannibalization

Cannibalization is a situation where several pages on a site compete for the same user task and get in each other’s way. Some of these overlaps can be prevented at the query-grouping stage itself.

Grouping mistakes that cause cannibalization

• A cluster is split across two pages. Queries with shared results were put into different groups, and a page was built for each.
• Different tasks are merged into one group. The page answers them poorly, and later a separate page is built for one of the tasks — it ends up overlapping with the first one.
• A page for every phrasing variant. A different word order, word form or synonym doesn’t make the task a new one.

Pages built for variants of the same phrasing quickly become nearly identical. Here’s what the search engines say about this:

• Google’s spam policies define doorway abuse as follows: “Doorway abuse is when sites or pages are created to rank for specific, similar search queries.” One of the examples given is “creating substantially similar pages that are closer to search results than a clearly defined, browseable hierarchy.”
• In its section “Don’t create content primarily for search engines,” Google includes a self-check question: “Are you creating a lot of material on different topics in hopes that some of your pages might perform well in search results?”
• According to the Yandex Webmaster help, a page that duplicates pages the robot already knows about may not make it into search: it has little chance of being in demand among users.

How to check the map before building pages and after publishing

Before building pages, check the map:

1. No group is assigned to two URLs.
2. No single URL has groups with different intents assigned to it.
3. Every group has a target URL or a recorded decision that no page is needed.

After publishing, check the map against the search engines’ own data:

• In Yandex Webmaster, the “Managing search query groups” tool (Yandex’s counterpart of the Search Console Performance report, broken down by query groups) helps you see which queries a site’s pages appear for in search and which pages appear for those queries.
• In Google Search Console’s Performance report, open the Queries tab, select a query to filter the data by it, and on the Pages tab check which URLs were shown.

If different pages on your site show up for one group’s query, check them against the map: the group might be assigned to the wrong URL, or two pages might be answering the same task. When it’s a case of duplicates of one page under different URLs, the primary version is marked with a redirect or the rel=”canonical” attribute — a separate article covers how to mark the primary version of duplicate URLs.

Clusters and the page map are also the basis for the links between pages: the article on internal linking covers how to link a cluster’s pages together.

Common mistakes

• Treating a tool’s output as final. Groups get carried over into the page map as-is, without a manual check.
• One threshold for every niche. A threshold tuned on one set of keywords gets reused on another without a trial run.
• Clustering an unclean list. Irrelevant queries and duplicates create extra groups.
• Groups without checking the results. Queries get grouped by similar words without ever opening the top results for disputed groups.
• A page for every phrasing variant. Synonyms and word forms of one task get scattered across different URLs.
• A cluster with no assigned URL. The group exists in the table, but it hasn’t been decided which page targets it.

Frequently asked questions

What is keyword clustering, in simple terms?

It’s sorting queries into groups so that one page on the site covers each group. The result is a list of groups, each with a URL assigned to it.

Which is better: clustering by search results or by meaning?

For large lists, it’s more convenient to build the base groups from search results. Grouping by meaning and knowledge of the business are needed to check disputed groups.

Which clustering threshold should I choose?

There’s no universal threshold. Pick a starting value, run a trial pass, and check a sample of groups: raise the threshold if they mix different tasks, lower it if they fragment one.

What should I do with queries that didn’t land in any group?

Open the results for each one. Add a query with the same task as a nearby group to that group; keep a query with a separate task as its own group; remove a query that’s off-topic.

Can two clusters be targeted with one page?

Yes, if the smaller cluster is a subtopic of the larger one and its top results show the same page types. If the page types differ, the clusters need separate pages.

Do I need to re-cluster the keyword set after adding new queries?

Not necessarily from scratch. Check the new queries with the same settings and compare them against the map: a full rerun can reshuffle groups that already have pages assigned to them.