Competitor keyword analysis: what to look at and how to avoid copying someone else’s structure

Why look at competitor keywords, and what not to expect from it

What the analysis gives you:

• wordings that weren’t in your list: synonyms, questions, industry terms;
• demand groups you have neither a page nor a section for;
• the page types that rank for the topic: articles, catalogs, service pages;
• a sense of how demand is split between a site’s sections.

What it doesn’t give you:

• a list of the queries the competitor chose for their pages. You only see what their pages rank for, and only within the limits of your data source;
• the competitor’s traffic: position and search volume are not the number of visitors;
• a ready-made structure for your site.

The queries a competitor buys PPC (paid search) advertising for are a different task with different data. This article is only about organic search.

How to find competitors in search using your own queries

A search competitor is whoever ranks for your query groups

Search results (SERP) are the page of results for a query; the top is its first positions. A search competitor is a site whose pages rank in the top for your query groups. That’s why the starting point of the analysis is your groups, not a list of companies you share customers with. A business rival may barely show up in search, while an unfamiliar site may rank in the top for every one of your topics.

How to build a list of competitors from the search results

1. Take several of your query groups of different types — for example, a service page group and a blog article group.
2. Open the search results for two or three queries from each group, in the region where the business operates.
3. Write down the pages from the top and the type of each one: service page, catalog, article, aggregator.
4. Mark the sites that repeat across several groups. These are candidates for analysis.
5. Don’t discard aggregators, directories and marketplaces — label them by type instead. They show what format the search results expect, but their queries are usually broader than your offering.

Region matters here. Google states that result rankings take into account the user’s location, language, device type and other factors: for example, the results for the query “bicycle repair” will differ in Paris and in Hong Kong. Yandex’s help says that queries can be region-dependent or region-independent, and for region-dependent ones, preference goes to sites tied to the search region. A list of competitors built from another city’s search results may not fit your situation.

The whole site or comparable pages

For a large site, domain-wide data mostly reflects its range — products, services and regions you may not have. It’s more useful to look at comparable pages — ones with the same job as yours: compare a service section with a service section, an article with an article. That way you see which queries the competitor’s site ranks for specifically in the part that overlaps with yours.

If you’re not sure the pages solve the same task, check the page type ranking in the top. There’s a separate article on how to tell from the search results which page a query needs.

How to find out which keywords a competitor’s site ranks for

Manually: search results, titles and sections

Without any tools, you can already see quite a lot:

• which of your queries the competitor’s page ranks in the top for;
• how its search results title (Title tag) and main on-page heading (H1) are worded;
• what subheadings and questions appear on the page;
• what sections exist in the site’s menu.

There’s one limitation, but it’s a real one: this way you only see the queries you checked yourself. Manual checking gives you almost none of the wordings you didn’t think of.

SEO analytics services: what they show

SEO analytics services are a class of tools that check the search results for a large set of queries and store the results in their own database. For a domain or a page URL, such a service usually shows:

• the queries from its database for which the page appeared in the results;
• the page’s position as of the snapshot date — the moment the service checked the search results;
• an estimate of each query’s search volume;
• the URL of the page that shows up for the query.

This way you can see competitor keywords as a list, not one query at a time. In practice, “scraping competitor keywords” usually means exporting such a list from a service, not crawling the search results with your own script.

For the same site, different services give different lists: they have different databases, regions and snapshot dates. So along with the queries, record where and when they were taken from. The article on search visibility explains why services’ estimates for the same site differ.

Why data about your own site is more accurate than data about someone else’s

Search Console and Yandex Webmaster (Yandex’s counterpart of Google Search Console) show which queries a site’s pages are visible for in search. But only for a site you have access to the data of. Search Console’s help states that a site’s owner has access to confidential information about that site’s Google Search results. Yandex explains its rights check this way: data about page indexing can contain confidential information, so it needs to confirm that a user has the right to manage the site or view data about it.

The conclusion follows: everything you know about a competitor’s queries is an outside estimate, while you get data about your own site straight from the search engine itself. The keyword research article explains how to see which queries your own page shows up for. If the site isn’t added to Yandex’s panel yet, start by adding it to Yandex Webmaster and confirming your rights to it.

Yandex Webmaster also has a “Query selection and market analysis” tool (in beta status as of the check date). Per its help page, the market-analysis part helps find pages that are already popular for search queries and assess whether a niche suits you. It’s another way to see who ranks in search for your topics. But for a query by a group of URLs, the help page asks you to list only the URLs of sites you’ve confirmed.

What data to look at for each query

Field	What it means	How to use it in your decision
Query	The wording the competitor’s page is visible for	Compare with your groups: do you have this task
Competitor’s page	The URL that’s visible for the query	Open it and check whether its task matches your page’s task
Page type	Service, catalog, article, aggregator	If the type doesn’t match yours, don’t treat the query as a primary one without checking the search results
Position and snapshot date	Where the page stood in the results at the time of the check	The higher the position, the stronger the signal that the query is a target one for the page; recheck an old snapshot
Search volume estimate	How many times the phrase was searched, per the data source	Use it to set priorities, not to forecast traffic
Region and search engine	Which search results the data was taken from	Use data for the region and search engine you actually operate in
Your page with the same task	Whether your site has a URL that answers the query	Determines which bucket it falls into when you compare against your own coverage

Limitations: date, region, database, search volume

• Date. Positions change, so the data is only valid as of the snapshot date.
• Region and engine. Data for one region and one search engine doesn’t carry over to others: results depend on region and device.
• Database. The service only sees queries from its own database. If a query isn’t there, that doesn’t mean nobody searches for it.
• Search volume. Yandex Wordstat (Yandex’s own keyword search-volume tool) shows statistics for search queries to Yandex. Its “number of queries” is the total count of queries containing the given phrase, for the specified region. It’s a demand estimate, not a forecast of visits to a page. Nested-phrase volumes aren’t additive: a longer phrase is already included in the count for the shorter one. SEO analytics services’ numbers are their own estimates, and they may not match Wordstat.
• Low positions. If a competitor’s page is visible for a query far down the results, that’s a weak signal that the query is a target one for it. The lower the position, the weaker the signal.

A separate article explains why positions and search volume aren’t the same as results.

How to filter out queries that aren’t yours

The general rules for cleaning up a query list — irrelevant queries, junk, duplicates — are described in the keyword map article. Here is only what applies to a list pulled from someone else’s site.

This example is hypothetical, for illustration. A company installs air conditioners in one city; its site has a service page and a blog. Two sites from the search results for its groups were taken for the analysis: “site A” is a service company from the same city, “site B” is an online store for climate equipment that operates nationwide.

Signal	Hypothetical example	What to do
Competitor’s brand and product names	“‹site A’s brand› installation”	Remove. Exception: brands you actually work with
Range or services you don’t offer	“AC refill”	Remove, or move to a “to decide” list if the service is genuinely planned
Regions and cities you don’t serve	“AC installation in ‹another city›”	Remove
A different meaning of the word	“hair conditioner”	Remove
A query for a different page type	“buy an air conditioner”: site B has a catalog, you have a service page	Don’t take it as a primary query; check the search results
The competitor’s page is visible by chance and low down	a query for which site B’s article sits far down the results	Don’t treat it as a demand signal for the topic
The query is already assigned to another of your pages	“how often to clean an air conditioner” — your article already answers it	Don’t take it for a new page

A query that passes the filter still isn’t signed off. Next you need to check demand and the search results for each new candidate — the same as for any other query.

Comparing against your own coverage: what you have, what you don’t

Coverage is the query groups that pages on your site already answer. Comparing someone else’s queries against your own coverage is sometimes called a gap analysis — finding the gaps. Every query that passes the filter falls into one of three buckets:

• Your page already covers the query. Change nothing, or strengthen the section if the competitor’s page answers it more fully.
• The query belongs to an existing page’s task, but the page doesn’t answer it. A candidate for that page’s supporting queries: add a section, a question, or a clarification.
• You have no page with that task. A candidate for a new group, but only after the check described below.

Before putting a query in the second or third bucket, check whether another page of yours already ranks for it. If it does, the query isn’t new: you need to decide which of your pages should answer it.

How to avoid copying someone else’s structure

What a competitor’s structure shows, and what it doesn’t

A competitor’s structure shows:

• which demand groups it split off into separate pages;
• which page types rank for the topic;
• what level of the site each group sits at.

And it doesn’t show:

• why it was done that way;
• whether it gets results;
• whether that layout fits your range, region and demand.

A competitor’s sections reflect its offering. If you have three services and it has thirty, its menu doesn’t fit you, even if the site ranks first. Read someone else’s structure as a list of questions to ask yourself: do we have this demand group, and does it need a separate page. Your own data answers those questions, not someone else’s menu. A separate article covers how to break your own demand down into sections and URLs.

Why not to create a page for every one of a competitor’s queries

A competitor may have many nearly identical pages for variants of the same wording. There’s no need to copy that. Before creating similar pages, it’s worth knowing what Google’s documentation says about this.

Google’s spam policies define this as doorway abuse: “Doorway abuse is when sites or pages are created to rank for specific, similar search queries.” One example given is “creating substantially similar pages that are closer to search results than a clearly defined, browseable hierarchy.”

Google’s guidance on helpful content includes self-check questions about using other people’s material. In the “Content and quality” section: “If the content draws on other sources, does it avoid simply copying or rewriting those sources, and instead provide substantial additional value and originality?” In the “Avoid creating search-engine-first content” section: “Are you mainly summarizing what others have to say without adding much value?”

For competitor analysis, this means: someone else’s query is a reason to check whether you can answer that task in your own way, not a reason to create a page. If a new page would just repeat someone else’s and add nothing, it’s better not to create it.

A competitor’s mistakes move in along with its structure

A competitor’s structure isn’t necessarily correct. Other sites have things like:

• two pages that answer the same query and get in each other’s way — this is called keyword cannibalization;
• sections created just for one query, with no child pages;
• outdated pages that are still visible in the results.

Copy the sections, and these mistakes will show up on your site too. A high-ranking site doesn’t prove that every one of its pages is needed.

How to move the queries you found into your own list

For each query, there’s one of three decisions:

• Add to an existing page — for queries from the first and second buckets.
• Create a new group and, possibly, a page — only if the group has its own task, demand is confirmed, and pages of a suitable type rank in the results. A group of queries with one task that one page answers is called a cluster; the article on clustering covers how to group new queries by task.
• Don’t take it — someone else’s topic, someone else’s brand, a different page type.

It’s convenient to keep the result in a single table. Below it’s filled in for the hypothetical example; the values are for illustration. Discarded queries stay in the table along with the reason: next time you won’t have to check them again.

Query	Source	Passed the filter	Your page with this task	Decision	Page	Reason
split-system installation	site A, ‹snapshot date›	yes	yes	add	service page	synonym of the main service
air conditioner installation in winter	site A, ‹snapshot date›	yes	yes, but doesn’t answer it	add	service page	needs a section on cold-weather installation
how to choose AC capacity for a room	site B, ‹snapshot date›	yes	no	new group	blog article — after checking demand and the results	its own task, articles rank in the results
AC refill	site A, ‹snapshot date›	no	no	don’t take	—	no such service; query on the “to decide” list
‹site A’s brand› installation	site A, ‹snapshot date›	no	—	don’t take	—	someone else’s brand
buy an air conditioner	site B, ‹snapshot date›	no	—	don’t take	—	no catalog; stores rank in the results
AC installation in ‹another city›	site B, ‹snapshot date›	no	—	don’t take	—	you don’t operate in this city

How to work through competitor keywords — step by step:

1. Pick several of your query groups of different types.
2. Open the search results for them in the region where the business operates.
3. Build a list of competitors — sites that repeat across several groups — and label their page type.
4. Pull the queries for comparable pages; record the source, region and snapshot date.
5. Remove other brands, ranges, regions, different word meanings, and queries for a different page type.
6. Compare what’s left against your own coverage and sort it into the three buckets.
7. Make a decision for each query and record the reason.
8. Check new candidates for demand and the search results before planning a page.

When there are many new groups and they change a site’s sections, this becomes a keyword set and structure task, not just a list. In that case it’s worth matching competitor queries against your entire keyword set and structure.

Frequently asked questions

Can you find out every keyword a competitor’s site ranks for?

No. You can see the queries its pages show up for in the results, and only within the limits of your data source: what you checked manually, or a service’s database. Which queries the competitor chose as targets isn’t visible from the outside.

How do you find out a competitor’s keyword set?

In full — there’s no way: the keyword set as a working document exists only for the site’s owner. It can be reconstructed approximately from the search results and SEO analytics services’ data. The methods and their limits are covered in the “How to find out which keywords a competitor’s site ranks for” and “Limitations” sections.

How many competitors should you analyze?

There’s no universal number. Take sites that repeat in the results across several of your groups, and compare comparable pages. If the next site adds almost no new queries, you don’t need to expand the list.

How often should you repeat competitor query analysis?

There’s no universal interval. Reasons to redo it: the results for your main groups have changed, a new site has appeared in the top, you’re launching a new section or service.

Is this data suitable for PPC advertising?

For advertising you look at different data: competitors’ ads and the queries they show up for. This article is about organic search — the queries pages are visible for in the regular results.