Noindex Tag

← Back to glossary · Category: Technical

What the Noindex Tag Means, in Plain Language

A noindex tag is an instruction, placed on a page, that tells search engines not to include that page in their search results. In its most common form it is a meta tag in the head of the page's HTML, written as a robots meta tag with the value noindex, which search engines read when they crawl the page and then act on by dropping the page from their index. The same instruction can also be delivered as an HTTP response header, called the X-Robots-Tag, which is the right form for files that have no HTML head to carry a meta tag, such as PDFs, images, and other non-HTML resources. Either way, the meaning is the same: this page may be visited directly by anyone with the link, and it may be linked to from other pages, but it should not appear as a search result.

The distinction between crawling and indexing is the whole point of the tag, and it is worth stating slowly. Crawling is the act of fetching a page and reading it. Indexing is the act of including a page in the collection that search results are drawn from. A noindex page is still crawled, because the search engine must fetch it in order to read the instruction, and it is still visitable, because the instruction says nothing about who may open the URL. What changes is only its presence in search results: once the instruction is read and acted on, Google will drop the page entirely from its search results, regardless of whether other sites link to it. That is a stronger and cleaner removal than a crawl block can ever give, because a crawl block leaves the search engine guessing from links alone.

Pages that commonly carry a noindex instruction are the pages a site needs for its own operation but does not need searchers to discover through search: internal search results, login and account pages, thank-you and confirmation pages, staging or preview pages that are temporarily public, printer-friendly duplicates, and thin utility pages that exist to serve a function rather than to answer a search query. A site that leaves such pages indexable does not usually suffer a dramatic penalty; it suffers a quiet dilution, as searchers encounter pages that were never meant for them, and as the site's indexed collection fills with pages that do not represent its best work. Noindex is the tool for keeping the indexed collection honest.

What the Noindex Tag Is Not

A noindex tag is not a way to hide a page from people. Anyone with the URL can still open a noindexed page, link to it, and share it, and the page will load exactly as before. Site owners who need a page to be genuinely private, because it contains personal data, unreleased work, or anything that must not be seen, need a password or another access control, not a meta tag. Noindex controls search visibility only, and treating it as privacy is a serious error with predictable consequences.

A noindex tag is not the same as a robots.txt block, and the two must not be confused, because combining them in the wrong way defeats both. A page that is blocked in robots.txt cannot be crawled, and a crawler that cannot crawl a page cannot read the noindex tag on it, so the page may remain indexed indefinitely, exactly the outcome the owner was trying to avoid. Google's documentation on blocking indexing states this condition plainly: for the noindex rule to be effective, the page must not be blocked by a robots.txt file, because the crawler has to crawl the page in order to see the meta tags and headers. The rule to remember is simple: noindex needs access, and a crawl block removes access.

A noindex tag is also not a deletion, and it is not instant. A page that is already indexed will remain visible until the search engine recrawls it and reads the new instruction, which takes time, and the exact timing depends on how often the page is crawled. A site owner who needs a result removed urgently should use the search engine's removals tool for the temporary removal, and keep the noindex instruction in place for the lasting one. Adding the tag and expecting the result to vanish within the hour is a misunderstanding of how crawling schedules work, and it leads to unnecessary alarm when the result is still there the next morning.

Finally, a noindex tag is not a substitute for a canonical tag. The two instructions answer different questions. Noindex says whether a page should appear in search results at all. Canonical says which version of similar pages should be treated as the primary one, when several versions exist and the owner wants one of them to represent the group. A page can carry both instructions, and in some situations it should, but one does not do the other's job, and swapping them produces exactly the confusion the owner was trying to avoid.

How the Noindex Tag Works

The mechanism is deliberately simple, and its simplicity is a strength. When Googlebot crawls a page, it extracts the robots meta tag or the X-Robots-Tag header from the response. If the extracted value includes noindex, Google drops the page entirely from its search results, regardless of whether other sites link to it. The page is not demoted, hidden behind other results, or shown only sometimes; it is removed from the results collection. If the tag is later removed, the page becomes eligible for indexing again on a subsequent crawl, which makes noindex a reversible decision rather than a permanent sentence. That reversibility is useful, because it lets owners temporarily exclude pages, such as seasonal or draft pages, without destroying their future value.

Why the Noindex Tag Matters

It matters first because it keeps a site's indexed collection representative of its best work. Search engines judge a site partly by the pages they have indexed for it, and a collection padded with login screens, thank-you pages, internal search results, and duplicate utility pages presents a distorted picture, both to the search engine and to any searcher who stumbles onto one of those pages through an unusual query. Removing the utility pages from the index does not make the good pages better, but it stops the weak pages from speaking for the site, and it focuses the crawl attention that would have been spent re-checking them onto the pages that deserve it. That focus is a form of crawl budget management, achieved from the indexing side rather than the crawling side.

It matters third as a safety net during changes. A redesign, a migration, a staging environment that must briefly be public, a seasonal page that should not be found out of season: all of these are situations where a page's search visibility needs to change faster than its existence does, and noindex provides that control without deleting anything. Because the decision is reversible, it can be used experimentally: exclude a set of pages, watch the effect on the site's indexed collection and on search traffic, and restore the pages if the exclusion harms more than it helps. A reversible control used thoughtfully is worth more than a permanent decision made in haste.

Common Noindex Tag Mistakes

The first common mistake is the blocked noindex, described above and worth repeating because it is both common and self-defeating: a page carries a noindex tag and is also blocked in robots.txt, so the crawler never reads the tag and the page stays indexed. The fix is to remove the block and let the crawler read the instruction, or to choose a different removal method, such as deletion or password protection, if crawl access cannot be allowed. Any audit of noindex pages should begin by checking that none of them is blocked, because a blocked noindex is worse than no noindex at all: it gives the owner false confidence while the page remains visible.

The second mistake is noindexing a page that should be indexed, usually through a template error. A staging template, a draft setting, or a site-wide setting that was meant for one section is applied to the whole site, or to a section of valuable articles, and the pages quietly drop out of results over the following weeks. The failure is quiet because the pages still load perfectly for visitors, and the traffic decline is easily blamed on other causes. A pre-launch check that inspects the robots meta tag on a sample of live pages, and a Search Console check for pages reported as excluded by noindex, catches this failure early, and the robots.txt entry in this batch describes the companion failure from the crawl side.

The fourth mistake is expecting instant removal. A noindex tag takes effect when the page is next crawled, and crawl frequency varies by page, so a page that is rarely crawled may remain visible for some time after the tag is added. Owners who check the next day and find the result still present sometimes conclude the tag failed and remove it, or add a robots block on top of it, both of which make the situation worse. The correct response to a slow removal is patience, plus a removals request through Search Console if urgency genuinely requires it, while the noindex instruction stays in place for the lasting effect.

How to Apply the Noindex Tag Step by Step

Start by listing the pages that should not appear in search, with a reason for each. Go through the site's sections and ask of each page type whether a searcher arriving at it from a query would be helped or confused: login and account pages, internal search results, thank-you and confirmation pages, staging and preview pages, printer-friendly or parameter duplicates, and utility pages that serve a function rather than answer a query are the usual candidates. Write the reason beside each type, because a reasoned list can be reviewed and challenged, while a list of URLs without reasons cannot, and because a future editor needs to know whether a page was excluded deliberately or by accident.

Next, choose the delivery method for each type. HTML pages take the meta tag in the head section, which most content systems and SEO tools can add per page or per template. Non-HTML files take the X-Robots-Tag header, which requires a server configuration, and the owner should confirm with whoever manages the server that the header is applied to the right files and not to the whole site by accident. Do not use a robots.txt rule for this job, and do not use the noindex rule inside a robots.txt file, which Google does not support. The method must match the file type, or the instruction will never be read.

After that, add the tag and verify the crawler's view. Add the meta tag or header, load the page, and confirm the tag is present in the served HTML or headers, not merely in a setting that claims to add it. Then use Search Console's URL inspection tool to confirm that Googlebot sees the noindex instruction, and check the page indexing report over the following days for the page's status. Verify a sample from each page type, not just one page, because template errors repeat across every page that shares the template, and a single verified page does not prove the rest.

Two related guides help keep these signals consistent: sitemap errors that block indexing, where blocked URLs listed as indexable create contradictory signals, and the canonical versus redirect guide for bloggers, which covers when a page should be redirected rather than merely excluded from results.

How Rankenza Handles the Noindex Tag

Rankenza's articles are written to be found, so they are not noindexed; the value of a noindex instruction for a Rankenza customer lies in the surrounding site, where utility pages and duplicates should stay out of search while the articles themselves stay in it. Each article starts from research into the top 10 pages for its keyword, filtered by what the searcher actually wants, and the draft is written in original words, reviewed by a human editor, and given internal links from the customer's own sitemap, so that the indexed collection grows with pages that deserve to be found, rather than with pages that dilute it. Where a customer's site has pages that should not appear in search, the correct advice is the one this guide gives: leave the page crawlable, add the noindex instruction, and verify that the crawler sees it, rather than blocking the page and hoping the tag will somehow be read.

How This Looks on Rankenza

The comparison below is method against method, without inventing traffic or ranking figures for either side.

Sources: What the Research Says

According to Google Search Central's guide to blocking indexing with noindex, when Googlebot crawls a page and extracts the tag or header, Google will drop that page entirely from search results, regardless of whether other sites link to it, and the same guide states that the noindex rule in a robots.txt file is not supported, and that the crawler must be able to crawl the page in order to see the meta tags and headers. Google's guide to controlling what is shared with search adds the surrounding framework, listing the noindex rule alongside removal and password protection as distinct tools for distinct jobs, and distinguishing an indexing instruction from a crawl block. Google's robots.txt introduction reinforces the boundary from the other side, stating that robots.txt is not a mechanism for keeping a page out of Google, and directing owners to noindex or password protection instead. Moz's Beginner's Guide to SEO covers the robots meta tag as part of the basic technical setup a site owner should understand. Where this guide gives no number, that is deliberate: the sources describe instructions and behaviours, not a measured outcome that would apply to every site.

• Google Search Central: Block Search Indexing with noindex
• Google Search Central: Control the Content You Share on Search
• Google Search Central: Robots.txt Introduction and Guide
• Moz: The Beginner's Guide to SEO

Frequently Asked Questions

Can I noindex a page and block it in robots.txt at the same time?

You can, but you should not, because the combination defeats the purpose. A blocked page cannot be crawled, and a crawler that cannot crawl a page cannot read the noindex tag on it, so the page may remain indexed indefinitely. Choose one approach: leave the page crawlable with a noindex tag if you want it out of results but still live, or delete it or password-protect it if you want it gone entirely.

How do I noindex a PDF or another non-HTML file?

Use the X-Robots-Tag HTTP header, served by the server alongside the file, with the value noindex. A PDF has no HTML head section to carry a meta tag, so the header is the only form available, and it is read by search engines in the same way as the meta tag. Confirm with whoever manages the server that the header applies to the intended files.

How long does noindex take to work?

It takes effect when the page is next crawled and the instruction is read, so the timing depends on how often the page is crawled, and a rarely crawled page may remain visible for some time. If removal is urgent, use Search Console's removals tool for a temporary removal while keeping the noindex tag in place for the lasting effect, and do not remove the tag when the temporary removal expires.

Does noindex hide a page from visitors?

No. A noindexed page still loads for anyone with the URL, and it can still be linked to and shared. Noindex controls search visibility only. Pages that must be private need a password or another access control, not a meta tag.