noindex in robots.txt ignored high
"noindex" directive in robots.txt no longer supported
Google has ignored Noindex lines in robots.txt since September 2019, so pages you meant to hide stay in the index.
What you see
Noindex: /private/ Unsupported rule in robots.txt: noindex — GSC email — Remove “noindex” statements from your robots.txt file
What’s actually happening
Pages you tried to suppress with a Noindex line in robots.txt keep appearing in Google results. Search Console may have sent an email titled around removing noindex statements from robots.txt. The lines do nothing now — worse, if you also Disallow those URLs, Google can't crawl them to see any real noindex signal, so they can linger as URL-only results.
Common causes
- robots.txt contains Noindex: lines, a rule Google stopped honoring on September 1, 2019 (announced July 2, 2019).
- The intended pages were never given a real noindex via a meta tag or HTTP header.
- The URLs are both Disallowed in robots.txt and meant to be noindexed — but Disallow blocks the crawl, so Google never reads the page-level noindex.
- Reliance on a non-standard directive that some other tools tolerated but Google never officially supported.
- Pages already indexed before the change remain indexed because nothing has since told Google to drop them.
How to fix it
- Delete the Noindex lines from robots.txtThey have no effect and only create false confidence. Remove every Noindex: entry. If GSC emailed you about it, this is the line it means.
- Add a real noindex on the pagesPut <meta name="robots" content="noindex"> in the <head>, or send an HTTP header X-Robots-Tag: noindex (best for PDFs, images, and other non-HTML files). This is the supported way to keep a page out of the index.
- Make sure the page is crawlable so the noindex is seenRemove any Disallow in robots.txt for those URLs. If Google can't fetch the page, it can't read the noindex tag and the URL may stay indexed as a bare link. Let it crawl, drop the page, then block if you want.
- Use 404/410 for pages that should just be goneIf the content shouldn't exist at all, return 410 Gone (or 404). Once crawled, Google drops those URLs from the index.
- Request removal for anything urgentFor pages that need to disappear fast, use the Removals tool in Search Console to hide them temporarily while the meta noindex propagates.
Stop it recurring
Control indexing with meta robots noindex or X-Robots-Tag headers and use robots.txt only for crawl control, never for de-indexing.
Related errors