Technical SEO control builder
Build a valid robots meta tag and X-Robots-Tag without memorizing directives. See conflicts, deprecated rules and likely search-preview impact before publishing.
<meta name="robots" content="index, follow, max-snippet:-1, max-image-preview:large">VALID
Presets set a defensible baseline. Every control remains editable.
A robots meta tag controls indexing and search presentation; it does not block page access, guarantee removal timing or control every AI system. Use X-Robots-Tag for PDFs and other non-HTML files.
The Robots Meta & AI Snippet Control Builder creates robots meta tags and X-Robots-Tag headers for indexing, link handling, text snippets, image previews, video previews, time-based removal and certain Google AI Search controls.
A robots meta tag gives supported search-engine crawlers page-level instructions about indexing and how content may be presented in search results. It can request noindex, nofollow, disable or limit snippets, restrict image or video previews and apply other supported controls. It does not block users from visiting the page.
Robots directives are often discussed as though there were only two choices: index or noindex.
Google's supported controls are much more granular.
A page can remain indexed while limiting text snippets. You can permit text snippets while restricting image previews. You can allow indexing but prevent Google from offering a translated search result.
The important part is choosing directives based on a real publishing requirement rather than adding restrictions by default.
| Control | Main purpose | Important limitation |
|---|---|---|
| robots.txt | Controls whether supported crawlers may request a URL. | A blocked URL can still potentially appear in Search if Google discovers it elsewhere. |
| robots meta | Controls indexing and search-result serving for an HTML page. | Google needs to crawl the page to discover the directive. |
| X-Robots-Tag | Provides similar instructions through an HTTP response header. | The crawler must be able to request the resource and read the header. |
Google must crawl the URL before it can read a robots meta tag or X-Robots-Tag. If robots.txt prevents the crawl, the indexing instruction may never be seen.
Start with maximum visibility, standard SEO, no snippet or noindex depending on the actual publishing requirement.
Decide whether the resource should remain eligible to appear in search results.
Allow Google to choose the snippet length, limit text extraction or disable text snippets entirely.
Control maximum image and video preview availability independently.
Add indexifembedded, unavailable_after or other specialized controls only when they solve a specific requirement.
Copy the meta tag or HTTP header, publish it and confirm the final live source or response headers.
For Google Search, the default behavior is effectively index and follow when the page is otherwise eligible. Google also defines all as the unrestricted default. Explicitly writing those values is normally optional.
No robots meta tag
No additional indexing or serving restrictions are requested.
<meta name="robots" content="index, follow">
Usually unnecessary but commonly generated by CMS and SEO plugins.
noindex tells Google not to show the page, media file or resource in search results after Google has crawled and processed the instruction.
Certain internal utility or workflow pages may be publicly accessible but provide no useful search destination.
Some duplicate or filtered views may intentionally remain accessible while being excluded from Search.
A page can remain live for users while search visibility is intentionally restricted.
Google must recrawl and process the URL before the directive affects its index. Search Console removal tools solve a different temporary-removal use case.
Requests exclusion of the page itself while not adding a page-level instruction against following its links.
Requests both exclusion of the page and that supported crawlers not follow links from that page.
It asks Google not to follow the links on that page. Without the page-level rule, Google may use links on the page to discover linked URLs.
When only selected links need qualification, appropriate link-level rel attributes are usually more targeted than restricting every link on the page.
nosnippet prevents Google from showing a text snippet or video preview for the page. A static image thumbnail may still be shown in some cases. Importantly, Google also says this prevents the page content from being used as a direct input for AI Overviews and AI Mode.
nosnippet is a serving restriction rather than a noindex directive.
Google will not display normal extracted text beneath the result.
Google says nosnippet also prevents the page's content from being used as direct input for AI Overviews and AI Mode.
Use it only when restricting reuse is more important than allowing descriptive search previews.
| Directive | Meaning |
|---|---|
max-snippet:-1 |
Allows Google to choose the snippet length it considers useful. |
max-snippet:150 |
Allows a textual snippet of up to approximately the specified number of characters. |
max-snippet:20 |
Allows only a very short text snippet. |
max-snippet:0 |
Equivalent to nosnippet. |
For example, if the page contains both max-snippet:50 and nosnippet, nosnippet wins.
data-nosnippet is an HTML attribute supported by Google on div, span and section elements. Text inside those elements can be excluded from Google's search-result snippets while other page content remains available.
<p>This paragraph may be used in a snippet.</p>
<div data-nosnippet>
This section should not appear in Google's snippet.
</div>
data-nosnippet does not automatically prevent structured data inside that element from being used for supported search-result features.
| Value | Google behavior |
|---|---|
max-image-preview:none |
No image preview should be shown for the page. |
max-image-preview:standard |
A standard-size image preview may be shown. |
max-image-preview:large |
A larger preview, potentially up to viewport width, may be shown. |
The directive defines the maximum permitted preview size. Google still decides whether an image appears and which presentation is appropriate.
Restricts image previews for the page's search presentation.
Asks Google not to index images on that page.
If the same image appears on other crawlable pages, Google may still discover and index that image through those other pages.
| Value | Meaning |
|---|---|
max-video-preview:-1 |
No publisher-specified limit on video preview duration. |
max-video-preview:30 |
Allows up to 30 seconds of video preview. |
max-video-preview:0 |
At most, a static image may be used according to the image-preview setting. |
When Google believes a page is not in the searcher's preferred language, it may provide a translated title link and snippet. notranslate tells Google not to offer that translated search-result experience.
Google says indexifembedded allows content to be indexed when it is embedded inside another page through an iframe or similar mechanism even though the standalone URL carries noindex.
It is designed for a specialized embedded-content use case, not as a general indexability directive.
It tells Google not to show the page in search results after the specified valid date and time. Google also says Googlebot will considerably reduce crawling of that URL after the expiration time.
Certain event or campaign pages may have a clearly defined Search expiration requirement.
Some licensed or contractual publishing arrangements may require a specific end date.
Specialized listings may need to stop appearing after a known deadline.
Google Search currently ignores noarchive. Google's cached-link feature no longer exists, so the directive has no remaining effect on Google's Search presentation.
The live AnswerEnginee builder labels noarchive as ignored by Google and retains it only for legacy compatibility scenarios.
<meta name="robots"
content="noindex">
X-Robots-Tag: noindex
PDFs cannot contain an HTML robots meta tag, so an HTTP X-Robots-Tag is the appropriate format.
X-Robots-Tag can be applied directly to image-file HTTP responses when file-level indexing controls are needed.
Server headers provide a way to control non-HTML resources at file, path or server-rule level.
| Directive | Google Search effect | Google AI effect |
|---|---|---|
noindex |
Page should not appear in Google Search after processing. | Removing Search eligibility also removes the normal Search foundation for appearing as a result. |
nosnippet |
No text snippet or video preview. | Google says the page content will not be used as direct input for AI Overviews or AI Mode. |
max-snippet:N |
Limits automatic textual snippet extraction. | Also limits the amount of content Google may use as direct input for AI Overviews and AI Mode. |
max-image-preview |
Limits image-preview size. | Controls preview permissions rather than serving as a general AI opt-out. |
These rules describe how Google Search uses and presents content. They do not automatically control every third-party AI system, crawler, model provider or training process.
Use when you do not want the resource to appear in Google Search at all.
Use when the URL may remain indexed but you do not want Google showing or directly reusing normal textual snippet content.
robots.txt:
User-agent: *
Disallow: /private-report/
Page HTML:
<meta name="robots" content="noindex">
If Googlebot is prohibited from requesting the page, Google cannot reliably read the robots meta tag contained inside it. Allow crawling when Google needs to observe a noindex or snippet-control instruction.
<meta name="robots" content="nofollow">
<meta name="googlebot" content="noindex">
For Googlebot, Google documents that these instructions combine into the effective behavior noindex, nofollow.
Initial HTML:
<meta name="robots" content="noindex">
JavaScript later removes it.
Serve the intended indexing state as early and consistently as practical rather than depending on later client-side correction.
No. Google states that structured data can remain usable for supported search features even when that markup appears inside a data-nosnippet element.
If you do not want specific structured information made available through schema markup, remove or modify the structured data itself.
| Configuration | Likely intent |
|---|---|
index, follow, max-snippet:-1, max-image-preview:large |
Broad Search visibility and rich preview permission. |
index, follow, nosnippet |
Keep the page indexable while disabling textual and video snippets. |
index, follow, max-snippet:100 |
Keep Search eligibility while limiting automatic text extraction. |
noindex, follow |
Exclude the page itself without adding page-level nofollow. |
noindex, nofollow |
Exclude the page and ask the crawler not to follow links on it. |
max-image-preview:none |
Restrict image previews while leaving text indexing rules unchanged. |
Plugins such as Yoast SEO can already generate indexability and robots directives at page and content-type level.
Themes and custom templates can also inject metadata into the document head.
WPCode or functions.php snippets can create additional directives that conflict with plugin output.
Identify the source of the existing directive and change the configuration at that source whenever possible.
Google may not be able to crawl the URL to discover the noindex.
A staging directive accidentally survives deployment to production.
A page is excluded from Search and every link is also restricted without a clear reason.
A publisher removes valuable search-result context without intending to restrict snippets or AI direct input.
Zero is effectively equivalent to nosnippet rather than “let Google choose.”
max-image-preview:none is mistaken for a complete image-indexing ban.
The directive is treated as active even though Google Search now ignores it.
Multiple plugins produce restrictions that combine into an unintended result.
Google-specific Search controls are assumed to govern every independent AI platform.
Do not add restrictive directives unless you can clearly explain the publishing requirement they solve.
Decide whether you want the crawler to access the URL separately from whether you want the URL indexed.
Prefer max-snippet or data-nosnippet when you need partial control rather than disabling all text previews.
Check the live source rather than relying solely on what the CMS editor says.
X-Robots-Tag restrictions may exist even when the HTML contains no robots meta tag.
Temporary staging noindex and robots blocks are common causes of post-migration visibility problems.
Treat nosnippet and max-snippet as Google Search content-use controls, not merely SERP-design settings.
PDFs, images and other non-HTML resources require HTTP-level controls when index restrictions are needed.
Build the configuration first, then verify crawling, indexing and serving evidence after implementation.
Direct answers to common questions about indexability, crawling, nofollow, snippets, image previews, X-Robots-Tag and Google's AI Search controls.
It is an HTML meta element that gives supported search-engine crawlers page-level instructions about indexing and search-result serving.
It is conventionally placed in the HTML head. Google documents that it can still respect robots meta tags found in the body, but placing page metadata correctly in the head is the cleaner implementation.
Google describes unrestricted indexing and serving as the default. Explicit index and follow values are therefore normally unnecessary.
It tells Google not to show the page, media or resource in its search results once Google has crawled and processed the directive.
No. In fact, Google needs to crawl the URL to discover and process the noindex rule.
That combination is unreliable because blocking the crawl can prevent Google from seeing the noindex instruction.
Yes. Google says it can sometimes index the URL without crawling the page if the URL is discovered through links elsewhere.
It asks Google not to follow links on that page.
No. They are separate rules. Use page-level nofollow only when you intentionally want that additional link-handling restriction.
Google defines none as equivalent to noindex, nofollow.
Google says nosnippet prevents text snippets and video previews from being shown for the page, although a static image thumbnail may still be possible.
No. It is a serving and content-use restriction, not a noindex directive.
Yes. Google currently says nosnippet prevents the page content from being used as direct input for AI Overviews.
Yes. Google's documentation says the same direct-input restriction applies to AI Mode.
It sets the maximum amount of text Google may automatically use for a search-result snippet, expressed as a character limit.
It allows Google to choose the snippet length it believes is most useful.
Google defines zero as equivalent to nosnippet.
Yes. Google says it also limits how much automatically extracted content may be used as direct input for AI Overviews and AI Mode.
Google applies the more restrictive rule, so nosnippet takes precedence.
It is an HTML attribute that lets you exclude selected text inside div, span or section elements from Google's automatically generated snippets.
Google's documented support is specifically for div, span and section elements.
No. It controls snippet use rather than making the content private or necessarily preventing indexing.
No. Google says structured data can remain usable even when declared within a data-nosnippet element.
It permits Google to display a larger image preview, potentially up to the width of the viewport.
No. It grants permission for a larger preview; Google still controls the actual search presentation.
It tells Google not to show an image preview for the page.
No. One controls preview size; noimageindex asks Google not to index images found on that page.
Potentially yes. Google may discover the same image through another page where it is not restricted.
Google says that at most a static image may be used, subject to the page's image-preview setting.
It means there is no publisher-specified maximum video-preview duration.
It asks Google not to offer a translated title and snippet version of that result.
No. They solve different problems. Hreflang helps Google understand language or regional URL alternatives.
It allows Google to index content when it is embedded within another page despite a noindex rule on its standalone URL.
No. Google says the directive only has an effect when accompanied by noindex.
It tells Google not to show the page in Search after a specified valid date and time.
No. It affects Google Search serving, not whether the page continues to exist on your website.
No. Google currently ignores noarchive because its cached-link feature no longer exists.
It is an HTTP response header that can carry robots directives without requiring an HTML meta element.
It is particularly useful for PDFs, images, videos and other non-HTML resources, or when server-level rules are more practical.
Yes. Use an X-Robots-Tag HTTP header because a PDF cannot contain an HTML robots meta element.
Yes. Google combines applicable rules and applies restrictive instructions when directives conflict.
Yes. SEO plugins, themes and custom code can each output directives, so verify the final live page.
No. Robots directives are not security controls. Private or confidential material should require proper authentication or access protection.
No. Google must recrawl and process the directive before the indexed state changes.
No. Support varies by crawler and platform. Google's documented AI effects apply specifically to Google's Search systems.
They are technical indexing and serving controls, not ranking-boost directives.
No. The live tool's score describes preview permissions, not ranking strength or quality.
No. The builder generates the code or header value you need to implement on your own website or server.
No. Implementation must first be live and accessible, after which the relevant crawler still needs to revisit and process it.
Use the Robots Meta & AI Snippet Control Builder to create valid indexing and preview directives, understand their Google AI Search implications and generate the correct HTML meta tag or X-Robots-Tag without memorizing the syntax.