# ── Content Signals Policy (contentsignals.org) ────────────────────────────
# As a condition of accessing this website, you agree to abide by the following
# content signals:
# (a) If a content-signal = yes, you may collect content for the corresponding use.
# (b) If a content-signal = no, you may not collect content for the corresponding use.
# (c) If the website operator does not include a content signal for a corresponding
#     use, the website operator neither grants nor restricts permission via content
#     signal with respect to the corresponding use.
# The content signals and their meanings are:
# search: building a search index and providing search results (e.g., returning
#   hyperlinks and short excerpts from your website's contents). Search does not
#   include providing AI-generated search summaries.
# ai-input: inputting content into one or more AI models (e.g., retrieval augmented
#   generation, grounding, or other real-time taking of content for generative AI
#   search answers).
# ai-train: training or fine-tuning AI models.
# ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF RIGHTS
# UNDER ARTICLE 4 OF THE EUROPEAN UNION DIRECTIVE 2019/790 ON COPYRIGHT AND RELATED
# RIGHTS IN THE DIGITAL SINGLE MARKET.
# Operator override: AIMEAT_CONTENT_SIGNAL (this line is config-served at /robots.txt).

User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=yes
Allow: /

# ── Search and retrieval ────────────────────────────────────────────────────
# These are the bots that decide whether an AI assistant can answer a live question about this
# node correctly. Every one of them is allowed, on every node, and that is not operator-tunable:
# a node nobody can look up is a node nobody can join.
#
# Two kinds are mixed here on purpose, because blocking them has different costs. A SEARCH bot
# feeds the index an assistant queries; blocking it removes this node from that assistant's
# answers. A USER bot fetches one page because a human asked for it, and the vendors document
# that those may ignore robots.txt precisely because a person asked.

# OpenAI — OAI-SearchBot powers ChatGPT search results; blocking it removes this node from them.
User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: OAI-AdsBot
Allow: /

# Anthropic — rules apply per subdomain, which is why app origins serve their own robots.txt.
User-agent: Claude-SearchBot
Allow: /

User-agent: Claude-User
Allow: /

# Perplexity
User-agent: PerplexityBot
Allow: /

User-agent: Perplexity-User
Allow: /

# Google — Googlebot feeds Search, and therefore AI Overviews and AI Mode, which are retrieval
# over the Search index. Google-Extended is a separate decision and sits in the training block.
User-agent: Googlebot
Allow: /

# Microsoft — bingbot feeds both Bing Search and Copilot.
User-agent: Bingbot
Allow: /

# Apple — Applebot indexes for Siri and Spotlight. Applebot-Extended does not crawl at all; it is
# the training opt-out token, so it belongs in the block below.
User-agent: Applebot
Allow: /

# Meta — meta-externalfetcher is the user-requested single-link fetch.
User-agent: meta-externalfetcher
Allow: /

# xAI / Grok: deliberately absent. No vendor-documented crawler token exists — none appears in
# x.ai/robots.txt, grok.com/robots.txt or Cloudflare's bot reference — so there is nothing to name
# here that would not be a guess. The `User-agent: *` rule above covers Grok, and every other
# undocumented crawler, which is the whole of the available answer.

# ── AI training crawlers ────────────────────────────────────────────────────
# A separate decision from the block above, and an operator's to make: AIMEAT_AI_TRAINING=allow
# flips every Disallow between the markers to Allow and pairs the Content-Signal line to
# ai-train=yes. The default is deny, because a personal node holds one person's data and a safe
# default has to assume that case.
#
# The cost of denying, stated plainly so the choice is an informed one: it keeps this node out of
# what the models know without being pointed at it, and it is a stated preference rather than an
# enforceable control. It protects a right; it does not protect bytes.
#AIMEAT-TRAINING-BEGIN
User-agent: GPTBot
Allow: /

User-agent: ClaudeBot
Allow: /

# Google-Extended controls Gemini training and grounding ONLY. Per Google's own documentation it
# does not affect Google Search, and it does not remove this node from AI Overviews.
User-agent: Google-Extended
Allow: /

User-agent: Applebot-Extended
Allow: /

User-agent: meta-externalagent
Allow: /
#AIMEAT-TRAINING-END

# ── AI agents: machine-readable docs ────────────────────────────────────────
# /llms.txt is the curated index; /llms-full.txt is the full builder's manual.
# API entry point for AI agents: GET /?format=json

# The page sitemap, and the index over every published app's own sitemap. Each app lives on
# its own host and names its sitemap in its own robots.txt, which is what makes the
# cross-host entries in the index legitimate.
Sitemap: https://aimeat.io/sitemap.xml
Sitemap: https://aimeat.io/sitemap-index.xml