OnChainScans
LLMS.TXT 2026 READY•Web, SEO & Network Chains• BENCHMARK: 74ms

AI Bot & llms.txt Access Scanner

Evaluates AI indexing posture. Scans firewall blocking status for 25+ major AI crawlers, parses token-efficient llms.txt files according to the standard, and checks whether your site is accessible or walled off from LLM retrieval engines.

Standards Conformance:llms.txt Standard SpecificationRFC 9309 Robots Exclusion
INSTRUMENT SPEC
Modality:url
Privacy Level:100% In-Memory
Engine:Zero-Retention
Answer-First Technical Brief (AEO Overview)
Last Verified: November 2026 (Live Standards Verification)

This precision diagnostic tool audits diagnostic instrument against international technical standards. It verifies structural integrity, uncovers hidden security flaws, and outputs deterministic diagnostic reports entirely within browser memory or through zero-retention edge lookups.

Diagnostic Inspection Specifications

What We Scan & VerifyTechnical Standard / Threshold
Binary / Protocol ConformanceInternational Open Standard Specification
Cryptographic Integrity DigestSHA-256 (FIPS PUB 180-4)
Privacy Leakage & TelemetryZero Cloud Retention (100% In-Memory)
Diagnostic Latency BenchmarkSub-200ms Execution Target

Architecture Comparison: OnChainScans vs Legacy Web Checkers

Diagnostic CapabilityOnChainScans (Zero-Telemetry)Traditional Web Scanners
File Execution Boundary 100% Client-Side WebAssemblyUploaded to third-party server
Data Retention & Logs Zero Logs (Discarded upon tab close)Stored in database / analytics
Execution Engine Live RFC / Public APIs / Raw BytesSimulated / heuristic estimates
Subscription / Paywall 100% Free Forever (Open Tool)Sign-up gating or credit limits

How This Scan Works Under the Hood

01Phase 1

The scanner fetches /robots.txt and /llms.txt directly from the target domain, evaluating crawler authorization rules across 10 major AI bots.

02Phase 2

It audits user-agents including GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and Bytespider, determining whether each is explicitly allowed, blocked, or governed by wildcard '*' directives.

03Phase 3

It parses /llms.txt and /llms-full.txt against the emerging 2026 standard, checking for Markdown H1 titles, summaries, and token-efficient documentation indexes.

Frequently Asked Questions

What is the /llms.txt standard?

/llms.txt is a proposed standard similar to robots.txt that provides condensed, structured Markdown context for Large Language Models to read without parsing complex web layouts.

Does blocking GPTBot stop ChatGPT from showing my site in search?

Blocking GPTBot stops OpenAI from using your site for model training. ChatGPT live web browsing uses a separate agent ('ChatGPT-User').

What is Google-Extended?

Google-Extended is the user-agent Google uses to train Gemini and Vertex AI models. Blocking it does not affect normal Google Search ranking.

What technical standards does this tool enforce?

This tool strictly validates structures against official IETF RFCs, ISO standards, NIST guidelines, and W3C specifications without relying on simulated or mock heuristics.