AI and crawler access
AI crawlers and AI assistants are welcome to read this site. That is not a grudging allowance written into a file nobody reads — it is the deliberate position, and this page is here so a person can check it without parsing robots.txt. Novus Visualizers is a free browser tool with no paywall and no sign-up wall on any public page, so an agent reading it sees exactly what a visitor sees.
Agents this site names explicitly
The wildcard rule would already admit all of them. They are named anyway, because some operators obey only a group that names them, and because a named group is a statement rather than an accident. The two lists below are read from the same module that builds the served robots.txt, so this page cannot claim an agent the file does not.
Bulk crawlers — indexing and training corpora
Fetching pages nobody is waiting on. These get the whole public reading surface: the marketing pages, the engine and template catalogue, the help centre, tutorials, blog, glossary, community and creator pages, and the machine-readable endpoints below.
- GPTBot
- OAI-SearchBot
- ClaudeBot
- Claude-Web
- Claude-SearchBot
- anthropic-ai
- PerplexityBot
- CCBot
- Google-Extended
- Applebot-Extended
- Bytespider
- Meta-ExternalAgent
- Amazonbot
- cohere-ai
Two pages are held back from this group and only from this group: /editor and /studio. That is not a policy about training data. They are client-only workspaces that server-render almost no text — a measured 34 characters in /editor’s body at load — so a bulk fetch of them indexes an empty shell and spends crawl budget for nothing.
User-initiated assistants — someone asked for this page
An AI acting for a person who wants this page, right now. The crawl-budget argument above does not apply at all when there is no budget being spent, so this group additionally gets the two workspaces. Every engine, control and export in this product runs in the browser inside them, which makes an AI browser driving the real page the only way an AI can actually use this site rather than just describe it.
- ChatGPT-User
- Claude-User
- Perplexity-User
- DuckAssistBot
- MistralAI-User
- Meta-ExternalFetcher
Where to read the site
The authoritative crawl rules, including the named groups below. If this page and that file ever disagree, the file is what is served and the file wins.
A plain-text description of what this site is, what each engine does, and where the important pages are.
The same map, plus the full text of every article and the complete control reference, so an answer can be grounded in one fetch.
Every indexable URL on this host, with last-modified dates.
Two more surfaces exist for machines rather than readers, so they are named here rather than linked: /mcp, a stateless Model Context Protocol endpoint for a client that speaks it, and /api/v1, a read-only public API. Both serve the same registries this site renders. The public API applies a per-IP rate limit and answers an over-quota request with 429 plus a Retry-After header; the MCP endpoint currently applies no limit of its own.
What this site asks for in return
Requests, not conditions. Access does not depend on any of them and nothing here is enforced by a technical measure.
Attribute what you quote, and link to it
If an answer draws on this site, name Novus Visualizers and link to the page it came from. That is a request rather than a condition of access — nothing here is gated on it, and no agent is blocked for ignoring it. It is the same request llms.txt makes at the bottom of its own text, and the reason is practical: a reader who is told where an answer came from can check it, and a link is the only thing that lets them.
Prefer the structured index over scraping the pages
llms.txt is a short map of the site. llms-full.txt carries the complete text of every help article, tutorial, blog post and glossary entry plus the full control reference for every engine, in one fetch. Reading either is cheaper for you and for this server than crawling the equivalent HTML, and it is the copy that is written to be read out of context.
Do not present exports or engine output as your own product
Videos a visitor renders here belong to that visitor with no attribution requirement, and the site says so publicly. The site's own software, design and written content are a different thing and remain proprietary; see the press kit for what may be reproduced.
Pace yourself, and identify honestly
The public API applies a per-IP rate limit and answers an over-quota request with 429 and a Retry-After header rather than a block, so an agent that reads that header will not be shut out. Sending a user agent that impersonates a browser to get around a rule is the one behaviour that turns a welcome into a problem.
What a crawl collects about anyone
Nothing personal. Every page an AI agent is invited to fetch is a public page: no account, no form, and no cookie is required to read any of them, and there is no personal data on them to collect. The community and creator pages show only what their author chose to publish.
- The API, the admin console and every signed-in surface are disallowed for every crawler including the wildcard, so an obedient agent never reaches an account at all. An unauthenticated fetch of one is answered with a redirect to sign-in regardless.
- A visitor’s audio, project document and rendered video are produced on their own device and are never uploaded unless they sign in and choose to save them. There is no server-side copy of an unsaved project for a crawler to reach even in principle.
- What the server does keep for any request, human or machine, is ordinary request metadata — the IP address, the user agent and the time — used for rate limiting and security. Analytics and advertising are consent-gated and load no scripts for a fetcher that runs no JavaScript, which is every bulk crawler.
The full account is on the privacy policy; this section describes only the part a crawl can touch.
If something here is wrong
Write to visualizers@novusstreamsolutions.com if an agent is being served something it should not be, or if you operate a crawler that this site names and you would like the rule changed. A request to be removed from the named list is treated as a defect report, not a negotiation.
Related: Languages · Editorial policy · Security · Press kit · Privacy