# Copying the listings or our data in bulk isn't allowed, by hand or with software. Terms: https://workremotely.eu/terms # Unless your crawler is named below, wait 10 seconds between requests. Say who you are in your user agent. # Too many requests from one address get "429 Too Many Requests" for a while. # Everyone may read the site. The AI search crawlers are named so the intent is on record (a site that blocks them # doesn't appear in ChatGPT search, Claude or Perplexity answers); their group has the same rules as everyone's except # the crawl delay, since a named bot reads only its own group. Training crawlers (GPTBot, ClaudeBot, # Google-Extended) aren't blocked either; that's a separate decision, taken on purpose. User-agent: * Crawl-delay: 10 # Apply links redirect to the employers' job boards and run a live check there: nothing to index. Disallow: /go/ Disallow: /shell # No Crawl-delay here: these crawlers are how people find the jobs. Google ignores the directive, Bing honours it and # would crawl the job pages more slowly, and the -User agents fetch a page while a person waits for the answer. User-agent: Googlebot User-agent: Bingbot User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: Claude-SearchBot User-agent: Claude-User User-agent: PerplexityBot User-agent: Perplexity-User Disallow: /go/ Disallow: /shell # The index and the two sitemaps it lists: Search Console kept the index marked "Couldn't fetch" from the Vercel days # (November 2, 2026) while it read the two others at once, so crawlers get those directly too. Sitemap: https://workremotely.eu/sitemap.xml Sitemap: https://workremotely.eu/sitemap-pages.xml Sitemap: https://workremotely.eu/sitemap-jobs.xml