wires/hypnagaga_old
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

blocking known AI user agents

Browse source
This commit is contained in:
madrilene 2024-02-12 09:19:04 +01:00
parent 2d6df5c356
commit 78b407dc27
1 changed files with 16 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

17
src/common/robots.njk
View file

@ -3,4 +3,19 @@ permalink: /robots.txt
eleventyExcludeFromCollections: true
excludeFromSitemap: true
---
User-agent: * Disallow: /404.html Sitemap: {{ meta.url }}/sitemap.xml
User-agent: *
Disallow: /404.html
User-agent: GPTbot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: Omgilibot
Disallow: /
Sitemap: {{ meta.url }}/sitemap.xml