wires/hypnagaga_old
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #51 from ItsCurious/main

Browse source 
Fixed robots.txt rules to be on separate lines
This commit is contained in:
Lene Saile 2024-06-09 08:56:49 +02:00 committed by GitHub
commit 21ae6d3513
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 16 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

18
src/common/robots.njk
View file

@ -3,5 +3,19 @@ permalink: /robots.txt
eleventyExcludeFromCollections: true
excludeFromSitemap: true
---
User-agent: * Disallow: /404.html User-agent: GPTbot Disallow: / User-agent: ChatGPT-User Disallow: /
User-agent: Google-Extended Disallow: / User-agent: Omgilibot Disallow: / Sitemap: {{ meta.url }}/sitemap.xml
User-agent: *
Disallow: /404.html
User-agent: GPTbot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: Omgilibot
Disallow: /
Sitemap: {{ meta.url }}/sitemap.xml