3166db9bcf
* chore: update robots.txt to disallow /settings and /my-account * Update robots.txt --------- Co-authored-by: Keith Williams <keithwillcode@gmail.com>
12 lines
355 B
Plaintext
12 lines
355 B
Plaintext
User-agent: *
|
|
Disallow: /sandbox
|
|
Disallow: /api
|
|
Disallow: /static/locales
|
|
Disallow: /settings
|
|
Disallow: /settings/my-account
|
|
|
|
# Specifically allow access to OG Image api, otherwise eg Twitter won't render these images.
|
|
Allow: /api/social/og/image
|
|
# Specifically allow access to api/app-store to allow search engines to fetch assets
|
|
Allow: /api/app-store/
|