mastodon/public
nightpool 45b2bb464b Change robots.txt to exclude only media proxy URLs (#10038)
* Revert "Change robots.txt to exclude some URLs (#10037)"

This reverts commit 80161f4351.

* Let's block media_proxy

/media_proxy/ is a dynamic route used for requesting uncached media, so it's
probably bad to let crawlers use it

* misleading comment
2019-02-17 19:47:17 +01:00
..
avatars/original
emoji update twemojie to v2 (#7911) 2018-07-06 01:58:07 +02:00
headers/original
sounds
500.html
android-chrome-192x192.png
apple-touch-icon.png
badge.png
browserconfig.xml
embed.js
favicon-dev.ico
favicon.ico
mask-icon.svg
mstile-150x150.png
oops.gif
robots.txt Change robots.txt to exclude only media proxy URLs (#10038) 2019-02-17 19:47:17 +01:00
sw.js
web-push-icon_expand.png
web-push-icon_favourite.png
web-push-icon_reblog.png