以Google的官方说明为例:
User-agent: Googlebot-Image
Disallow: /images/dogs.jpg
但你实际并不想只屏蔽谷歌,还想屏蔽百度、360、搜狗等等,为了方便,直接修改为:
User-agent:*
Disallow: /images/dogs.jpg
第一次写robot.txt,注意项:
- 一般以User-agent为分界符
- 一般很少设置Allow,绝大多数情况下是设置Disallow
bilibili的robots.txt:
User-agent: *
Disallow: /include/
Disallow: /mylist/
Disallow: /member/
Disallow: /images/
Disallow: /ass/
Disallow: /getapi
Disallow: /search
Disallow: /account
Disallow: /badlist.html
Disallow: /m/
Youtube的robots.txt:
# robots.txt file for YouTube
# Created in the distant future (the year 2000) after
# the robotic uprising of the mid 90's which wiped out all humans.
User-agent: Mediapartners-Google*
Disallow:
User-agent: *
Disallow: /channel/*/community
Disallow: /comment
Disallow: /get_video
Disallow: /get_video_info
Disallow: /live_chat
Disallow: /login
Disallow: /results
Disallow: /signup
Disallow: /t/terms
Disallow: /timedtext_video
Disallow: /user/*/community
Disallow: /verify_age
Disallow: /watch_ajax
Disallow: /watch_fragments_ajax
Disallow: /watch_popup
Disallow: /watch_queue_ajax
Sitemap: https://www.youtube.com/sitemaps/sitemap.xml
我为什么要写这篇文章?
因为今天制作了打赏功能,不想二维码被谷歌图片收录,所以去找了一下如何屏蔽图片。
为什么要屏蔽收款码?
收款码虽然暴露给别人问题不大,但是有一个最严重的事情,就是诈骗。
别人如果拿你的收款码去诈骗,然后你的支付宝就会被封号,这种可能性是有的……
也许考虑过诈骗问题:央行规定这种印刷出来的固定不变的静态二维码每天收款不能超过500……
参考文档:
Today's comments have reached the limit. If you want to comment, please wait until tomorrow (UTC-Time).
There is 18h38m26s left until you can comment.