Websites that block Googlebot

942 of the 65,831 readable sites in the census (1.4%) disallow Googlebot (Google, classic-search) from the site root in robots.txt. Classic search crawlers, shown as the baseline.

SitePopularityrobots.txtRead on
api.whatsapp.comTop 1,00022 disallowed2026-10-10
reddit.comTop 1,00022 disallowed2026-10-10
wpa.qq.comTop 1,00022 disallowed2026-10-10
mailchi.mpTop 1,00022 disallowed2026-10-10
chat.whatsapp.comTop 1,00022 disallowed2026-10-10
openssl.orgTop 1,00022 disallowed2026-10-10
co.comTop 1,00022 disallowed2026-10-10
qcloud.comTop 1,00022 disallowed2026-10-10
urldefense.comTop 1,00022 disallowed2026-10-10
1drv.msTop 1,00022 disallowed2026-10-10
wa.govTop 1,00022 disallowed2026-10-10
fbcdn.netTop 1,00022 disallowed2026-10-10
aka.msTop 1,00022 disallowed2026-10-10
xiaohongshu.comTop 1,00021 disallowed2026-10-10
rs6.netTop 10,00022 disallowed2026-10-10
gstatic.comTop 10,00022 disallowed2026-10-10
arte.tvTop 10,00022 disallowed2026-10-10
helsinki.fiTop 10,00022 disallowed2026-10-10
mcusercontent.comTop 10,00022 disallowed2026-10-10
click.linksynergy.comTop 10,00022 disallowed2026-10-10
cnki.netTop 10,00022 disallowed2026-10-10
home.blogTop 10,00022 disallowed2026-10-10
lexisnexis.comTop 10,00022 disallowed2026-10-10
weibo.cnTop 10,00022 disallowed2026-10-10
pxf.ioTop 10,00022 disallowed2026-10-10
connect.ok.ruTop 10,00022 disallowed2026-10-10
ad.doubleclick.netTop 10,00022 disallowed2026-10-10
nv.govTop 10,00022 disallowed2026-10-10
england.nhs.ukTop 10,00022 disallowed2026-10-10
1wcasino.comTop 10,00022 disallowed2026-10-10
secureserver.netTop 10,00022 disallowed2026-10-10
mjt.luTop 10,00022 disallowed2026-10-10
px.a8.netTop 10,00022 disallowed2026-10-10
magento.comTop 10,00022 disallowed2026-10-10
page-stats.deTop 10,0002 disallowed2026-10-10
toronto.eduTop 10,00022 disallowed2026-10-10
overdrive.comTop 10,00022 disallowed2026-10-10
asahi-net.or.jpTop 10,00022 disallowed2026-10-10
hubs.lyTop 10,00022 disallowed2026-10-10
hs-sites.comTop 10,00022 disallowed2026-10-10
casinonsvenska.euTop 10,00022 disallowed2026-10-10
pr.gov.brTop 10,00022 disallowed2026-10-10
akkadia.orgTop 10,00022 disallowed2026-10-10
arcor.deTop 10,00022 disallowed2026-10-10
qrco.deTop 10,00022 disallowed2026-10-10
depositfiles.comTop 10,00022 disallowed2026-10-10
catholicnewsagency.comTop 10,00022 disallowed2026-10-10
ebird.orgTop 10,00022 disallowed2026-10-10
asiaone.comTop 10,00022 disallowed2026-10-10
cmail20.comTop 10,00022 disallowed2026-10-10
cmail19.comTop 10,00022 disallowed2026-10-10
xanga.comTop 10,00022 disallowed2026-10-10
shapeways.comTop 10,00022 disallowed2026-10-10
litmus.comTop 10,00022 disallowed2026-10-10
law-gov.cnTop 10,00022 disallowed2026-10-10
redirectingat.comTop 10,00022 disallowed2026-10-10
skimresources.comTop 10,00022 disallowed2026-10-10
dzszyjsxx.comTop 10,00022 disallowed2026-10-10
megaphone.fmTop 10,00022 disallowed2026-10-10
widen.netTop 10,00022 disallowed2026-10-10
startersites.ioTop 10,00010 disallowed2026-10-10
href.liTop 10,00022 disallowed2026-10-10
china-heating.org.cnTop 10,00022 disallowed2026-10-10
missingkids.orgTop 10,00022 disallowed2026-10-10
nccu.edu.twTop 10,00022 disallowed2026-10-10
campaign.gov.ukTop 10,00022 disallowed2026-10-10
sendibm3.comTop 10,00022 disallowed2026-10-10
nzn-oa.comTop 10,00022 disallowed2026-10-10
todayonline.comTop 10,00022 disallowed2026-10-10
yidianzixun.comTop 10,00022 disallowed2026-10-10
sp1-brevo.netTop 10,00022 disallowed2026-10-10
createsend1.comTop 10,00022 disallowed2026-10-10
backblazeb2.comTop 10,00022 disallowed2026-10-10
treasurydirect.govTop 10,00022 disallowed2026-10-10
dblp.orgTop 10,00022 disallowed2026-10-10
judiciary.ukTop 10,00022 disallowed2026-10-10
jotformeu.comTop 50,00022 disallowed2026-10-10
cryptome.orgTop 50,0002 disallowed2026-10-10
derstandard.deTop 50,00022 disallowed2026-10-10
share.transistor.fmTop 50,0001 disallowed2026-10-10
byt360.comTop 50,0002 disallowed2026-10-10
perma.ccTop 50,0001 disallowed2026-10-10
rstyle.meTop 50,00020 disallowed2026-10-10
ly.comTop 50,0001 disallowed2026-10-10
sfn.orgTop 50,00022 disallowed2026-10-10
brighteon.comTop 50,0001 disallowed2026-10-10
easy.coTop 50,00022 disallowed2026-10-10
yangkeduo.comTop 50,00022 disallowed2026-10-10
cnnic.cnTop 50,0002 disallowed2026-10-10
jhtongchuang.comTop 50,00022 disallowed2026-10-10
lanzouw.comTop 50,00022 disallowed2026-10-10
cnnic.net.cnTop 50,0002 disallowed2026-10-10
lanzouu.comTop 50,00022 disallowed2026-10-10
antipope.orgTop 50,00011 disallowed2026-10-10
scontent.xx.fbcdn.netTop 50,00022 disallowed2026-10-10
lanzout.comTop 50,00022 disallowed2026-10-10
zx3315.cnTop 50,00022 disallowed2026-10-10
gnome-look.orgTop 50,00022 disallowed2026-10-10
fedorapeople.orgTop 50,00022 disallowed2026-10-10
kbstar.comTop 50,00021 disallowed2026-10-10
jornaldenegocios.ptTop 50,00022 disallowed2026-10-10
lanzoul.comTop 50,00022 disallowed2026-10-10
lanzoub.comTop 50,00022 disallowed2026-10-10
brainfacts.orgTop 50,00022 disallowed2026-10-10
yelp.co.ukTop 50,00022 disallowed2026-10-10
lanzoue.comTop 50,00022 disallowed2026-10-10
wilmerhale.comTop 50,00022 disallowed2026-10-10
lanzouq.comTop 50,00022 disallowed2026-10-10
hihostels.comTop 50,00014 disallowed2026-10-10
lanzoum.comTop 50,00022 disallowed2026-10-10
cat-v.orgTop 50,00022 disallowed2026-10-10
rockbox.orgTop 50,00022 disallowed2026-10-10
rpb-15.ruTop 50,0001 disallowed2026-10-10
macaulaylibrary.orgTop 50,00022 disallowed2026-10-10
pling.comTop 50,00022 disallowed2026-10-10
stargard.plTop 50,0002 disallowed2026-10-10
lanzouv.comTop 50,00022 disallowed2026-10-10
hpkucunmaoyi.comTop 50,0001 disallowed2026-10-10
opendesktop.orgTop 50,00022 disallowed2026-10-10
yelp.deTop 50,00022 disallowed2026-10-10
sharethrough.comTop 50,00022 disallowed2026-10-10
ha.org.hkTop 50,00022 disallowed2026-10-10
tradepub.comTop 50,00022 disallowed2026-10-10
inube.comTop 50,00022 disallowed2026-10-10
canva.cnTop 50,00010 disallowed2026-10-10
lanzoux.comTop 50,00022 disallowed2026-10-10
roodo.comTop 50,0001 disallowed2026-10-10
irdai.gov.inTop 50,00022 disallowed2026-10-10
lanzouo.comTop 50,00022 disallowed2026-10-10
factmags.comTop 50,0001 disallowed2026-10-10
jotform.meTop 50,00022 disallowed2026-10-10
userstyles.worldTop 50,00022 disallowed2026-10-10
thebell.co.krTop 50,00022 disallowed2026-10-10
ebay.ioTop 50,00022 disallowed2026-10-10
flipboard.socialTop 50,00011 disallowed2026-10-10
goooogla.comTop 50,0001 disallowed2026-10-10
buzzflash.comTop 100,00022 disallowed2026-10-10
kde-look.orgTop 100,00022 disallowed2026-10-10
gigapan.comTop 100,00022 disallowed2026-10-10
cqhot.cnTop 100,00021 disallowed2026-10-10
cisd.com.cnTop 100,0001 disallowed2026-10-10
dgs.ptTop 100,0002 disallowed2026-10-10
worldtaekwondo.orgTop 100,00021 disallowed2026-10-10
blaw.org.cnTop 100,00022 disallowed2026-10-10
jotform.usTop 100,00022 disallowed2026-10-10
newsmth.netTop 100,00022 disallowed2026-10-10
zefix.chTop 100,00022 disallowed2026-10-10
quicket.co.zaTop 100,00022 disallowed2026-10-10
officeplus.cnTop 100,0001 disallowed2026-10-10
magiran.comTop 100,00020 disallowed2026-10-10
jotform.coTop 100,00022 disallowed2026-10-10
sta426.comTop 100,0001 disallowed2026-10-10
upmedia.mgTop 100,00022 disallowed2026-10-10
iaria.orgTop 100,00022 disallowed2026-10-10
gigapan.orgTop 100,00022 disallowed2026-10-10
jotformpro.comTop 100,00022 disallowed2026-10-10
etree.orgTop 100,00022 disallowed2026-10-10
lanzoup.comTop 100,00022 disallowed2026-10-10
claws-mail.orgTop 100,0007 disallowed2026-10-10
sanin-chuo.co.jpTop 100,00022 disallowed2026-10-10
lanzouj.comTop 100,00022 disallowed2026-10-10
ehso.comTop 100,00021 disallowed2026-10-10
xingxiancn.comTop 100,0001 disallowed2026-10-10
lyg.gov.cnTop 100,0006 disallowed2026-10-10
tvpodolsk.ruTop 100,0002 disallowed2026-10-10
egullet.orgTop 100,00022 disallowed2026-10-10
kde-apps.orgTop 100,00022 disallowed2026-10-10

The list shows the sites that have a page here, most popular first. This reports what the file asks. It does not show whether a crawler can actually reach the site: many sites also block crawlers at their CDN or firewall, and some crawlers ignore robots.txt.

page 1