Websites that block Google-Extended

6,434 of the 65,831 readable sites in the census (9.8%) disallow Google-Extended (Google, training) from the site root in robots.txt. Disallowing these keeps a site's content out of future model training. It does not remove what was already collected.

SitePopularityrobots.txtRead on
ngel.inkTop 100,0002 disallowed2026-10-10
zvw.deTop 100,0004 disallowed2026-10-10
texomashomepage.comTop 100,00013 disallowed2026-10-10
uowm.grTop 100,0008 disallowed2026-10-10
transalt.orgTop 100,00011 disallowed2026-10-10
corriereadriatico.itTop 100,00014 disallowed2026-10-10
gbci.orgTop 100,00020 disallowed2026-10-10
burnleyexpress.netTop 100,0008 disallowed2026-10-10
hznu.edu.cnTop 100,00021 disallowed2026-10-10
abc7amarillo.comTop 100,0007 disallowed2026-10-10
test-aankoop.beTop 100,0007 disallowed2026-10-10
eetasia.comTop 100,0008 disallowed2026-10-10
amestrib.comTop 100,00020 disallowed2026-10-10
microscopy-uk.org.ukTop 100,00015 disallowed2026-10-10
produccioncientificaluz.orgTop 100,00014 disallowed2026-10-10
tendanceouest.comTop 100,0009 disallowed2026-10-10
camera-wiki.orgTop 100,00020 disallowed2026-10-10
windpowermonthly.comTop 100,0002 disallowed2026-10-10
e-rara.chTop 100,00016 disallowed2026-10-10
ptonline.comTop 100,00010 disallowed2026-10-10
buscape.com.brTop 100,0008 disallowed2026-10-10
thetowntalk.comTop 100,00020 disallowed2026-10-10
lanutrition.frTop 100,0005 disallowed2026-10-10
birminghamal.govTop 100,0009 disallowed2026-10-10
tou.tvTop 100,00010 disallowed2026-10-10
wfpl.orgTop 100,00011 disallowed2026-10-10
lapcatsoftware.comTop 100,0009 disallowed2026-10-10
archives-pmr.orgTop 100,00013 disallowed2026-10-10
pathlms.comTop 100,0002 disallowed2026-10-10
mv-voice.comTop 100,0005 disallowed2026-10-10
yugatech.comTop 100,0008 disallowed2026-10-10
guelphmercury.comTop 100,00020 disallowed2026-10-10
ncnewsonline.comTop 100,00014 disallowed2026-10-10
egullet.orgTop 100,00022 disallowed2026-10-10
thenorthwestern.comTop 100,00020 disallowed2026-10-10
rairfoundation.comTop 100,0008 disallowed2026-10-10
energyconnects.comTop 100,0008 disallowed2026-10-10
jomo-news.co.jpTop 100,00019 disallowed2026-10-10
foodbabe.comTop 100,0008 disallowed2026-10-10
dja.comTop 100,00020 disallowed2026-10-10
dailynorseman.comTop 100,00017 disallowed2026-10-10
jaworzno.plTop 100,00021 disallowed2026-10-10
lafranceagricole.frTop 100,00020 disallowed2026-10-10
messenger-inquirer.comTop 100,00014 disallowed2026-10-10
krcgtv.comTop 100,0007 disallowed2026-10-10
wageindicator.orgTop 100,00020 disallowed2026-10-10
brewcrewball.comTop 100,00017 disallowed2026-10-10
crawfishboxes.comTop 100,00017 disallowed2026-10-10
nhk-ondemand.jpTop 100,0008 disallowed2026-10-10
wellsaidlabs.comTop 100,0003 disallowed2026-10-10
digitalnewsasia.comTop 100,0005 disallowed2026-10-10
nycpride.orgTop 100,00014 disallowed2026-10-10
antyweb.plTop 100,0008 disallowed2026-10-10
durhamregion.comTop 100,00020 disallowed2026-10-10
kde-apps.orgTop 100,00022 disallowed2026-10-10
fox44news.comTop 100,00013 disallowed2026-10-10
fantasiafestival.comTop 100,00013 disallowed2026-10-10
kbzk.comTop 100,0002 disallowed2026-10-10
amwell.comTop 100,00020 disallowed2026-10-10
nwt.seTop 100,00019 disallowed2026-10-10
jaci-inpractice.orgTop 100,00013 disallowed2026-10-10
mmsonline.comTop 100,00010 disallowed2026-10-10
rosenheim24.deTop 100,00014 disallowed2026-10-10
themorningsun.comTop 100,00018 disallowed2026-10-10
imprintnews.orgTop 100,0007 disallowed2026-10-10
typotheque.comTop 100,00013 disallowed2026-10-10
wizardzines.comTop 100,0002 disallowed2026-10-10
fox47news.comTop 100,0009 disallowed2026-10-10
bjpenn.comTop 100,00011 disallowed2026-10-10
economx.huTop 100,00012 disallowed2026-10-10
maizenbrew.comTop 100,00017 disallowed2026-10-10
blessyouboys.comTop 100,00017 disallowed2026-10-10
kiasma.fiTop 100,00020 disallowed2026-10-10
retronews.frTop 100,0008 disallowed2026-10-10
cobouw.nlTop 100,0002 disallowed2026-10-10
pulsenews.co.krTop 100,0009 disallowed2026-10-10
equativ.comTop 100,00011 disallowed2026-10-10
vai.comTop 100,00010 disallowed2026-10-10
leeandlow.comTop 100,00020 disallowed2026-10-10

The list shows the sites that have a page here, most popular first. This reports what the file asks. It does not show whether a crawler can actually reach the site: many sites also block crawlers at their CDN or firewall, and some crawlers ignore robots.txt.

page 1 page 2 page 3 page 4 page 5