Websites that block OAI-SearchBot

3,958 of the 65,831 readable sites in the census (6.0%) disallow OAI-SearchBot (OpenAI, search) from the site root in robots.txt. These feed AI answer engines that cite and link their sources. Disallowing them can take a site out of AI answers and out of the referral traffic that comes with them.

SitePopularityrobots.txtRead on
minyu-net.comTop 100,00019 disallowed2026-10-10
minpo.jpTop 100,00019 disallowed2026-10-10
gigapan.orgTop 100,00022 disallowed2026-10-10
eveningsun.comTop 100,00020 disallowed2026-10-10
ehime-np.co.jpTop 100,00019 disallowed2026-10-10
dnews.co.krTop 100,00020 disallowed2026-10-10
payless.comTop 100,00017 disallowed2026-10-10
dailydemocrat.comTop 100,00018 disallowed2026-10-10
leidschdagblad.nlTop 100,00018 disallowed2026-10-10
mcknightsseniorliving.comTop 100,0006 disallowed2026-10-10
scottishdailyexpress.co.ukTop 100,0008 disallowed2026-10-10
sankakucomplex.comTop 100,00017 disallowed2026-10-10
virginradio.co.ukTop 100,00018 disallowed2026-10-10
jinxcasino.netTop 100,00020 disallowed2026-10-10
granmadrid-casino.netTop 100,00020 disallowed2026-10-10
domoticz.comTop 100,00014 disallowed2026-10-10
myhighplains.comTop 100,00013 disallowed2026-10-10
cloverbingo.netTop 100,00020 disallowed2026-10-10
manualsonline.comTop 100,00021 disallowed2026-10-10
dtcc.eduTop 100,00019 disallowed2026-10-10
jotformpro.comTop 100,00022 disallowed2026-10-10
times-news.comTop 100,00014 disallowed2026-10-10
wumingfoundation.comTop 100,00018 disallowed2026-10-10
soatok.blogTop 100,00020 disallowed2026-10-10
fox21online.comTop 100,00014 disallowed2026-10-10
eurogamer.deTop 100,00019 disallowed2026-10-10
gaycitynews.comTop 100,00020 disallowed2026-10-10
samsungcard.comTop 100,00018 disallowed2026-10-10
studis-online.deTop 100,00016 disallowed2026-10-10
ipsoa.itTop 100,00017 disallowed2026-10-10
etree.orgTop 100,00022 disallowed2026-10-10
aberdeennews.comTop 100,00020 disallowed2026-10-10
farmer.plTop 100,00019 disallowed2026-10-10
thetexan.newsTop 100,00014 disallowed2026-10-10
austindailyherald.comTop 100,00019 disallowed2026-10-10
lanzoup.comTop 100,00022 disallowed2026-10-10
papamurphys.comTop 100,00020 disallowed2026-10-10
cleverbridge.comTop 100,0007 disallowed2026-10-10
insider.co.ukTop 100,0007 disallowed2026-10-10
haarlemsdagblad.nlTop 100,00018 disallowed2026-10-10
claws-mail.orgTop 100,0007 disallowed2026-10-10
yahoo.com.cnTop 100,00012 disallowed2026-10-10
bento.deTop 100,00015 disallowed2026-10-10
puppylinux.comTop 100,0008 disallowed2026-10-10
irif.frTop 100,00020 disallowed2026-10-10
moi.gov.cyTop 100,00021 disallowed2026-10-10
bnn.caTop 100,0005 disallowed2026-10-10
resume.seTop 100,00019 disallowed2026-10-10
propeller.huTop 100,00014 disallowed2026-10-10
ipdb.orgTop 100,00021 disallowed2026-10-10
the1a.orgTop 100,0007 disallowed2026-10-10
agweek.comTop 100,00020 disallowed2026-10-10
fvn.noTop 100,00017 disallowed2026-10-10
evanstonroundtable.comTop 100,00011 disallowed2026-10-10
corren.seTop 100,00018 disallowed2026-10-10
centraloregondaily.comTop 100,00014 disallowed2026-10-10
thestoryoftexas.comTop 100,00017 disallowed2026-10-10
newsandtribune.comTop 100,00014 disallowed2026-10-10
archinform.netTop 100,00012 disallowed2026-10-10
gramps-project.orgTop 100,00021 disallowed2026-10-10
cambridgeindependent.co.ukTop 100,00017 disallowed2026-10-10
psychiatryadvisor.comTop 100,0006 disallowed2026-10-10
27east.comTop 100,00014 disallowed2026-10-10
yinpinhuiju.comTop 100,00019 disallowed2026-10-10
parconazionale5terre.itTop 100,00020 disallowed2026-10-10
transcend.orgTop 100,0002 disallowed2026-10-10
troyrecord.comTop 100,00018 disallowed2026-10-10
whotv.comTop 100,00013 disallowed2026-10-10
roaminghunger.comTop 100,00016 disallowed2026-10-10
fnhk.czTop 100,00017 disallowed2026-10-10
zaap.bioTop 100,00020 disallowed2026-10-10
tvtonight.com.auTop 100,0008 disallowed2026-10-10
actionnewsnow.comTop 100,00014 disallowed2026-10-10
fltimes.comTop 100,00014 disallowed2026-10-10
ruralradio.comTop 100,00016 disallowed2026-10-10
ksml.fiTop 100,00013 disallowed2026-10-10
du.aeTop 100,00020 disallowed2026-10-10
firstforwomen.comTop 100,0007 disallowed2026-10-10
sanin-chuo.co.jpTop 100,00022 disallowed2026-10-10
warringtonguardian.co.ukTop 100,00018 disallowed2026-10-10
letelegramme.comTop 100,00020 disallowed2026-10-10
schooljobs.comTop 100,00020 disallowed2026-10-10
thedailynewsonline.comTop 100,00014 disallowed2026-10-10
dcourier.comTop 100,00014 disallowed2026-10-10
kakaobank.comTop 100,00021 disallowed2026-10-10
bendigoadvertiser.com.auTop 100,00014 disallowed2026-10-10
sentinelandenterprise.comTop 100,00018 disallowed2026-10-10
lanzouj.comTop 100,00022 disallowed2026-10-10
norfolkdailynews.comTop 100,00014 disallowed2026-10-10
skeptikon.frTop 100,00020 disallowed2026-10-10
blueridgenow.comTop 100,00020 disallowed2026-10-10
dalloz.frTop 100,00020 disallowed2026-10-10
yourcompany.comTop 100,00020 disallowed2026-10-10
upr.siTop 100,00021 disallowed2026-10-10
mmajunkie.comTop 100,00020 disallowed2026-10-10
wdel.comTop 100,00014 disallowed2026-10-10
figures.comTop 100,00014 disallowed2026-10-10
ehso.comTop 100,00021 disallowed2026-10-10
jdnews.comTop 100,00014 disallowed2026-10-10
sacredsites.comTop 100,00011 disallowed2026-10-10
easd.orgTop 100,0006 disallowed2026-10-10
times-herald.comTop 100,00014 disallowed2026-10-10
veikkaus.fiTop 100,00020 disallowed2026-10-10
productionhub.comTop 100,00018 disallowed2026-10-10
starmagazine.comTop 100,0007 disallowed2026-10-10
texomashomepage.comTop 100,00013 disallowed2026-10-10
uowm.grTop 100,0008 disallowed2026-10-10
lyg.gov.cnTop 100,0006 disallowed2026-10-10
corriereadriatico.itTop 100,00014 disallowed2026-10-10
gbci.orgTop 100,00020 disallowed2026-10-10
burnleyexpress.netTop 100,0008 disallowed2026-10-10
hznu.edu.cnTop 100,00021 disallowed2026-10-10
amestrib.comTop 100,00020 disallowed2026-10-10
trainsim.comTop 100,00018 disallowed2026-10-10
produccioncientificaluz.orgTop 100,00014 disallowed2026-10-10
tendanceouest.comTop 100,0009 disallowed2026-10-10
camera-wiki.orgTop 100,00020 disallowed2026-10-10
e-rara.chTop 100,00016 disallowed2026-10-10
buscape.com.brTop 100,0008 disallowed2026-10-10
aeroleads.comTop 100,0007 disallowed2026-10-10
thetowntalk.comTop 100,00020 disallowed2026-10-10
thisislocallondon.co.ukTop 100,00018 disallowed2026-10-10
tou.tvTop 100,00010 disallowed2026-10-10
guelphmercury.comTop 100,00020 disallowed2026-10-10
ncnewsonline.comTop 100,00014 disallowed2026-10-10
egullet.orgTop 100,00022 disallowed2026-10-10
thenorthwestern.comTop 100,00020 disallowed2026-10-10
jomo-news.co.jpTop 100,00019 disallowed2026-10-10
liege.beTop 100,0005 disallowed2026-10-10
dja.comTop 100,00020 disallowed2026-10-10
jaworzno.plTop 100,00021 disallowed2026-10-10
lafranceagricole.frTop 100,00020 disallowed2026-10-10
messenger-inquirer.comTop 100,00014 disallowed2026-10-10
wageindicator.orgTop 100,00020 disallowed2026-10-10
nycpride.orgTop 100,00014 disallowed2026-10-10
durhamregion.comTop 100,00020 disallowed2026-10-10
kde-apps.orgTop 100,00022 disallowed2026-10-10
fox44news.comTop 100,00013 disallowed2026-10-10
amwell.comTop 100,00020 disallowed2026-10-10
nwt.seTop 100,00019 disallowed2026-10-10
quest.nlTop 100,0003 disallowed2026-10-10
themorningsun.comTop 100,00018 disallowed2026-10-10
lesfrontaliers.luTop 100,00012 disallowed2026-10-10
kiasma.fiTop 100,00020 disallowed2026-10-10
facbook.comTop 100,00014 disallowed2026-10-10
leeandlow.comTop 100,00020 disallowed2026-10-10
twilog.orgTop 100,00014 disallowed2026-10-10

The list shows the sites that have a page here, most popular first. This reports what the file asks. It does not show whether a crawler can actually reach the site: many sites also block crawlers at their CDN or firewall, and some crawlers ignore robots.txt.

page 1 page 2 page 3