.be websites and AI crawlers

Of 195 readable .be sites in the census, 23.1% disallow GPTBot and 18.5% disallow ClaudeBot.

SitePopularityrobots.txtRead on
youtu.beTop 1,000none disallowed2026-10-10
google.beTop 10,000none disallowed2026-10-10
kuleuven.beTop 10,000none disallowed2026-10-10
ugent.beTop 10,000none disallowed2026-10-10
belgium.beTop 10,000none disallowed2026-10-10
rtbf.beTop 10,00015 disallowed2026-10-10
fgov.beTop 10,000none disallowed2026-10-10
vrt.beTop 10,000none disallowed2026-10-10
lesoir.beTop 10,00018 disallowed2026-10-10
hln.beTop 10,00015 disallowed2026-10-10
uclouvain.beTop 10,000none disallowed2026-10-10
uliege.beTop 10,000none disallowed2026-10-10
lalibre.beTop 10,00015 disallowed2026-10-10
telenet.beTop 10,000none disallowed2026-10-10
ulb.ac.beTop 10,000none disallowed2026-10-10
nieuwsblad.beTop 10,00018 disallowed2026-10-10
vub.ac.beTop 10,000none disallowed2026-10-10
blogspot.beTop 10,000none disallowed2026-10-10
standaard.beTop 10,00018 disallowed2026-10-10
ulg.ac.beTop 10,000none disallowed2026-10-10
uantwerpen.beTop 10,000none disallowed2026-10-10
wallonie.beTop 10,000none disallowed2026-10-10
dhnet.beTop 10,00015 disallowed2026-10-10
seld.beTop 10,000no robots.txt rules2026-10-10
demorgen.beTop 50,00017 disallowed2026-10-10
sudinfo.beTop 50,00015 disallowed2026-10-10
rtl.beTop 50,00018 disallowed2026-10-10
gva.beTop 50,00018 disallowed2026-10-10
7sur7.beTop 50,00015 disallowed2026-10-10
ebay.beTop 50,0008 disallowed2026-10-10
hbvl.beTop 50,00018 disallowed2026-10-10
tripadvisor.beTop 50,0009 disallowed2026-10-10
amazon.com.beTop 50,00018 disallowed2026-10-10
cfwb.beTop 50,0004 disallowed2026-10-10
namur.beTop 50,0005 disallowed2026-10-10
sporza.beTop 50,00015 disallowed2026-10-10
eventbrite.beTop 50,0001 disallowed2026-10-10
brugge.beTop 50,0009 disallowed2026-10-10
deltaweb.beTop 50,0003 disallowed2026-10-10
sciensano.beTop 50,00016 disallowed2026-10-10
sciencesdelafamille.beTop 50,0001 disallowed2026-10-10
qastan.beTop 50,000none disallowed2026-10-10
baloisetreklions.beTop 50,0003 disallowed2026-10-10
humo.beTop 50,00017 disallowed2026-10-10
restonsimples.beTop 100,0001 disallowed2026-10-10
jobat.beTop 100,000none disallowed2026-10-10
atac-atletiek.beTop 100,0001 disallowed2026-10-10
backontrackbelgium.beTop 100,0003 disallowed2026-10-10
thomasmore.beTop 100,0001 disallowed2026-10-10
autoscout24.beTop 100,0005 disallowed2026-10-10
ticketmaster.beTop 100,0001 disallowed2026-10-10
test-aankoop.beTop 100,0007 disallowed2026-10-10
liege.beTop 100,0005 disallowed2026-10-10