.id websites and AI crawlers

Of 281 readable .id sites in the census, 10.0% disallow GPTBot and 8.9% disallow ClaudeBot.

SitePopularityrobots.txtRead on
google.co.idTop 10,000none disallowed2026-10-10
mothercare.co.idTop 10,0005 disallowed2026-10-10
ui.ac.idTop 10,000none disallowed2026-10-10
lazada.co.idTop 10,000none disallowed2026-10-10
shopee.co.idTop 10,000none disallowed2026-10-10
telkomuniversity.ac.idTop 10,000none disallowed2026-10-10
s.idTop 10,000none disallowed2026-10-10
kgmedia.idTop 10,000none disallowed2026-10-10
ugm.ac.idTop 10,000none disallowed2026-10-10
kaskus.co.idTop 10,000none disallowed2026-10-10
republika.co.idTop 50,00016 disallowed2026-10-10
uma.ac.idTop 50,000none disallowed2026-10-10
lynk.idTop 50,00011 disallowed2026-10-10
kompas.idTop 50,00019 disallowed2026-10-10
kontan.co.idTop 50,00019 disallowed2026-10-10
katadata.co.idTop 50,000none disallowed2026-10-10
grid.idTop 50,00016 disallowed2026-10-10
jakartaglobe.idTop 50,0003 disallowed2026-10-10
unhas.ac.idTop 50,0001 disallowed2026-10-10
um.ac.idTop 50,00011 disallowed2026-10-10
esaunggul.ac.idTop 50,0003 disallowed2026-10-10
tripadvisor.co.idTop 50,0009 disallowed2026-10-10
upnvj.ac.idTop 50,000none disallowed2026-10-10
medcom.idTop 50,0003 disallowed2026-10-10
unja.ac.idTop 50,00020 disallowed2026-10-10
disway.idTop 50,000none disallowed2026-10-10
m88link.idTop 100,00011 disallowed2026-10-10
bni.co.idTop 100,000none disallowed2026-10-10
z-lib.idTop 100,0001 disallowed2026-10-10
uin-suska.ac.idTop 100,000none disallowed2026-10-10