gedaliyah@lemmy.world to Technology@lemmy.worldEnglish · 5 months agoReddit blocking all major search engines, except Googlereadwrite.comexternal-linkmessage-square186fedilinkarrow-up1679arrow-down115
arrow-up1664arrow-down1external-linkReddit blocking all major search engines, except Googlereadwrite.comgedaliyah@lemmy.world to Technology@lemmy.worldEnglish · 5 months agomessage-square186fedilink
minus-squaredaniskarma@lemmy.dbzer0.comlinkfedilinkEnglisharrow-up2·5 months agoIf you publish your data in a friendly manner bots would have no need to crawl your site. Data that is more interesting and requested a lot could even be served over p2p. This moderl would generate less cost that dealing with constant bot scrappers. It is not a technical discussion. Or a discussion about associated cost. It’s a discussion about morals and economic models.
If you publish your data in a friendly manner bots would have no need to crawl your site.
Data that is more interesting and requested a lot could even be served over p2p.
This moderl would generate less cost that dealing with constant bot scrappers.
It is not a technical discussion. Or a discussion about associated cost. It’s a discussion about morals and economic models.