so i was thinking, sure reddit is killing of it’s api but that doesn’t stop people from emulating reddit, and scraping it
wouldn’t a good compromise for people who still want the content of reddit, which are mainly the lurkers who don’t need to interact anyways, wouldn’t it be nice to have a sort of activitypub reddit proxy which enables you to see reddit posts on your lemmy client? sure it might be in a little legal gray zone, but who are they gonna sue haha if revanced is still legal because you compile it yourself, I’m sure it’ll be easy enough to find a way for this to be legal as well
Scraping is pretty fragile and not suitable for many things, also reddit goes to shit anyway, my Homepage with 130 subs in hot recommends the same posts since the blackout startet and they are all old.
but who are they gonna sue haha
The admin of the instance that provides such a proxy probably.
I’m a bit conflicted. On one hand I’d like to see reddit die, on the other hand I wish I had a backup of subreddits such as r/HFY. Maybe it would be enough to just dump the current archive into an instance and let various lemmy communities that map the subreddits grow from there.
There is a project to archive all Reddit before the API change.
More info:
https://old.reddit.com/r/DataHoarder/comments/142l1i0/archiveteam_has_saved_over_108_billion_reddit/
https://github.com/ArchiveTeam/reddit-grab
If scraping is against their terms of service (and it probably is), then it doesn’t seem like much of a legal gray zone to me. I think they would sue the people running the scrapers.
I used to work for a company that was constantly fighting scrapers. They loved our data! I have no idea how successful the bad guys were at doing it, but there were ways we could slow it down, block it, etc. Also, if you spend enough money with your CDN, there are lots of ways to deal with bots and scrapers. None of it is 100% effective, but you can sure make it a pain in the ass for your casual Lemmy admin.
I say we make our own content here, instead of pulling it from Reddit.