Aha, I am doing the exact same thing 😂 https://lemmyverse.net
It looks really good! When I saw it on my feed my first response was “oh shoot someone beat me” 😂
Hehe they do look quite similar so great minds right 😁 I had the same thourght when I saw yours! I’ll add a link to yours to the list on my about page if you want 😀
that would be great, i’ll make sure you add yours as well 😀
Excuse me, this is a capitalist society. Please compete to the death
I wanna give both of you props for your work on these. It’s pretty cool to see the community jump on things like this. I’ve been half considering an attempt to whip up an android app just for a personalized experience, but I’m worried I’ll be too ambitions lol
For both of you, wouldn’t it be nice to let me add my home instance so you could provide quick links to subscribe (and open the community in my home instance).
I have a feature planned so you can login and one click subscribe/unsubscribe all on one page. Stretch goal though I’ve gotta patch some bugs first 😁
Potentially dumb question, but I’ve seen lots of people linking to this website:
What’s new/different about https://browse.toast.ooo or https://lemmyverse.net?
Is it because they also show instances, not just communities? Or is it more just a hobby project (nothing wrong with that, I’m 💯 on board with building things for the sheer enjoyment of it - even if it’s already a ‘solved’ problem)
I can only speak for myself, but I started work on mine since the Feddit one is fixed wide mode and the searching/filtering is less than ideal, I wanted to be able to see more details. Feddit one doesn’t show instance stats iirc. There are also other lists, but I didn’t find any of them super user-friendly… If Lemmy is gonna succeed Reddit there needs to be some super friendly tools to assist community discovery.
Nice! I sort of hope these community discovery tools & improvements eventually make their way back ‘upstream’ into Lemmy’s UI over time. Imagine not needing a separate discovery app because Lemmy’s built-in one is already the best one available 😄
I hope so too! The current problem is that new Lemmy instances start with no communities and you have to search to get federation started. If there was an easier way to do a global search within a fresh Lemmy instance that’d be nice!
For me these two community browsers linked here are a little more usable then the browse.feddit.de version (I am on desktop). Everyone is just trying to help grow the community in whatever way they can, and redundancies and choices never hurt. There is talks of each instance hosting a community browser over on the github issues, so this likely will be implemented in some form for each server as well.
Saw yours last night and prefer it. But wow, the communities have gone from 6k to over 10k overnight by your stats. Is that true?
Yeah they really shot up! I’m pretty sure it’s an accurate number. I did an crawl last night and picked up another ~3000 communities. A lot might be from instances that are under high load/timing out so might not have been picked up previously. Imma do some more analysis with my data in the future.
looks great, but it’s not finding anything for Deus Ex
https://lemmyverse.net/communities?query=deus+ex
but there’s 3 here
https://programming.dev/search?q=deus ex&type=Communities&listingType=All&page=1&sort=TopAll
Odddddd, I’ll have to check it out.
I think I figured it out actually, the one from lemmit.online was marked as suspicious and you have an option to show/hide those
the other one is from my little instance that no one knows about :(
I added
lemmy.mods4ever.comto the list ;)Haha thank you. Is that a manual list or is it pulled from the instance lists on other instances?
Is that a manual list or is it pulled from the instance lists on other instan
There is a manual seed list that starts the process, then instances are discovered from the
/instanceslist of each instance.I randomly checked this again and not seeing any of my communities or my mods4ever instance in lemmyverse.net
I did see you pull /instances from programming.dev and it is in this list, and it’s also in the instances list for beehaw and probably a bunch of others
idk what would have caused mods4ever to disappear from your data, but there might be more instances affected than just mine
Probably the best one I have found so far.
By the way, do you have any plan to expose an API or daily extract of the data you have?
While those websites are useful for manual searches, I think it would benefit the feddiverse much more if there was a way to integrate all those lists into an app. At least without resorting to web scrapping.
Update: https://data.lemmyverse.net/
Thank you for the information I’ll definitely check that out.
Cool!
Cans has source? I’ve been hacking on a per-instance community scraper (that pulls a list of communities from the community directory page) for !lemmy411[email protected], in an attempt to have a regularly published community list, but I’m a bit bogged.
I have links to the source for my crawler (written in nodejs with redis task queuing) on https://lemmyverse.net (GitHub link top right) 🤗 It’s a bit messy at the moment but has all the community crawling stuff.
Wow. Stickied THAT to [email protected] thanks!
I’m probably going to put the source up shortly but how I decided to find communities is by scraping my instances “federated with” list from https://toast.ooo/api/v3/site
Then going through the /.well-known/nodeinfo -> /nodeinfo/v2.0.json (or whatever url the well-known gives) to check the software to make sure it’s a Lemmy instance
Then using the Lemmy rest API to paginate through the open communities for that instance
How are you adding instances? I have a few communities I’d love to have listed.
In my other comment I described how I find communities but with this alpha I have a set list of instances to use
Are your communities listed on your current instance (lemmy.ko4abp.com)? I can add that to the temporary list if you’d like
They are! I would like that and I will be involved with your project if you decide to take the source public!
I’m operating a small instance, nothing here seems to be listed. They always appeared on browse.feddit.de after some time, so I suppose it searches for subs actively. Think you can do something like that in the future?
For this alpha I used a set list of servers but for the full release I plan to include as many servers as I can
Im currently rewriting the instance/community discovery system to make it more efficient
@[email protected] are you keeping stats over time?
I would love to sort by growing, or new. A lot of the time when you search by popilar, comments, etc the main ones are always at the top. I would like to see some of those up and coming ones that are growing at the moment.
This is especially important as i am on a small lemmy, and I wont get a heap of new
lemsas there wont be a lot of people adding new subscriptions, i need to find them myself.for browse.toast i am planning to have something similar to that – attempting to bring the light to smaller communities
discovery of new communities is a high priority rn
@[email protected] , that is cool. I will have a look at toast
You are right that discovery is a big issue to using lemmy in a distributed fashion (ie, using your local, small lemmy instance).
I am getting interesting content, but obviously the critical mass is still coming to get a heap of content, as well as the discovery type of thing.
looks great, but it’s not finding anything for Deus Ex
https://browse.toast.ooo/communities?search=deus+ex
but there’s 3 here
https://programming.dev/search?q=deus ex&type=Communities&listingType=All&page=1&sort=TopAll
Thanks for sharing !
Is it better than jerboa android app?
