I’ve seen the occasional blip here, but this is the first time I’ve seen a complete outage of this instance. Hoping @Salamander wanders through and gives us the scoop?
Yes, sorry!
I often check the available disk space and expand it when it reaches ~90% occupancy. I increased it by 20 GB last Sunday night before traveling for work and did not expect it to fill up so quickly, but I underestimated the current rate at which the disk is being filled. This morning the disk was filled but I did not notice until after work.
I will keep a larger empty buffer to avoid this happening, but I also want to find some way to save disk space. The overwhelming majority of the space is occupied by images that are duplicated from other instances.
I highly appreciate your work! I know lemm.ee purged all federated images and disabled image uploads after the CSAM incidents, and they’re still doing fine because users use third party image hosts and images load directly from the other instances anyway, so maybe that’s a solution (at least the latter part)?
Also, I noticed all the activity that happened during the down time hasn’t been federated. Could that be solved in some way?
It’s a pain in the butt not having image uploads.
Oh I managed just fine. But I guess I was forced to (on my .ee account, that is).
Edit: but since the main issue is (understandably) media from other instances, I’m sure removing that would be adequate.
It’ll be harder for most people who don’t know how to set profile picture using the API or network requests field, since the way the Lemmy Frontend is done it requires Uploads in the Frontend, but doesn’t actually require them to set the image (just a URL to the Image, I have mine set to a Codeberg Repo containing the images, allows me to change them whenever I choose, and also guarantees stability since image hosts can be fiddly and also allows Tor users to see them more easily (many image hosts have connection issues under Tor, causing connectivity to be inconsistent).
Haha when I noticed it this morning I was like “oh I bet they’re at work but will be able to mess with it later”
Thanks for doing what you’re doing. I appreciate you
