I’m looking at starting a service that involves hosting a lot of LLM models, which are often going to be 16GB+ (compressed). I did a bit of searching for cloud storage providers with cheap egress, and the cheapest I could find is $0.01 per GB, which would still be $0.16+ per download.

How do sites like Huggingface or CivitAI do it? Lots of VC funding?

  • @[email protected]
    link
    fedilink
    English
    3
    edit-2
    1 year ago

    Storj does it at 7 USD/TB. And there are providers that technically provide unlimited bandwidth, like Hetzner’s dedicated servers; they still have some abuse limits, but even working within the limits should make it much cheaper. This means custom engineering though.