- cross-posted to:
- devops
- cross-posted to:
- devops
Also, interesting comment I found on HackerNews (HN):
This post was definitely demoted by HN. It stayed in the first position for less than 5 minutes and, as it quickly gathered upvotes, it jumped straight into 24th and quickly fell off the first page as it got 200 or so more points in less than an hour.
I’m 80% confident HN tried to hide this link. It’s the fastest downhill I’ve noticed on here, and I’ve been lurking and commenting for longer than 10 years.
You must log in or register to comment.
Cloudflare took down our website after trying to force us to pay 120k$ within 24h
Yikes. That sounds bad.
I’m a SysOps engineer at a fairly large online casino.
Okay all my sympathy is gone. Online casinos deserve to die.
That said, my feelings towards economic vampires aside, the way the events unfolded is concerning to say the least. Cloudflare has been racking up evil-corp points quite rapidly in recent months.
As a person who works in server hosting (not as devops or IT), I’m often privy to customer interactions. I feel like my company does a really good job at damage control - where if we fuck up, some rep gets on the phone and makes things right. We’ve eaten costs on behalf of our customers.
But sometimes, you just gotta tell a customer to go fuck themselves.
And those customers, those biggest complainers are often in online gambling, crypto, adult content, or racist shit.
We get DDos’d a lot from it. But I’m glad the company I work for doesn’t bow down to garbage companies.
I’m honestly not surprised.
I used to hook up with a guy who was 100% convinced that he could game the system. It had something to do with break frequencies from various services and certain time windows for playing. He won sometimes, but he obviously didn’t talk much about his losses. He wasn’t a very happy person, and I think gambling offered an easy release.
That’s my big issue with gambling. It’s a business preying on addicts leaving many in financial ruin, and overall they do nothing for society at large. Here in Sweden it is regulated, but you honestly don’t notice it. There are so many internet casinos vanishing and cropping up on an almost daily basis. If you turn on the radio the adverts are like 40% online casinos, 40% sex toy sites, and 20% various services, like tyre shifting, glass repairs, etc.
A lot of those exploit EU rules on open markets to dodge proper local licensing (I’m also from Sweden)
That explains why they all seem so samey. E.g. online casinos never have any sort of physical presence like scratch cards or what have you, even though we have plenty of scratch cards.
No they don’t, at least for Sweden. I remember when they regulated the market in Sweden (I was working for a gambling company at the time and I had to run the security & compliance for the Swedish license). There is no such thing as open market for gambling where the market is regulated (Sweden, Denmark, Estonia, not sure if Norway finally regulated).
As far as I know, a handful of companies got regulated at the first round, some failed and could not operate in Sweden (this might mean you actually need to deny access to users from Sweden - since you do KYC you know) for quite some time (before they eventually managed to get the license).
The problem (why the other user mentions all similar sites) is that the big companies (say Kindred group, Betsson) tend to spin up many alternative brands with different looks to attract different customers.
Also, most of the companies that operate in Scandinavia use the Maltese license, but that works only in unregulated markets (Finland, Iceland and Norway for example - unless something changed in the last 3 years). That said, getting a license once you have another is quite simple usually. The Swedish license for example is easier to get than and very similar to the Danish one, so if you operate in Denmark you can just fill in the paperwork and you should be easily able to pick that one up.
I despise gambling, I don’t gamble myself and I consider it a tax on those who don’t know math. That said, I worked for a gambling company and I know that different companies target different types of customers. Also they have responsible gambling programs that are more or less serious (some of which might be required by regulations). The company I worked for operated in Scandinavia and was sportsbook heavy (vs casino heavy), and had quite serious measures against suspected addicts (immediate block, calling the person on the phone if there were any signs like long sessions etc., proof of income to set limits proportional to income etc.), because it was considered bad for business. Many companies in general are terrible, and especially those who depend on casino games, where the margins are fixed and the dynamics are more prone to create addiction (available 24/7, quick feedback etc.).
If it had been a sports betting site OP would have said so. The fact that they said “casino” says it all.
Many do both, I would say the vast majority. Same regulations and licenses apply, in fact. Simply some companies invest more in casino (which are purchased games from vendors in the vast majority of cases), some invest more in sportsbook. I guess the OP’s case is the former, but it’s not a very relevant distinction to make.
Removed by mod
I just wonder how much was left out
That’s fair, this is one part of the story, and it’s not like screenshots can’t be doctored. Any screenshot taken from the web is ridiculously easy to manipulate.
Key.
Key.
Key.
If it’s providing games of skill like online poker, it’s actually a very intellectually stimulating game. People have made a ton of instructional videos and many books on the poker variations.
After playing poker professionally I was able to leverage the skills of bankroll management and emotional control to become successful in investing in the stock market.
I held all of my stocks through the entire pandemic to rebound from a loss over multiple years holding tech to a $600,000 profit by buying at the bottom. If I hadn’t played poker I probably wouldn’t be able to stomach looking at a six digit loss in 2021. I only sold my bonds which I used to buy more stocks at a cheaper price (which was the point of the bond allocation)
I used to be in credit risk for a very large stock market company.
Calling the bottom of the market is the same as betting big and getting 21 in blackjack.
Super cool when it happens, but not skill. The number of grown men I had to hear crying because they were dollar cost averaging down to the bottom until they went broke still disturbs me.
I’m happy this worked for you, but it was not skill.
You can’t go broke with 1x leverage, and I bought $AMD all the way down from $100 to $70
If it went to $50 I wouldn’t go broke, if it went $1 I wouldn’t go broke. I just would have missed a bigger opportunity
If it goes from $100 to $1, there’s not much left to go before bankruptcy/delisting. Say hello to swaths of BBBY bag holders… oh wait, no bags left there!
It went to $2 in like 2013, close to bankruptcy. But it didn’t go bankrupt, and that’s all I’m betting on. My point is you don’t need to care about where the bottom is as long as you’re buying the dip.
Especially if you are just buying $VTI which won’t go to $1 any time soon.
I’m really glad for you, that sounds amazing. I don’t think you’re the rule, though. I think you’re the exception. I also feel like it wouldn’t be unfeasible to have competitive/e-sports poker while still strictly regulating online casinos.
I think we should keep games of skill and pure slot machine strictly separated
Is it really so crazy that if you practice gambling you might end up good at gambling? I dont see any difference between playing the stock market and playing cards for money.
Stocks are just the rich white man slot machine.
Yes, that’s the point, I’m good at combinatorics, probability. These mathematical skills have a lot of carryover
Would you advise others that learning through increasingly higher stakes is a good way to practice these skills and apply them to make a living?
I admit I dont have much issue with gambling as recreation/sport, but I dont know its a benefit to society to treat gambling as a profession.
Stock brokers gambling with others money is a whole other thing.
Only to the point that you get bored and do something useful with your new knowledge.
People enter tournaments for all kinds of games and those tournaments have money prizes and entry fees. I think it’s unfair to single out poker since it’s a game of skill.
It just so happens it doesn’t make sense to play without even the smallest stakes. Otherwise the best strategy is to go all in with any hand and try to double up quickly (if the chips are free, there’s no downside to doing this)
Even like $2 buy in games are much tighter than play money games
Everything in your post seems to give reasons for recreational gambling, and I do agree that the stakes are part of the game, and one with no stakes is markedly different. It does seem though that this is all in service of fair play, and to reward those for requiring they pay to prove they are in good faith.
To me I dont think the potential reward is the point with recreational gambling. You might even give your winnings back in a friendly game were you to find out that the stakes bled out into real life.
However I dont see how all of this applies to gambling as a profession and as a part of society in larger ways such as stock markets and Crypto currency. What’s the supposed benefits of that?
I would argue that the professional setting is not recreational at all, and in many cases is abusive, with there seeming to be some intent to disguise how abusive it is to the victim.
Nah, you don’t play with stakes that could matter to someone. In my case, our buy-ins in the home game are $28 when converted to dollars and nobody bats an eye at dropping $100
The tiny reward does make it more interesting because you actually care about winning. It’s better to do $20 stakes and keep the money than play for $100 stakes and have to give it back because someone was irresponsible with their money
It’s still gambling and getting people addicted.
People get addicted to alcohol and caffeine. Should we can those too?
Removed by mod
Jesus. Something shady is happening with cloudflare.
That does not inspire confidence.
Is there? The casino is on a cheap $250 a month plan they don’t belong on and they broke ToS with the domains. While also costing Cloudflare money each month (as the casino admits themselves, their traffic alone is worth up to $2000 a month).
It’s absolutely in the right of Cloudflare to drop a customer that’s bothersome. Casinos usually are (regulations, going around country restrictions), them costing them money on top is a massive issue.
120k a year is a big slap of course, but it’s probably the amount Cloudflare would want to keep them on as a customer. If they leave, so be it.
I’ve seen it several times before at companies I worked at. They cheaped out and went with a tiny service plan to coast by. Or even broke ToS because it would be cheaper. That usually got stopped by plans getting dropped (GitLab Bronze for example), cheap plans getting limited, or the sales team sending a ‘friendly’ message that we’re abusing their plan and how we’re going to fix it. If you don’t play along at that point you’re going to get the hammer dropped on you.
It also wasn’t 24h as the title says, the first communication happened in April. At that point they should have started to scramble, either upgrading to a bigger tier immediately or switching providers. And it’s totally normal to go to the sales team when you break the ToS of your plan or you abuse a smaller plan. They’re going to discuss terms, it’s not a technical issue.
Edit: And I should also say, the whole “paying for a whole year is extortion” is bullshit too. Their CFO or CEO told Cloudflare they are looking at switching providers (as they looked at Fastly). So of fucking course Cloudflare is going to demand a full year upfront. Otherwise the casino could pay for a single month and during that month they switch away to another provider. So Cloudflare would still be thousands in the red with that ex-customer after they used so much traffic the last few years.
That Cloudflare were justifiably unhappy with the situation and wanted to take action is fine.
What’s not fine is how they approached that problem.
In my opinion, the right thing for Cloudflare to do would have been to have an open and honest conversation and set clear expectations and dates.
Example:
"We have recently conducted a review of your account and found your usage pattern far exceeds the expected levels for your plan. This usage is not sustainable for us, and to continue to provide you with service we must move you to plan x at a cost of y.
If no agreement is reached by [date x] your service will be suspended on [date y]."
Clear deadlines and clear expectations. Doesn’t that sound a lot better than giving someone the run-around, and then childishly pulling the plug when a competitor’s name is mentioned?
Considering the perspective of the poster, the misleading title, etc - are you actually sure they didn’t?
Until Cloudflare responds to the post, it is IMO most beneficial to assume that the OP is being truthful and forthright. Doing so puts pressure on Cloudflare to either clarify or rectify the situation, whereas treating Cloudflare as though they are above suspicion accomplishes nothing.
After all, OP is very much the little guy here.
Eh, I have a couple of issues with that. For one, I doubt CF would even respond to this. I could easily see them using this very writeup to sue, with all the admissions in it.
The bigger part though, is calling an online casino, whose own IT team (the writer) admitted they were knowingly abusing the plan they were on, the “little guy”.
Are they small in comparison to Cloudflare? Absolutely, those schmucks have way too much control of the internet. Calling an online casino, whose own staff lied in the title, the little guy though… Doesn’t sit right with me.
No, I’m not going to side with them, or with CF. I’m going to make my assumptions off what I know (two terrible companies, one of which has a liar writing an article where they pretend to not have admittted to their own lies about the subject), and I’m going to assume this:
- Terrible casino used a plan they know they shouldn’t have been on.
- Terrible casino would have known what their traffic looked like for a long time.
- Awful CF noticed, and said “Hey guys, wrong plan, talk to sales.”
- Terrible casino threatened to just leave awfuo CF.
- Awful CF demands a year up front to ensure their costs are covered for previous abuse of the TOS.
- Awful CF figures “screw it, they are stringing us along, just cut them off so we don’t spend more money. TOS violation makes it easy.”
- Idiot IT from terrible online casino writes an article (stupidly) in which they admit to TOS violations, and pretends not to know about their own traffic from a resource they are relying on.
Seems pretty obvious to me. Barring further details, my assumptions are based on what I know, and I am perfectly happy sticking to that.
You do you.
From the additional info I read, it sounds more like the traffic wasn’t the main issue.
Gambling is forbidden in a lot of countries or heavily regulated. Cloudflare uses a common IP pool for all customers, so a casino customer would possibly get their IPs blacklisted (by various ISPs). The Enterprise tier of Cloudflare has “Bring your own IP (ByoIP)”, which they probably wanted to force onto this problematic customer to protect their business.
So it’s actually a problem, not just them paying not enough (which is another reason to get rid of them as fast as possible).
That would have been a mature thing to do.
The first communications were intentionally misleading though. CF wasn’t trying to solve a problem, they were trying to sell a service. If CF had just led with “upgrade or we nuke your site” then that’s scummy, but fair. Leading these guys on about technical problems and “trust & safety” bullshit was not fair at all.
Is that the first communication though? I would really like to hear Cloudflare’s side of the story.
There were 3 issues at once, so “trust & safety” is definitely part of it.
- Too much traffic use, this is purely a billing issue and CF probably wouldn’t even care (they haven’t for years) despite losing money
- Violating ToS with the domains, a minor infraction probably, but enough to cancel the contract
- This is the big one: CF uses one pool of IPs for all customers, the IP of a gambling site (like a casino) will get banned by ISPs of various countries (Gambling being illegal, strictly regulated and so on). This is the trust & safety issue, CF is actively hurting by keeping this customer. The enterprise plan they want to push them to has ByoIP (Bring your own IP), which would probably have been one condition of keeping them on. CF could have communicated better (if we got the full story here…), but for $250 a month they’d much rather kick the customer off their service
So maybe fucking say that?
And understandably you wouldn’t switch plans if all you’re talking to is sales without context.
The biggest red flag is the up-front payment for a year, gives the indication that they are in actual financial trouble, meaning short in cash right now.
Fucking idiots could have been just increasing the price yearly without any resistance, it’s unlikely a big casino would care about an extra 50-100 per month.
I’m pretty heavily invested in cloudflare. This news is definitely making me reconsider that investment.
What I can say, is their stock is looking very healthy. There are a lot of people buying a lot of stock for them and the prospect over the next 3 to 5 months looks very promising. The only way they wouldn’t have cash on hand as if they’re spending a ridiculous amount of cash on some project that I’m not aware of, and I feel like I would be aware of it.
This is very peculiar. Definitely warrants further investigation.
The only way they wouldn’t have cash on hand as if they’re spending a ridiculous amount of cash on some project that I’m not aware of, and I feel like I would be aware of it.
Maybe someone dipshit in marketing heavily invested in LLMs, since that’s the current hype among dipshits?
Cloudflare is publicly traded. They had $1.6 billion in cash or equivalents in December. Maybe they want to grease up the quarter to show better growth against the market, but that is a fuckload of cash.
or maybe it’s just a lower level manager who wants to polish up their revenue numbers to ask for a raise / promotion :) capitalists are ugly little critters like that.
As I said in another comment: The up-front payment is the only thing that makes sense for Cloudflare. You got a customer that’s costing you money each month. They broke ToS. You offer them a deal still to keep the services running. And their CEO/CFO tells you they are looking at other providers like Fastly.
If Cloudflare gave them a monthly contract then the casino would simply pay for a month and switch over their services to a competitor in that time. So Cloudflare loses all the money from the past (where the casino used far too much traffic) and will barely recoup 10k (minus the running cost, so more likely 7k at the high end) for a single month. It’s just not worth it.
So they offer: Stick with us for a full year at least or get fucked. Which is fair.
This scenario would mean major negligence on their part, as they had been with Cloudflare for years. When it was clear their services were costing more than the business plan paid for, that’s when they should have been contacted with clear numbers and a sheepish admission that “unlimited” doesn’t actually mean unlimited. It certainly seems shady to me that they attempted to make it about a TOS violation, that there’s no public information about enterprise level and pricing, and that the second they said they were talking to a competitor they had their data purged. It sounds like a failed attempt at extortion to me.
Read to me as:
Look, for a ToS-breaking [and/or] legally questionable site, we need a LOT to make it worth our while given we could be named as co-defendants someday - and obviously we’re not saying [cough] you’re a sketchy business we don’t want, because if we said that then we shouldn’t take bribes and should cancel you no matter what, so please read in between the lines.
I don’t think I particularly agree with this take, but it’s an interesting perspective.
If you are cloudflare and you suspect they broke ToS you quote which ToS has been broken, you specify which country blocking the customer is trying or has tried to circumvent and you force the customer to either move away or enforce geo-blocking for those countries (or have a separate account for those with your own IPs). There is no reason to cancel the whole account if the blocking is country-specific and there is no way that 10k a month is anyway a sufficient benefit for cloudflare for their IPs to be blocked in a country (affecting potentially hundreds or thousand of customers).
Exactly my thoughts
It’s because CF could see that moving to another provider would not be too difficult for them. If they went month to month then they would be gone after one month. So CF decided to go with extortion instead. Either pay for $120k, or CF will set fire to your business.
The biggest red flag is the up-front payment for a year
Another comment pointed out this was probably to prevent them from signing up for a month then using that month to bounce to another provider
I think it’s far more likely there’s some sales goal and or performance indicator at play here.
0F,CloudFlare don’t need to subsidise an online casino with millions of subscribers, at everyone else’s expense. Sure CF are a bunch of gigglefucks but this time I think they made a good decision.
Unless the casino is doing something illegal, it’s really not their decision to make. If they don’t want to subsidize them, all they’d have to do is be transparent and fair in their pricing. They way CF handled it instead just seems unprofessional and deceitful.
Exactly right.
If they are somehow losing money routing traffic then their pricing is fundamentally wrong, which is just as big of a black eye for cloudflare.
Subsidise how? They were using their existing plan as intended and even willing ditch the grey-area parts. If CF cannot afford to offer their plans as they are, they should change the offered plans, not hunt for easy prey.
Clearly CF were losing money on this account, so their other customers were subsidising.
Ah fuck it, I’m clearly at the bottom of a dog pile here, and I don’t want to be friends with any of you, nor am I going to start thinking that an online casino deserves anything but contempt, so I’ll be off.
No no, you’re really not far off. Few, if any people here are advocating for CF to continue to provide the same services for the same price. It seems clear to most (including the author) that a price increase was justified. The problem we’re all having is how they went about it, agnostic of the client.
(I don’t care who the client was and don’t care one way or the other about online casinos.)
I read the post and it doesn’t sound abusive at all
Plus: cloudflare kept putting them in touch with the sales department. Not legal. Not technical support
It’s just shit customer service, even if the customer is making a ton of money compared to your fees. Should a casino pay more for other services, too, just because they" don’t need a subsidy"?
As strange as this may sound… if you’re having serious technical problems, it’s the sales team you want to talk to.
Sales people have way more pull at tech companies than the engineering teams do. If your sales rep sounds an alarm, people listen. When tech support sounds an alarm, nobody bats an eye.
In this particular situation, they should be reaching out to cloudflare’s legal team. But, with their own legal team.
Good luck with the lawsuit for breach of contract when you broke the contract. I’m sure the judge will be amused.
It’s not the decision to ask more money, it’s how they made it and in violation of their own terms of service, also extortion, so yes they are dipshits.
Found the thread on HN. Here’s what (I’m guessing) a mod had to say:
It set off the flamewar detector, got flagged by users, and got downweighted by a mod.
The ‘customer support of last resort’ genre is common and not usually a good fit for HN [1]. If people feel this story is unusually relevant and interesting, I’m not sure I agree—long experience has taught us that one-sided articles like this nearly always leave out critical information—but I also don’t mind yielding in an occasional specific case, so I’ve rolled back the penalties on this thread.
The issue from our point of view is not about story X or company Y—it’s a systemic one: the most popular genres of submission (especially the rage-inducing ones) get massively over-represented by default, so countervailing mechanisms are needed [2] if we’re to have a space for the more intellectually curious stories that the site is meant for.
Okay, that’s understandable
Cracking insight - well done!
I love hacker news. The internet needs more things like this
HN thread is here and it’s on the front page 7 hours old: https://news.ycombinator.com/item?id=40481808
Many mentions made that a significant part of the issue seemed to be Cloudflare IP addresses getting banned in some countries. They wanted the customer to switch to a bring-your-own-IP plan.
Also, the discussion took place over 1 month, not 24 hours.
I think the HN thread is reasonably informative and nuanced. CF didn’t do great but it was somewhat a fog of war situation.
Yeah this substac just reads as we abused cloudflare then were surprised they didn’t take us saying no well.
The irony here, is this is the kind of vague and obtuse fuckery online casinos and sportsbooks pull with their customers all the time.
The irony here is that the article author confirms that they break TOS of CF and he still has a Pikachu face. Reddit discussion is pretty positive that CF is right in their decision and that new provider will shut them down at some time as well.
even if they were breaking tos (and i don’t think it sounds quite so cut and dry), shouldn’t the response be to notify them and allow them to fix it, or just terminate the account? demanding a ton of money to make the problem seems a skeevy way of handling it on cloudflare’s part.
They had two weeks to fix, instead they stood their ground and argued.
They very well knew that they were costing a lot more than the $250 they were paying and couldn’t get a deal anywhere else
Realistically, this is why you pay for Akamai. You don’t get these shenanigans.
How the fuck were they still on a $250 dollar a month plan when they pumped through $2000 a month worth of traffic? That’s shady on the companiy’s part and Cloudflare shouldn’t have allowed it to happen in the first place.
Each party played their part here and did shitty things. Sounds like the tech equivalent of a crackhead arguing about selling stuff to the pawn shop employee.
The $250/month plan supposedly includes unlimited traffic. If there’s actually a limit where you’re supposed to switch to a more expensive plan with no standardized price, maybe CF should say what the limit is?
They absolutely should have outlined a traffic limit for the $250 a month plan. That’s on Cloudflare for allowing it.
That said, if you make wildly excessive use of that loophole it probably shouldn’t surprise you if they do something like this. They called it “trust and safety” because it allows them to do anything they want under the guide of security.
Really, they didn’t define their service clearly and wanted to fire them as a customer unless they paid up for what they felt they were owed.
If something is marketed as “unlimited”, I don’t think there is such a thing as “wildly excessive use”. This isn’t a competitive eater going to an all-you-can-eat buffet and being mad about getting kicked out. It’s a business using a service in a way that’s seemingly in-line with what they paid for.
It’s the same definition of “unlimited” that Telcos use: you pay for unlimited but it really is XXgb of data per month, after that they either disconnect you or throttle your traffic at a glacial pace…
And in both cases, that is bullshit. Just because it happens doesn’t mean we should accept it.
A man walks into whorehouse at half past seven, inquires about prices, and learns that it’s 250 per night, per person for the room. “Everything they consent to is available to the customer” says the proprietor. Gladly he pays and climbs up the steps with his hand clasped tenderly, finally landing upon a plain pink cushion, whereupon he proceeds to fuck the absolute shit out of his companion for six full hours. The brothel quakes in rhythm with their dual shrieks of ecstasy for the full duration.
As he begins dressing himself across from the nearly comatose prostitute, the proprietor returns, requesting two hundred and ninety dollars for the extended stay and sixty for the damage to her employee. It was at that moment that the man realized that the madame was a 70 foot tall crustacean from the Paleozoic era. He yells “goddamn Loch Ness monster, I ain’t giving you no three fifty!”
…huh?
South Park reference. Probably the funniest episode in the whole show outside of “Hare Club for Men”.
“Unlimited” doesn’t exist in this universe. It’s always “Unlimited under fair use”.
If you pay for your water park ticket and they offer unlimited free drinking water fountains, you can’t pay for your ticket, call up Nestlé and bring in the water trucks.
Besides the IP poisoning from the casino, ToS violations and so on, just using this much traffic would probably be enough cause for a cancellation (or a forced plan upgrade).
I worked for Akamai for 7 years.
This is why, if your CDN infra is core to the operation of your business, you make your systems accommodate multi-CDN integration. Cutting one CDN off shouldn’t be significantly difficult, and it comes in handy during contract negotiations. All the major players work this way.
Well that all reads like extortion.
“Pay us money or we will destroy your business.” Pretty cut and dry extortion. The entire article was infuriating to read.
deleted by creator
Sounds like a shake down, and it couldn’t have happened to a more deserving group.
Still, real lesson in how Cloudflare does business.
THE HOUSE ALWAYS WINS MY FRIEND
True, and this time “The House” wasn’t the casino.
somebody has out gambled the gamblers. It’s finally happened!
deleted by creator
HN is a libertarian hellhole full of divorced incel energy
The tl;dr seems to be this was a money losing account for Cloudflare, and they couldn’t squeeze them so they weaseled out with some TOS violation to prevent losing money on what was promised to be unlimited traffic, they have better lawyers so they’re not worried.
Cloudflare 100% in the wrong here, they are closing accounts for TOS violations when they are just unprofitable, I would very strongly consider how tightly to couple with them knowing how cavalier they are about squashing small businesses.
If enough of these happen though, they’ll get destroyed by a class action lawsuit, and they’d deserve every bit of it
CF doesn’t give a fuck about 80tb of traffic. These guys were in severe TOS violation that could affect all CF customers if CF IPs got blocked. Given 48 hours to bring their own IPs and switch to (expensive AF anywhere) enterprise account and finally shut down TWO WEEKS later after trying to weasel their way out of this instead of accepting they need to pay to play this stupid game.
We’ve been CF customers forever and enshitification is definitely affecting all of their services and mostly customer support, but in this instance I’m 100% on the side of CF.
I’m 100% on the side of CF.
100%?
We scheduled a call with their “Business Development” department. Turns out the meeting was with their Sales team,
…
So we scheduled another call, now with their “Trust and Safety” team. But it turns out, we were actually talking to Sales again.
This is the part that’s ridiculous to me. If CloudFlare thinks they’re violating TOS that’s fine. If they’re willing to let them continue with their business as-is as long as they pay more? That’s fine. But, scheduling calls with one group and it turns out it’s actually CloudFlare’s sales team on the phone, that’s ridiculous.
Well, the way he describes it does sound messed up, but if the only solution CF is willing to accept is for them to bring their own IPs and that is only available with an enterprise plan, what kind of conversation were they expecting? And like I said in another thread, enshitification at CF affected their customer service the most. We went from being able to to speak directly to devs, to people who actually understood the problem, to first tier support that didn’t understand shit to 0 tier support that barely understands English.
It seems that you’ve misunderstood what the issue is here from cloudflare’s perspective. The customer was using cloudflare IP addresses, which is causing a knock-on effect for the rest of cloudflare’s customers and putting cloudflare as a business themselves at risk. The alternative was for the customer to use their own IP addresses as cloudflare advised . I’m not sure what you think ‘Business development’ teams do but I certainly wouldn’t be expecting engineering advice from them.
The customer was using cloudflare IP addresses, which is causing a knock-on effect for the rest of cloudflare’s customers and putting cloudflare as a business themselves at risk.
Right, so sales should not be involved in any way.
The alternative was for the customer to use their own IP addresses as cloudflare advised .
Again, sales should not have been involved in any way.
I’m not sure what you think ‘Business development’ teams do but I certainly wouldn’t be expecting engineering advice from them.
They are at least not identical to sales. They work with sales, but there’s at least some engineering component of the job. In this case if you were told you were meeting with the business development team, you’d expect that there would be talk about an engineering solution to the problem. Not just paying cloudflare more money.
These articles are always embellished, so I would take it with a grain of salt.
I worked for an online casino in the past. What they do is a standard in the industry. The company I worked for was a small startup and onwed hundreds of domains, mostly just to protect the brand, 98% of which redirected to the main domain, with a few serving slightly different sites for different jurisdictions (e.g. Ontario regulations require that everything happens under a .ca domain). The “blocking evasion” doesn’t require CF to do anything, besides forcing the customer to block traffic from certain countries (the ones where you are suspected to evade the block). At this point - if the casino is really operating in the black or gray markets - they can just set ingress to their site outside CF for those countries only if they really wanted. I worked also for a company who was doing this to allow traffic from Russia, changing every day mirrors (and they had an IT department of maybe 20, it was a joke), and Russia was the main market for them.
If what is told in the article is true - I.e. 95% of the traffic was through the main website - then it doesn’t look like they were really doing this sort of evading deliberately, considering that in that 5% you have all your alternative TLDs plus the traffic from gray/black markets. Having hundreds of domains and some small percentage of traffic from black markets is something that just happens, it’s different from continuously registering new domains for providing access where the previous ones got DNS blocked (this is domain block). It doesn’t seem this is what they were doing based on the article, and if they were, then CF emails didn’t mention it, which is insane.
Obviously we don’t know the full story, so everything has to he taken with a grain of salt.
I did a quick search through Cloudflare’s TOS and did not find anything about gambling. What was the TOS violation here?
What I’m seeing is Cloudflare communicating very poorly about what actions the customer would need to take to keep their site operating, why, and what the timeline would be. “We’ve determined operating your casino website on Cloudflare IP addresses is an unacceptable risk to our other customers and we require that you upgrade to an Enterprise plan within two weeks or your service will be terminated” is clear, concise, and I believe entirely fair. What they did here makes me think they’re an unreliable and unpredictable service provider.
Gambling is not TOS violation. Exposing CFs IPs to be blocked would affect ALL customers so CF is naturally aggressively protecting those Running any business that puts CFs IPs at risk is the TOS violation here.
I wish I was the fly on the wall during that meeting, but I have very little doubts casino understood the problem very well and were trying to weasel their way out of paying for an enterprise service (to anyone) and having to use their own IPs which are trivial to block. And if you continue buying more and rotating it will likely quickly get you on the black list with anyone still selling them.
I may be simplifying and maybe casino’s CTO and the entire tech team are a bunch of naive newborns, but I really fucking doubt it.
Again, I’m not seeing an unambiguous TOS violation here. They have some catch-all stuff about creating an undue burden and an even broader clause saying, essentially they can drop any customer without cause. I have no doubt Cloudflare is legally in the clear, but when I read about something like this, I think I wouldn’t set anything important up with Cloudflare as a critical part of its infrastructure.
Of course, the author could be leaving out a bunch of context to make himself look good.
If the article was about a non profit or a legit small business with a web presence, I would agree with you. We’re talking about massively risky business with spectacular profit margins.
I just don’t believe that CF suddenly realized these guys are rolling in money and wanted their cut. The risk just wasn’t worth it to CF confirmed by the fact that they did not negotiate at all and happily lost the casino as their client.
We’re easily making enough to pay $120k/yr to CF, but they are not creating that much value for us and we’re not introducing any risk to them so what we pay makes sense for both sides.
Maybe I haven’t been clear enough.
I have no objection to Cloudflare or any other service provider dropping a risky or unprofitable customer. That’s normal and fair in business.
What I don’t like is their apparent poor communication and failure to provide a clear (and reasonably distant) deadline so that the author’s company could find a solution that avoided downtime. Were I on that company’s board, I’d likely be pretty unhappy with the author for not having a contingency plan prepared in advance, but as a third-party observer my main takeaway is that if I rely on Cloudflare and they suddenly decide they don’t like something I’m doing, I’m screwed.
Your conclusion is based on only one side of the story. And this story is coming from an unnamed business that’s using social media to shit on a provider that dropped them.
But even assuming that’s true, name any other large provider that would behave differently. AWS will terminate your services instantly and their support is even worse than CF. Apple is the same and then will take 2 weeks to reply. Google is a ghosting champion.
Just to be clear I’m talking about B2B relationships. Not end user communication.
Okay, yes this is an issue. But small business? This was a multinational casino site… that doesn’t scream small business to me.
Online casinos can become international very simply, it doesn’t necessarily mean it’s a big company. You usually get a license and can operate in that country + a number of gray markets. Ofc there are also huge companies, but “international” doesn’t mean much for an online business.
Yes… but 4 million active users is quite high. I doubt anyone would consider that “small business”.
Yes, that’s true. I guess that is for sure a better metric that being “international”.
Multi CDN integration is a thing. And fuck CF. Unlimited means unlimited. Stop trying to lie to your customers and change the rules.
If the IP’s were an issue, then they wouldn’t have offered to make the issue go away with $$$.
But, the guy admits that what they were doing with the domains was expressly permitted in the “Enterprise” class service. If it was expressly prohibited in the “Business” class service, then they set themselves up for the shakedown.
I mean more money would solve the IP issue.
Step 1: Get more money Step 2: Buy additional dedicated IPs for the casino Step 3: Profit from the profits of a casino.
250$ a month for their service seems like cloudflare was straight up losing money on the deal. Although cloudflare seemed to have given them extra time than they said before terminating service, which they didn’t have to do. That being said, I think both sides suck here.
Nah. CF initiated a contract renegotiation, and then suspended services right after being informed the customer was price leveling.
That’s crappy.
They gave less than a single billing period notice for a price increase.
That’s crappy.
They sent a price increase for 40x the current billings, with no corroborated cost or value.
See where I’m going here?
I agree. It’s shitty for Cloudflare to just straight up destroy this company’s DNS, but also it seems like the company violated the ToS. They had about two weeks to migrate to something else, but instead they just continued debating with CF. Also, this company doesn’t have a secondary DNS server in case CF ever went down? That’s pretty stupid on their part. Redundant systems are key, I hope they learned that lesson haha
Isn’t CF advertising themselves as the solution to needing multiple DNS’ with their failsafes, switchovers and load balancing?
If I need to maintain multiple anyway, what’s the benefit of CF to begin with? There are a million CDNs out there I could use instead, if I still have to maintain the network architecture.
Regardless of what they tell you, if you care about uptime, you ensure this yourself. I feel this is 60% the company’s fault and 40% Cloudflare’s.
Do you believe everything that companies tell you? If Google or Apple tell you “we’re the solution to everything, you never need to buy anything else”, do you listen to them?
No, but if I use a service to solve a specific problem only to see the need to have a failover in place regardless, I might as well not use the service.
Thanks for actually reading my comment unlike the other guy
$250 a month unlimited business plan, is for a business with a Website, like a restaurant website that has a menu and takes reservations, maybe a small online store that handles a few dozen SKUs. When the website is the business they move you to an enterprise solution.
I really love cloudflare especially for my hobby projects but in this case they asked for outright Ransome. From this I learnt to keep Nameservers & domain sellers different. I am going to transfer domain away from nameserver.
Not just “this case”, there’s been countless cases like this with CF.
Could you provide a couple?
You can google for cloudflare issues ranging from providing hosting for actual nazi sites to extorting customers by threatening the exact scenario se saw in this blog post. Feel free to google “cloudflare account suspended” to see many posts about people having not just DDoS mitigation disabled, but everything related to an account deleted and disabled. Many of those people had the audacity to, get this, rely on DDoS protection! The nerve, right?
So no sources then?
If I have to dig, I’m most likely only getting one side of the story. This article pointed out that the customer broke the TOS and knew they were getting way better of a deal than they should’ve. I’m not so confident a random post online from angry customers is going to be so forthright.
That’s why I’m asking. If you’ve seen some particularly interesting stories, it would certainly be easier for you to find them them me. I’m not looking for butthurt customers who got caught breaking the rules, I’m looking for legitimate cases of CF bullying rules-following customers into paying more.
I told you how to find them so you wouldn’t have to bitch about my cherry picking. I can’t help if you’d rather bury your head in the sand, and it makes no difference to me what you believe.
Also, interesting comment I found on HN:
What is HN?
Hackernews
Hackernews, unironically named to appeal tech circles, but run by venture capital fund y-combinator, mainly to promote companies they invest in.
As such it’s mostly used by techbros (MBA types) and tech companies to show-off, start drama, push their PR, damage control, and occasionally post news.
It’s like linkedin, in reddit format. It’s all about your connections.
Sorry for confusion, I edited the post to make it clear
Pro tip: Don’t waste your time over there.
It’s incredibly selective about which topics it’s good for. Want insight into advanced mathematics or new programming languages and people there have amazing insight. But they bring the same level of confidence to the discussion when talking about topics they’ve no idea about.
That just sounds like the Internet in a nutshell for various topics.
It’s the Pravda of the VC-centric tech scene and has been for a very very long time.
(I am referencing the Soviet Union implementation thereof, for clarity)
It’s never going to bite the hand that feeds it, where people will voting-ring or the owners will just force-edit it to prevent that from happening. Outside of that, sometimes it might say something useful. The problem is that today’s problems are not because of a lack of advanced mathematics understanding or new programming languages.
It’s the Pravda of the VC-centric tech scene and has been for a very very long time.
At least someone else gets it.
It’s the Pravda of the VC-centric tech scene and has been for a very very long time.
A very interesting description. I only occasionally read HN via links from other sources, but I wouldn’t be surprised if there is a lot of truth to your characterization.
they bring the same level of confidence to the discussion when talking about topics they’ve no idea about.
Generally, I’ve found the discussion quality across these sites to be something like this:
HN > Lemmy > Reddit > 4chan
But yes, I have seen examples of incorrect confidence and bad-faith arguments on all of them. I don’t think it can be escaped in a public forum of humans. :)
I’d suggest they’re just as wrong about programming languages and maths as any other topic.
Repoint your DNS, send everything to legal, delete Facebook hit the gym
