You are saying a lot of things that sound good to you without much grounding. You claiming this is a āwidespread and significant issueā is going to need some backing up, because I may be cautious about not claiming more knowledge than I have, but I know enough to tell you itās not particularly well understood, nobody is in a position to predict the workarounds and itās by no means the only major issue. The social media answer would be to go look it up, but itās the weekend and I refuse to let you give me homework. I have better things to do today.
Thatās the problem with being cautious about things. Not everybody has to. Not everybody knows they should or when. I donāt know if youāre dunning kruger incarnate or an expert talking down to me (pretty sure itās not the second, though).
And Iām pretty sure of that because yeah, it is an infinite doomsday slippery slope scenario. That I happen to know well enough to not have to be cautious about not having done all the reading.
I mean, your original scenario is that. Youāre sort of walking it back here where itās just some effect, not the endgame. And because now youāre not saying āif AI actually replaces programmers wholesaleā anymore the entire calculation is different. It goes back to my original point: What data will AI use to train? The same data they have now. Because it will NOT in fact replace programmers wholesale and the data is not fungible, so there still will be human-generated code to train on (and whatever the equivalent high enough quality hybrid or machine-generated code is that clears the bar).
AI has a problem with running out of (good) data to train on, but that only tells you there is a hard limit to the current processes, which we already knew. Whether current AI is as good as itās going to get or there is a new major breaktrough in training or model design left to be discovered is anybodyās guess.
If there is one, then the counter gets reset and we will see how far that can take the technology, I suppose. If there is not, then we know how far weāve taken it and we can see how far itās growing and how quickly itās plateauing. There is no reason to believe it will get worse, though.
Will companies leap into it too quickly? They already have. Weāre talking about a thing thatās in the past. But the current iteration of the tech is incapable of removing programmers from the equation. At most itās a more practical reference tool and a way to blast past trivial tasks. There is no doomsday loop to be had unless the landscape shifts signfiicantly, despite what AI shills have been trying to sell people. This is what pisses me off the most about this conversation, the critics are buying into the narrative of the shills aggressively in ways that donāt really hold up to scrutiny for either camp.
I mean, Iāll be honest, beyond the allegations of Dunning Kruger, I think this is actually just a grammatical mixup. I didnāt mean to write it the way you read it, and that may be my fault.
āIfā AI replaces human programmers wholesale, new human code will stop being created
It starts with āifā. As in, itās not a prediction of the future, itās a response to the hypothetical future of AI being advocated for by techbros and corporations.
And āwholesaleā doesnāt mean universally, it just means a lot.
And ānew human code will stop being createdā is true - I wasnāt saying all human code will stop being created. But AI replacing humans will stop humans from creating code. Many human projects will end, be reduced in scope, or wonāt start, as AI is forced into projects that it isnāt yet ready for.
New human code will stop being created is a true - if ambiguous - statement. I do apologize for the ambiguity.
But given that AI does not perform well with retraining on AI output - and Iām sorry but Iād be happy to hear from anyone who can tell me thatās not a given - the ouroborous eats its own tail in more ways than one.
Less human code means less AI training. More AI creating code with less human input therefore leads to less developments and advancements in programming in general.
I appreciate that you didnāt mean to say what you said, but words mean things. I can only respond to what you say, not what you meant.
Especially here, where the difference entirely changes whether youāre right or not.
Because no, āless human codeā doesnāt mean āless AI trainingā. It could mean a slowdown in how fast you can expand the training dataset, but again, old code doesnāt disappear just because you used it for training before. You donāt need a novel training dataset to train. The same data we have plus a little bit of new data is MORE training data, not less.
And less human code is absolutely not the same thing as ānew human code will stop being createdā. Thatās not even a slip of the tongue, those are entirely different concepts.
There is a big difference between arguing that the pace of improvement will slow down (which is probably true even without any data scarcity) and saying that a lack of new human created code will bring AI training to a halt. That is flat out not a thing.
That this leads to āless developments and advancements in programming in generalā is also a wild claim. How many brilliant programmers need to get replaced by AI before thatās true? Which fields are generating ādevelopments and advancements in programmingā? Are those fields being targeted by AI replacements? More or less than other fields? Does that come from academia or the private sector? Is the pace of development slowing down specifially in that area? Is AI generating ādevelopments and advancementsā of its own? Is it doing so faster or slower than human coders? Not at all?
People say a lot of stuff here. Again, on both sides of the aisle. If you know the answers to any of those questions you shouldnāt be arguing on the Internet, you should be investing in tech stock. Try to do something positive with the money after, too.
Iād say itās more likely youāre just wildly extrapolating from relatively high level observations, though.
Hah, alright. I tried to bring this back to productive conversation, but we donāt share the same fundamentals on this topic, nor do we apparently share an understanding of grammatical conventions, or an understanding of how to productively address miscommunications. For example, one of my first responses started by clarifying that āitās not that AI will successfully replace programmersā
I understand that the internet is so full of extreme, polarizing takes, and itās hard to discuss nuance on here.
Iām not trying to give you homework for this conversation - we can absolutely wrap this up.
I just highly recommend that you look into the technological issues of AI training on AI output. If you do discover that Iām wrong, I absolutely do not ask you to return and educate me.
But believe it or not I would be extremely excited to learn Iām wrong, as overcoming that obstacle would be huge for the development of this technology.
Hm. Thatās rolling the argument back a few steps there. None of the stuff weāve talked about in the past few posts has anything to do with the impact of AI-on-AI training.
I mean, you could stretch the idea and argue that there is a filtering problem to be solved or whatever, but that aside everything Iām saying would still be true if AI training exploded any time itās accidentally given a āHello worldā written by a machine.
A lack of new human created code will bring AI training to a halt. Thatās just not a thing
I didnāt roll back anything. The entire conversation has ultimately been us disagreeing on this one point, and we clearly canāt overcome that with more back and forth, so Iām happy to agree to disagree. Cheers.
But that point is not the same as LLMs degrading when trained on its own data.
Again, it may be the same as the problem of āhow do you separate AI generated data from human generated dataā, so a filtering issue.
But itās not the same as the problem of degradation due to self-training. Which Iām fairly sure youāre also misrepresenting, but I REALLY donāt want to get into that.
But hey, if you donāt want to keep talking about this thatās your prerogative. I just want to make it very clear that the reasons why thatāsā¦ just not a thing have nothing to do with training on AI-generated data. Your depiction is a wild extrapolation even if you were right about how poisonous AI-generated data is.
You are saying a lot of things that sound good to you without much grounding. You claiming this is a āwidespread and significant issueā is going to need some backing up, because I may be cautious about not claiming more knowledge than I have, but I know enough to tell you itās not particularly well understood, nobody is in a position to predict the workarounds and itās by no means the only major issue. The social media answer would be to go look it up, but itās the weekend and I refuse to let you give me homework. I have better things to do today.
Thatās the problem with being cautious about things. Not everybody has to. Not everybody knows they should or when. I donāt know if youāre dunning kruger incarnate or an expert talking down to me (pretty sure itās not the second, though).
And Iām pretty sure of that because yeah, it is an infinite doomsday slippery slope scenario. That I happen to know well enough to not have to be cautious about not having done all the reading.
I mean, your original scenario is that. Youāre sort of walking it back here where itās just some effect, not the endgame. And because now youāre not saying āif AI actually replaces programmers wholesaleā anymore the entire calculation is different. It goes back to my original point: What data will AI use to train? The same data they have now. Because it will NOT in fact replace programmers wholesale and the data is not fungible, so there still will be human-generated code to train on (and whatever the equivalent high enough quality hybrid or machine-generated code is that clears the bar).
AI has a problem with running out of (good) data to train on, but that only tells you there is a hard limit to the current processes, which we already knew. Whether current AI is as good as itās going to get or there is a new major breaktrough in training or model design left to be discovered is anybodyās guess.
If there is one, then the counter gets reset and we will see how far that can take the technology, I suppose. If there is not, then we know how far weāve taken it and we can see how far itās growing and how quickly itās plateauing. There is no reason to believe it will get worse, though.
Will companies leap into it too quickly? They already have. Weāre talking about a thing thatās in the past. But the current iteration of the tech is incapable of removing programmers from the equation. At most itās a more practical reference tool and a way to blast past trivial tasks. There is no doomsday loop to be had unless the landscape shifts signfiicantly, despite what AI shills have been trying to sell people. This is what pisses me off the most about this conversation, the critics are buying into the narrative of the shills aggressively in ways that donāt really hold up to scrutiny for either camp.
I mean, Iāll be honest, beyond the allegations of Dunning Kruger, I think this is actually just a grammatical mixup. I didnāt mean to write it the way you read it, and that may be my fault.
āIfā AI replaces human programmers wholesale, new human code will stop being created
It starts with āifā. As in, itās not a prediction of the future, itās a response to the hypothetical future of AI being advocated for by techbros and corporations.
And āwholesaleā doesnāt mean universally, it just means a lot.
And ānew human code will stop being createdā is true - I wasnāt saying all human code will stop being created. But AI replacing humans will stop humans from creating code. Many human projects will end, be reduced in scope, or wonāt start, as AI is forced into projects that it isnāt yet ready for.
New human code will stop being created is a true - if ambiguous - statement. I do apologize for the ambiguity.
But given that AI does not perform well with retraining on AI output - and Iām sorry but Iād be happy to hear from anyone who can tell me thatās not a given - the ouroborous eats its own tail in more ways than one.
Less human code means less AI training. More AI creating code with less human input therefore leads to less developments and advancements in programming in general.
I appreciate that you didnāt mean to say what you said, but words mean things. I can only respond to what you say, not what you meant.
Especially here, where the difference entirely changes whether youāre right or not.
Because no, āless human codeā doesnāt mean āless AI trainingā. It could mean a slowdown in how fast you can expand the training dataset, but again, old code doesnāt disappear just because you used it for training before. You donāt need a novel training dataset to train. The same data we have plus a little bit of new data is MORE training data, not less.
And less human code is absolutely not the same thing as ānew human code will stop being createdā. Thatās not even a slip of the tongue, those are entirely different concepts.
There is a big difference between arguing that the pace of improvement will slow down (which is probably true even without any data scarcity) and saying that a lack of new human created code will bring AI training to a halt. That is flat out not a thing.
That this leads to āless developments and advancements in programming in generalā is also a wild claim. How many brilliant programmers need to get replaced by AI before thatās true? Which fields are generating ādevelopments and advancements in programmingā? Are those fields being targeted by AI replacements? More or less than other fields? Does that come from academia or the private sector? Is the pace of development slowing down specifially in that area? Is AI generating ādevelopments and advancementsā of its own? Is it doing so faster or slower than human coders? Not at all?
People say a lot of stuff here. Again, on both sides of the aisle. If you know the answers to any of those questions you shouldnāt be arguing on the Internet, you should be investing in tech stock. Try to do something positive with the money after, too.
Iād say itās more likely youāre just wildly extrapolating from relatively high level observations, though.
Hah, alright. I tried to bring this back to productive conversation, but we donāt share the same fundamentals on this topic, nor do we apparently share an understanding of grammatical conventions, or an understanding of how to productively address miscommunications. For example, one of my first responses started by clarifying that āitās not that AI will successfully replace programmersā
I understand that the internet is so full of extreme, polarizing takes, and itās hard to discuss nuance on here.
Iām not trying to give you homework for this conversation - we can absolutely wrap this up.
I just highly recommend that you look into the technological issues of AI training on AI output. If you do discover that Iām wrong, I absolutely do not ask you to return and educate me.
But believe it or not I would be extremely excited to learn Iām wrong, as overcoming that obstacle would be huge for the development of this technology.
Hm. Thatās rolling the argument back a few steps there. None of the stuff weāve talked about in the past few posts has anything to do with the impact of AI-on-AI training.
I mean, you could stretch the idea and argue that there is a filtering problem to be solved or whatever, but that aside everything Iām saying would still be true if AI training exploded any time itās accidentally given a āHello worldā written by a machine.
I didnāt roll back anything. The entire conversation has ultimately been us disagreeing on this one point, and we clearly canāt overcome that with more back and forth, so Iām happy to agree to disagree. Cheers.
But that point is not the same as LLMs degrading when trained on its own data.
Again, it may be the same as the problem of āhow do you separate AI generated data from human generated dataā, so a filtering issue.
But itās not the same as the problem of degradation due to self-training. Which Iām fairly sure youāre also misrepresenting, but I REALLY donāt want to get into that.
But hey, if you donāt want to keep talking about this thatās your prerogative. I just want to make it very clear that the reasons why thatāsā¦ just not a thing have nothing to do with training on AI-generated data. Your depiction is a wild extrapolation even if you were right about how poisonous AI-generated data is.