misk@sopuli.xyz to Technology@lemmy.worldEnglish · edit-21 day agoApple Intelligence summary botches a headline, causing jitters in BBC newsroomwww.theregister.comexternal-linkmessage-square61fedilinkarrow-up1248arrow-down12
arrow-up1246arrow-down1external-linkApple Intelligence summary botches a headline, causing jitters in BBC newsroomwww.theregister.commisk@sopuli.xyz to Technology@lemmy.worldEnglish · edit-21 day agomessage-square61fedilink
minus-squarebrucethemoose@lemmy.worldlinkfedilinkEnglisharrow-up5·1 day agoFor RAG data? It works. But its too slow for the weights. What generative models fundamentally do is run a full pass through the multi-gigabyte weights for every ‘word’ or diffusion step, so even 128-bit DDR5 like you find on desktop CPUs is too slow.
SSDs?
For RAG data? It works.
But its too slow for the weights. What generative models fundamentally do is run a full pass through the multi-gigabyte weights for every ‘word’ or diffusion step, so even 128-bit DDR5 like you find on desktop CPUs is too slow.