ylai@lemmy.ml to LocalLLaMA@sh.itjust.worksEnglish · 1 year agoMozilla's Llamafile 0.8.2 Scores Big With New AVX2 Performance Optimizationswww.phoronix.comexternal-linkmessage-square3linkfedilinkarrow-up132arrow-down11
arrow-up131arrow-down1external-linkMozilla's Llamafile 0.8.2 Scores Big With New AVX2 Performance Optimizationswww.phoronix.comylai@lemmy.ml to LocalLLaMA@sh.itjust.worksEnglish · 1 year agomessage-square3linkfedilink
minus-squarexcjslinkfedilinkEnglisharrow-up1·edit-210 months agoI just wanted to update this to mention that there are a lot of custom low level performance improvements for CPU based inferencing in Llamafile: https://justine.lol/matmul/
I just wanted to update this to mention that there are a lot of custom low level performance improvements for CPU based inferencing in Llamafile: https://justine.lol/matmul/