Wander@yiffit.net to LocalLLaMA@sh.itjust.worksEnglish · 1 year agoWhat is better: higher quantiation or higher parameter count?message-squaremessage-square4fedilinkarrow-up12arrow-down10file-text
arrow-up12arrow-down1message-squareWhat is better: higher quantiation or higher parameter count?Wander@yiffit.net to LocalLLaMA@sh.itjust.worksEnglish · 1 year agomessage-square4fedilinkfile-text
For example, does a 13B parameter model at 2_K quantiation perform worse than a 7B parameter model at 8bit or 16bit?
minus-squarerufus@discuss.tchncs.delinkfedilinkEnglisharrow-up2·1 year agoWell, a few of those extra numbers are my fault. I edited my answer a few times. And lemmy reportedly counts every edit as an additional comment. (When user and community are on different instances.) I hope they fix that soon.
Yes
Well, a few of those extra numbers are my fault. I edited my answer a few times. And lemmy reportedly counts every edit as an additional comment. (When user and community are on different instances.) I hope they fix that soon.