@mshonle
The numbers are mysterious and important!
@ced1401
A small comment for the algo, great video, great channel, i hope it will grow without the need for the dirty tricks you mention at the end🙂 "Quantization DESTROYS high precision and SHOCKS the entire industry !!"😀
@frobertrti
Great video. I realize that I have used quantization for years with only a vague understanding. Thanks
@theosib
I published a paper maybe 10 years ago on this. Naïve quantization of the model increases error rate both for continuous function modeling and classifiers. But if you perform training where quantization is applied during the forward pass, this results in increased error in the output, which is then reduced through back propagation, as long as the precision isn't too low for the problem. Weights and all of the backward-flowing signals have to be floating point so as to handle small error values and accumulate small weight adjustments. This also makes it "more differentiable," to to speak. But then everything gets quantized when forward propagating to ensure that the quantization faithfully affects error on the output nodes.
@Hydroculator
Your videos keep getting better and better. These should be required viewing for people, even just LLM users, who are just starting out. There's so much terminology that people just seem to use and not actually understand. This is so much better than trying slog through a dry Wiki page or scientific paper.
@DigitalMirrorComputing
"Quantize" is every drummer's favourite word!! haha This was a beautiful explanation!! Another great video Julia, that just inspired me to do one on a similar topic! Please keep them coming and thank you!
@markg5891
Wow, amazing! I learned a lot! Thank you for explaining quantization, a very technical complicated subject, in a more digestible form!
@delight163
What an amazing video, the creativity in your presentation is making it extremely digestible, thank you
@bbanahh
You just reminded me about Season 2 of Severance, which I had completely forgotten — thank you!
@balainblue
Loved it ! So much good information ... I had to watch it multiple times. :)
@golfer-e5l
Thanks a million - this was something I was struggling to understand. The diagrams are excellent!
@siamak81
Excellent video! I'm surprised that your videos are not as popular as they should be.
@romainjouhameau2764
I'm really impressed with the quality of your videos. They're really helpful Thanks a lot!
@Minoya1220
7:36 i definitely agree, i think the only way we could go lower for training would be to use a learning algorithm that isnt back propagation and doesnt depend on differentiability. Awesome video!!
@duongkstn
great explanation, thank you !. love your channel !
@uiteoi
That's a very informative video Julia, and much easier to comprehend than other videos. I'll nonetheless watch it again later with more attention to fully understand each detail.
@romgenie
Great explanations on quantization. I've known and understood digital music in this way, but it absolutely never occured to me that it was essentially quantization! Very insightful. Can 't wait for the Microsoft 1-bit paper review.
@DeepZipper
I can't wait for the video with more details about QAT! A while ago I've done some inference with few of the official QAT versions of Gemma3 and the quality was very good IMO. As far as I know QAT requires more expensive training, but for sure it makes a huge difference at inference time on a low end hardware . I hope that in the future more major providers of LLMs on HF train their models with QAT.
@UsevaladMilasheuski
A super underrated channel imho
@juliaturc1