Generative AI models, like large language models, often exceed the capabilities of consumer-grade hardware and are expensive to run. Compressing models through methods such as quantization makes them more efficient, faster, and accessible. This allows them to run on a wide variety of devices, including smartphones, personal computers, and edge devices, and minimizes performance degradation.
Join this course to:
1. Quantize any open source model with linear quantization using the Quanto library.
2. Get an overview of how linear quantization is implemented. This form of quantization can be applied to compress any model, including LLMs, vision models, etc.
3. Apply “downcasting,” another form of quantization, with the Transformers library, which enables you to load models in about half their normal size in the BFloat16 data type.
By the end of this course, you will have a foundation in quantization techniques and be able to apply them to compress and optimize your own generative AI models, making them more accessible and efficient.
Generative AI models, like large language models, often exceed the capabilities of consumer-grade hardware and are expensive to run. Compressing models through methods such as quantization makes them more efficient, faster, and accessible. This allows them to run on a wide variety of devices, inclu…
Some points apply to every course of this kind; see how we rank.
DeepLearning.AI is in Tier 2: excellent universities and the companies that build the technology of our institution ranking (90/100). Andrew Ng's courses.
This course provides an introduction to mathematical modeling of computational problems. It covers the common algorithms, algorithmic paradigms, and data structures used to solve these problems. The course emphasizes the relationship between algorithms and programming, and introduces basic performa…
Instructor: Prof. Erik Demaine, Dr. Jason Ku, Prof. Justin Solomon View the complete course: https://ocw.mit.edu/6-006S20 YouTube Playlist: https://www.youtube.com/playlist?list=PLUl4u3cNGP63EdVPNLG3ToM6LaEUuStEY This course is an introduction to mathematical modeling of computational problems, as…
This course teaches techniques for the design and analysis of efficient algorithms, emphasizing methods useful in practice. Topics covered include: sorting; search trees, heaps, and hashing; divide-and-conquer; dynamic programming; amortized analysis; graph algorithms; shortest paths; network flow;…
Instructor: Dennis Freeman, Kendra Pugh This course provides an integrated introduction to electrical engineering and computer science, including modern software engineering, linear systems analysis, electronic circuits, and decision-making. The lecture videos provide an overview of each topic, whi…
View the complete course: http://ocw.mit.edu/6-046JS15 Instructors: Erik Demaine, Srinivas Devadas, Nancy Ann Lynch 6.046 introduces students to the design of computer algorithms, as well as analysis of sophisticated algorithms. License: Creative Commons BY-NC-SA More information at http://ocw.mit.…
View the complete course: http://ocw.mit.edu/6-851S12 Instructor: Erik Demaine Videos of 6.851 Lectures Spring 2012 License: Creative Commons BY-NC-SA More information at http://ocw.mit.edu/terms More courses at http://ocw.mit.edu
