Solve the 4-bit LLM setup problems all at one time — I was trying to run open-source LLM using the Huggingface Transformers model in 4-bit quantization mode. Why 4-bit quantization? because I can only run a 4-bit 30B LLM model in a GPU with 24G VRAM. Read several articles such as Making LLMs even more accessible with bitsandbytes, 4-bit quantization and…

Using the Latest Diffusers Monkey Patching function to load LoRA produces exactly the same result compare with A1111 — Pull the latest code from Hugging Face’s Diffusers code repository, and found that the newest code updated related to LoRA loading is updated and can do Monkey-Patching LoRA loading now. To install the latest Diffusers: pip install -U git+https://github.com/huggingface/diffusers.git@main

A Gen AI approach to business data analysis — By Andrew Zhu and Jacob H. Marquez The “Chat” in ChatGPT conveys that ChatGPT can provide information as a sort of dialog — albeit with an information cut date of September 2021 that reflects the temporal limit of ChatGPT’s awareness. But ChatGPT goes beyond being merely a powerful tool for…

Explain the Attention mechanism with a concrete sample — The Transformer machine learning model brought by the paper: Attention Is All You Need [1], completely transformed the machine learning area. And the Attention mechanism function as the core of the Transform model. The original paper throws us a formula like this without too much explanation: This article is trying…

Setup a CUDA-enabled WSL2 in Windows for LLM and Stable Diffusion models without performance sacrifice — There will be a time you need to run your models in a Linux environment. However, your most powerful GPU is running a Windows, and you don’t want to replace the whole system with Linux, and those oh, those games. WSL2 is a good choice that balances the gap between…

One tweet from https://twitter.com/finbarrtimbers says: I enjoyed the CUDA driver installation for 2 days plus another day to reinstall the whole system around 2 years ago. Now in 2023, with the correct steps, you can only enjoy CUDA drivers installation for just 5 mins in Ubuntu, here are the steps: …

Build A Dual GPUs PC for Machine Learning and AI with Minimum cost — Background and Building Target Both Stable Diffusion and offline LLM models require a huge amount of RAM and VRAM. To run and learn those models, I bought an RTX 3090 for its 24G VRAM. Actually, my aging Intel i7–6700k can still work well with a single RTX 3090, but when I throw another GPU…

Test various methods to boost Stable Diffusion package Diffusers' performance and lower VRAM usage — This is a testing note of running Diffusers with the latest PyTorch 2.0⁴. The purpose is to test out if all those optimization methods work or not if they work, how much performance improved. This article is going to cover the following performance & optimization methods: What is the baseline…

Overcoming token size limitations, custom model loading, LoRa support, textual inversion support, and more — Stable Diffusion WebUI from AUTOMATIC1111 has proven to be a powerful tool for generating high-quality images using the Diffusion model. However, while the WebUI is easy to use, data scientists, machine learning engineers, and researchers often require more control over the image generation process. …