Solve the 4-bit LLM setup problems all at one time — I was trying to run open-source LLM using the Huggingface Transformers model in 4-bit quantization mode. Why 4-bit quantization? because I can only run a 4-bit 30B LLM model in a GPU with 24G VRAM. Read several articles such as Making LLMs even more accessible with bitsandbytes, 4-bit quantization and…

Using the Latest Diffusers Monkey Patching function to load LoRA produces exactly the same result compare with A1111 — Pull the latest code from Hugging Face’s Diffusers code repository, and found that the newest code updated related to LoRA loading is updated and can do Monkey-Patching LoRA loading now. To install the latest Diffusers: pip install -U git+https://github.com/huggingface/diffusers.git@main

A Gen AI approach to business data analysis — By Andrew Zhu and Jacob H. Marquez The “Chat” in ChatGPT conveys that ChatGPT can provide information as a sort of dialog — albeit with an information cut date of September 2021 that reflects the temporal limit of ChatGPT’s awareness. But ChatGPT goes beyond being merely a powerful tool for…

Explain the Attention mechanism with a concrete sample — The Transformer machine learning model brought by the paper: Attention Is All You Need [1], completely transformed the machine learning area. And the Attention mechanism function as the core of the Transform model. The original paper throws us a formula like this without too much explanation: This article is trying…

Setup a CUDA-enabled WSL2 in Windows for LLM and Stable Diffusion models without performance sacrifice — There will be a time you need to run your models in a Linux environment. However, your most powerful GPU is running a Windows, and you don’t want to replace the whole system with Linux, and those oh, those games. WSL2 is a good choice that balances the gap between…

One tweet from https://twitter.com/finbarrtimbers says: I enjoyed the CUDA driver installation for 2 days plus another day to reinstall the whole system around 2 years ago. Now in 2023, with the correct steps, you can only enjoy CUDA drivers installation for just 5 mins in Ubuntu, here are the steps: …

Build A Dual GPUs PC for Machine Learning and AI with Minimum cost — Background and Building Target Both Stable Diffusion and offline LLM models require a huge amount of RAM and VRAM. To run and learn those models, I bought an RTX 3090 for its 24G VRAM. Actually, my aging Intel i7–6700k can still work well with a single RTX 3090, but when I throw another GPU…

Test various methods to boost Stable Diffusion package Diffusers' performance and lower VRAM usage — This is a testing note of running Diffusers with the latest PyTorch 2.0⁴. The purpose is to test out if all those optimization methods work or not if they work, how much performance improved. This article is going to cover the following performance & optimization methods: What is the baseline…

Overcoming token size limitations, custom model loading, LoRa support, textual inversion support, and more — Stable Diffusion WebUI from AUTOMATIC1111 has proven to be a powerful tool for generating high-quality images using the Diffusion model. However, while the WebUI is easy to use, data scientists, machine learning engineers, and researchers often require more control over the image generation process. This is where the diffusers package…