Waifu Diffusion has gained significant traction as a specialized adaptation of the original Stable Diffusion anime model. The concept of fine-tuning is central here; it involves taking a pre-trained model and further customizing it with a specific dataset.
For Waifu Diffusion v1.4, the latest iteration, the base model of Stable Diffusion v2 was adapted using an extensive dataset of 5,468,025 anime text-image samples, sourced from Danbooru, a well-known anime imageboard.
How to install Waifu Diffusion step by step?
This article will guide you through the process of downloading and operating Waifu Diffusion via AUTOMATIC1111’s Stable Diffusion WebUI.
Step 1: The installation process
Getting Waifu Diffusion up and running is a straightforward affair, typically clocking in under 10 minutes, not counting the time needed for downloads. Ensure you have at least 10GB of hard drive space available before you begin.
As for system requirements, the following specifications are advised for optimal performance:
- A minimum of 16GB RAM
- A GPU from NVIDIA’s GTX 7xx series or newer, boasting at least 2GB of VRAM (AMD GPUs are compatible, but NVIDIA is the preferred choice)
- An operating system that’s either Linux, Windows 7/8/10,11, or Mac M1/M2 (Apple Silicon)
- The aforementioned 10GB of disk space, which will accommodate the models as well.
By meeting these specs, you’ll be well-prepared to make the most out of your Waifu Diffusion experience.
Step 2: Downloading the model files
The core of Waifu Diffusion lies in its model files, which are essentially the AI’s training data and capabilities encapsulated. These files have a ‘.ckpt‘ extension, signifying ‘checkpoint.’
To get the latest version of the Waifu Diffusion model, you can download it from the following link: Waifu Diffusion Model. Make sure to download the file named ‘wd-1-4-anime_e2.ckpt.’
Step 3: Downloading the web user interface
The Web UI serves as your control panel for generating content. Among the various options, AUTOMATIC1111’s Stable Diffusion WebUI stands out as the most popular choice. It’s easy to see why—the development team is incredibly responsive, and the interface is continuously updated with new features.
Depending on your operating system, the installation process varies:
On Windows:
- Download the ‘sd.webui.zip‘ file and extract its contents.
- Navigate to the extracted folder and double-click on ‘update.bat‘ to update the Web UI to its latest version. If a security warning pops up, click the “Run Anyway” button. Close the window once the update is complete.
- Next, double-click on ‘run.bat‘ to launch the Web UI. Note that the first launch will take some time as it downloads all the required packages, including the Stable Diffusion v5.1 model.
- Once all packages and the model have been successfully downloaded and installed, a message will appear stating “Running on local URL: http://XXX.X.X.X:XXXX.”
- Open your web browser and navigate to the URL displayed in the message to access the Web UI.
On Mac (M1/M2):
- Homebrew is a package manager essential for installing the required software packages. To check if you already have it, open Terminal (use Command + Spacebar and search for “Terminal”) and type brew -v. If it’s installed, you’ll see the version number. If not, you’ll get a “command not found: brew” message.
- If you don’t have Homebrew, visit Homebrew’s website and follow the installation instructions. You’ll need to copy the command under “Install Homebrew” and paste it into the Terminal. Enter your computer password if prompted.
- Keep the Terminal open and follow the “Next steps” instructions to add Homebrew to your PATH.
- In the Terminal, run the following command to install the necessary packages:
brew install cmake protobuf rust [email protected] git wget
Wait for the download to complete. - Use the Terminal to navigate to the folder where you want to place the Web UI. For example, to place it on the desktop, type cd desktop. The cd command means “change directory,” and you can use ls to list all folders in your current directory.
- Once you’re in the desired folder, type:
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui
This will create a new folder named “stable-diffusion-webui” containing the Web UI. - In the Terminal, move into this new folder by typing:
cd stable-diffusion-webui - To start the Web UI, use the following command:
./webui.sh
The first time you run this, it may take some time to install all the required packages and download the Stable Diffusion v1.5 model. - Once you see the message “Running on a local URL: http://XXX.X.X.X:XXXX,” you’re all set. Open your web browser and navigate to the displayed URL.
By following these steps, you’ll be well on your way to exploring the creative possibilities that Waifu Diffusion offers.
Step 4: Organizing the downloaded files
Once the download is complete, move both the model and WebUI files to the folder named ‘stable-diffusion-webui/models/Stable-diffusion.’
This folder should be within the main directory of the WebUI you downloaded earlier. As you’ll be storing all future models here, it’s advisable to give the model a descriptive name for easy identification later on.
Step 5: Launching the Web UI
The process to start the Web UI varies based on your operating system:
- For Windows users: Double-click on ‘webui-user.bat’ to initiate the interface.
- For Mac users: Open the terminal and run the command ‘./webui.sh.’
You’ll know the Web UI is up and running when you see the message “Running on local URL: http://127.0.0.1:7860” displayed. Open your web browser and navigate to this URL. At the top of the interface, you’ll find a toggle labeled “Stable Diffusion Checkpoint.” Switch this to the Waifu Diffusion model you’ve just downloaded, which should be named ‘wd-1-4-anime_e2.ckpt.’
By following these steps, you’ll have successfully set up Waifu Diffusion and can start exploring its capabilities.
How to write Waifu Diffusion prompts like a pro?
To generate high-quality outputs, initiate your prompts with terms like “masterpiece” or “hyper quality.” This will guide the AI in generating top-notch results that meet your expectations.
On the flip side, if you aim to pinpoint possible shortcomings or constraints in the model, initiate your prompts with terms that indicate imperfections. Such terms might encompass “poor resolution,” “flawed anatomy,” “imperfect hands,” “text overlay,” “glitches,” “absent fingers,” “additional digit,” “incomplete digits,” “truncated,” “subpar quality,” “inferior quality,” “average quality,” “compression flaws,” “copyright marks,” “digital watermark,” “user ID,” “out of focus,” or “creator’s tag.” Utilizing these negative prompts will direct the AI to produce results that deliberately feature these issues, enabling you to evaluate the model’s weak points.
Stable Diffusion models that can elevate your generation process
In the seed field, you have two options for controlling the randomness of the generated content. Inputting “-1” will randomize the output, giving you a unique result each time. Alternatively, specifying a particular seed number will ensure that the generated content remains consistent across multiple runs. This is useful if you’re looking to create a series of related images or texts.
Featured image credit: Kerem Gülen/Midjourney
