Stability AIの画像生成AIモデルSDXL 1.0がリリースされました
既存の生成AIが苦手としていた細かい描写が改善されているだけでなく、
画像サイズが1024 x 1024となってます(デフォルト)
今回はSDXL 1.0をgoogle colabで動かしてみたいと思います

この記事でわかること

google colabとDiffusersでSDXLを動かす方法

1.実行環境

Google Colab
Diffusers transformers
base model = stabilityai/stable-diffusion-xl-base-1.0

2.コード

!pip install diffusers --upgrade
!pip install invisible_watermark transformers accelerate safetensors

from diffusers import DiffusionPipeline
import torch

base = DiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16,
    variant="fp16",
    use_safetensors=True
)
base.to("cuda")

refiner = DiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-refiner-1.0",
    text_encoder_2=base.text_encoder_2,
    vae=base.vae,
    torch_dtype=torch.float16,
    use_safetensors=True,
    variant="fp16",
)
refiner.enable_model_cpu_offload()


prompt = "任意のポジティブプロンプト"

n_steps = 50
high_noise_frac = 0.8

image = base(
    prompt=prompt,
    num_inference_steps=n_steps,
    denoising_end=high_noise_frac,
    output_type="latent",
).images

image = refiner(
    prompt=prompt,
    num_inference_steps=n_steps,
    denoising_start=high_noise_frac,
    image=image,
).images[0]

image.save('output_refiner.png')
image

3.生成結果

positiveプロンプト

prompt = "masterpiece, ultra high res, 1girl, (photo realistic:1.2) small_breasts smile detailed beautiful skin (black eyes) cute young, looking_at_viewers windy soft_light from front miko, red hakama, miko_clothes, miko_dress,"

結果

positiveプロンプト

prompt="masterpiece, ultra high res, 1 girl, (photo realistic:1.2),small_breasts,cleavage,smile,detailed beautiful skin, face focus, (black eyes), cute, young, looking at viewers, (scoop neck tee:2.0), ,collarbone, ((black hair)), windy, soft light, hand between legs, from front"

結果