What is Diffusers?
huggingface
/
diffusers
π€ Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.
π€ Diffusers is the go-to library for state-of-the-art pretrained diffusion models for generating images, audio, and even 3D structures of molecules. Whether you're looking for a simple inference solution or training your own diffusion models, π€ Diffusers is a modular toolbox that supports both. Our library is designed with a focus on usability over performance, simple over easy, and customizability over abstractions.
π€ Diffusers offers three core components:
- State-of-the-art diffusion pipelines that can be run in inference with just a few lines of code.
- Interchangeable noise schedulers for different diffusion speeds and output quality.
- Pretrained models that can be used as building blocks, and combined with schedulers, for creating your own end-to-end diffusion systems.
Installation
We recommend installing π€ Diffusers in a virtual environment from PyPI or Conda. For more details about installing PyTorch and Flax, please refer to their official documentation.
PyTorch
With pip (officialβ¦
What is Flux
https://blackforestlabs.ai/announcing-black-forest-labs/
1. Create a virtual env
python3 -m venv fluxtest
source fluxtest/bin/activate
2. Login to Hugging Face via CLI
https://huggingface.co/docs/huggingface_hub/main/en/guides/cli
pip install -U "huggingface_hub[cli]"
huggingface-cli login
3. Install packages
pip install torch==2.3.1
pip install git+https://github.com/huggingface/diffusers.git
pip install transformers==4.43.3 sentencepiece==0.2.0 accelerate==0.33.0 protobuf==5
4. Run a Python script
image.py
import torch
from diffusers import FluxPipeline
import diffusers
_flux_rope = diffusers.models.transformers.transformer_flux.rope
def new_flux_rope(pos: torch.Tensor, dim: int, theta: int) -> torch.Tensor:
assert dim % 2 == 0, "The dimension must be even."
if pos.device.type == "mps":
return _flux_rope(pos.to("cpu"), dim, theta).to(device=pos.device)
else:
return _flux_rope(pos, dim, theta)
diffusers.models.transformers.transformer_flux.rope = new_flux_rope
pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", revision='refs/pr/1', torch_dtype=torch.bfloat16).to("mps")
prompt = "japanese girl, photo-realistic"
out = pipe(
prompt=prompt,
guidance_scale=0.,
height=1024,
width=1024,
num_inference_steps=4,
max_sequence_length=256,
).images[0]
out.save("image.png")
Finally, run a Python script to generate an image.
python image.py
output
Top comments (5)
I did exactly your code and I came up with an blurry image
what are your Mac specs?
Solve it, downgrade torch to 2.3.1
You can try this blog step by step, it is work for me.
towards-agi.medium.com/how-to-flux...
I have exactly this problem too. How did you solve it?