azimuthalobserver
Original Project found here: https://huggingface.co/Djrango/Qwen2vl-Flux
Qwen2vl-Flux is a state-of-the-art multimodal image generation model that enhances FLUX with Qwen2VL's vision-language understanding capabilities. This model excels at generating high-quality images based on both text prompts and visual references, offering superior multimodal understanding and control.
ComfyUI currently doesn't support and there is no available nodes to load the CLIP+LLM portion into it
This is just for reviewing/testing the finetuned trained part of the Flux model
CFG set to 1 on KSampler
Rendered an image in 150s using 8GB GPU @ 512px / 10 steps using the bf16 model
-
This model comes will be available in 3 formats named after the folder it should be in
diffusion_models - This one is in diffusers format, it is just the merged safetensors file from HuggingFace page
checkpoints - This one has been converted to Flux Transformers format and prefix for stable_diffusion compatibility, does not include CLIP and VAE
unet - I will provide the q4_0 and q8 variants, make a comment if you'd like to see any other quants
What is Qwen 2vl Flux?
Qwen 2vl Flux is a highly specialized Image generation AI Model of type Safetensors / Checkpoint AI Model created by AI community user azimuthalobserver. Derived from the powerful Stable Diffusion (Flux.1 D) model, Qwen 2vl Flux has undergone an extensive fine-tuning process, leveraging the power of a dataset consisting of images generated by other AI models or user-contributed data. This fine-tuning process ensures that Qwen 2vl Flux is capable of generating images that are highly relevant to the specific use-cases it was designed for, such as base model.
With a rating of 0 and over 0 ratings, Qwen 2vl Flux is a popular choice among users for generating high-quality images from text prompts.
Can I download Qwen 2vl Flux?
Yes! You can download the latest version of Qwen 2vl Flux from here.
How to use Qwen 2vl Flux?
To use Qwen 2vl Flux, download the model checkpoint file and set up an UI for running Stable Diffusion models (for example, AUTOMATIC1111). Then, provide the model with a detailed text prompt to generate an image. Experiment with different prompts and settings to achieve the desired results. If this sounds a bit complicated, check out our initial guide to Stable Diffusion – it might be of help. And if you really want to dive deep into AI image generation and understand how set up AUTOMATIC1111 to use Safetensors / Checkpoint AI Models like Qwen 2vl Flux, check out our crash course in AI image generation.
Popularity
Info
Latest version (diffusion_models bf16): 1 File
About this version: diffusion_models bf16
This model should go in the diffusion_models folder
Use the Load Diffusion Models node
4 Versions
Go ahead and upload yours!
