Company
🔥 Just Launched
Introducing the AI Playground — Your LLM Battleground to Test Powerful AI Models!
Try now
Contact us
Contact us
Join the Discord
Join the Discord
September 12, 2023

Run Stable Diffusion XL with an API

Clarifai Computer Vision By Sumanth P

Table of Contents:
 

Run Stable Diffusion XL 1.0 with an API-1

Stable Diffusion XL 1.0 is the latest state-of-the-art latent diffusion model from Stability AI for high-resolution image synthesis. SDXL is open-source, designed to improve the visual quality of generated images while maintaining transparency and reproducibility.

You can now try out Stable Diffusion XL 1.0 in Clarifai Platform and access it through the API.

Table of Contents

  1. Introduction

  2. Try out Stable Diffusion XL 1.0 in Clarifai Platform

  3. Running Stable Diffusion XL 1.0 with Python

  4. Best Usecases

  5. Evaluation

  6. Advantages

Introduction:

Stable Diffusion XL 1.0 is an image generation model that excels in producing highly detailed and photorealistic 1024x1024 px image compared to its previous versions, Stable Diffusion 2.1 and Stable Diffusion 1.5.

It can generate realistic faces, legible text within images, and better overall image composition. SDXL achieves these results using shorter and simpler prompts while still offering features like image-to-image prompting, inpainting, and outpainting. 

Stable Diffusion XL 1.0 is an enhanced version of the Stable Diffusion model, employing a three times larger UNet backbone to capture more detailed features and produce superior images. To enhance the image quality and diversity, SDXL incorporates innovative conditioning schemes, including multi-scale conditioning, cross-modal attention, and multi-aspect ratio training. These schemes enable SDXL to generate images that closely match the input textual descriptions while covering a wide range of visual styles and variations.

Furthermore, SDXL utilizes a separate refinement model that employs a noising-denoising process on the latents produced by the model. This refinement step helps eliminate artifacts and further improves the overall visual fidelity of the generated images.

Running Stable Diffusion XL 1.0 model with Python

You can run Stable Diffusion XL 1.0 Model using the Clarifai's Python client.

Check out the Code Below:

######################################################################################################
# In this section, we set the user authentication, user and app ID, model details, and the URL of
# the text we want as an input. Change these strings to run your own example.
######################################################################################################
# Your PAT (Personal Access Token) can be found in the portal under Authentification
PAT = ''
# Specify the correct user_id/app_id pairings
# Since you're making inferences outside your app's scope
USER_ID = 'stability-ai'
APP_ID = 'stable-diffusion-2'
# Change these to whatever model and text URL you want to use
MODEL_ID = 'stable-diffusion-xl'
MODEL_VERSION_ID = '0c919cc1edfc455dbc96207753f178d7'
RAW_TEXT = 'I love your product very much'
# To use a hosted text file, assign the url variable
# TEXT_FILE_URL = 'https://samples.clarifai.com/negative_sentence_12.txt'
# Or, to use a local text file, assign the url variable
# TEXT_FILE_LOCATION = 'YOUR_TEXT_FILE_LOCATION_HERE'
############################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
############################################################################
from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2
channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)
metadata = (('authorization', 'Key ' + PAT),)
userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)
# To use a local text file, uncomment the following lines
# with open(TEXT_FILE_LOCATION, "rb") as f:
# file_bytes = f.read()
post_model_outputs_response = stub.PostModelOutputs(
service_pb2.PostModelOutputsRequest(
user_app_id=userDataObject, # The userDataObject is created in the overview and is required when using a PAT
model_id=MODEL_ID,
version_id=MODEL_VERSION_ID, # This is optional. Defaults to the latest model version
inputs=[
resources_pb2.Input(
data=resources_pb2.Data(
text=resources_pb2.Text(
raw=RAW_TEXT
# url=TEXT_FILE_URL
# raw=file_bytes
)
)
)
]
),
metadata=metadata
)
if post_model_outputs_response.status.code != status_code_pb2.SUCCESS:
print(post_model_outputs_response.status)
raise Exception("Post model outputs failed, status: " + post_model_outputs_response.status.description)
# Since we have one input, one output will exist here
output = post_model_outputs_response.outputs[0].data.image.base64
print(output)
view raw sdxl.py hosted with ❤ by GitHub

You can also run Stable Diffusion XL 1.0 Model using other Clarifai Client Libraries like Javascript, Java, cURL, NodeJS, PHP, etc here

Model Demo in the Clarifai Platform:

Try out the Stable Diffusion XL 1.0 model here: clarifai.com/stability-ai/stable-diffusion-2/models/stable-diffusion-xl

SDXL

Best Use Cases

SDXL can be used for various applications, including but not limited to: 

  • Text-to-image synthesis 
  • Image editing and manipulation 
  • Data augmentation for computer vision tasks 
  • Artistic image creation 

Evaluation

SDXL was evaluated on several datasets, including ImageNet, COCO, and LSUN. They show that SDXL achieves competitive performance with state-of-the-art image generation models, including BigGAN and StyleGAN2. They also provide ablation studies to analyze the contribution of different components of the model to its performance.

Performance of the SDXL model was evaluated  using several standard image quality metrics, including Fréchet Inception Distance (FID), Inception Score (IS), and Learned Perceptual Image Patch Similarity (LPIPS).

  • FID measures the distance between the distributions of real and generated images in the feature space of a pre-trained Inception network.
  • IS measures the diversity and quality of the generated images based on the output of the same network.
  • LPIPS measures the perceptual similarity between the generated and real images based on the output of a pre-trained VGG network. 

Advantages

  • Improved Text Generation: SDXL can generate more readable and contextually relevant text within images, which sets it apart from previous AI image generation models.
  • Better Human Anatomy: The model exhibits fewer issues with human anatomy, resulting in more accurate and realistic representations of people in generated images.
  • Diverse Artistic Styles: SDXL offers a wide range of artistic styles, allowing users to experiment and customize image outputs according to their preferences and requirements.
  • Short Prompt Understanding: SDXL understands and responds well to shorter prompts, streamlining the content generation process and saving time for users.
  • State-of-the-art performance: SDXL achieves state-of-the-art performance on several benchmark datasets, including ImageNet, COCO, and LSUN. 

Keep up to speed with AI

  • Follow us on X to get the latest from the LLMs

  • Join us in our Slack Community to talk LLMs

Previous Return to Blog Menu Next