-
Careers
-
AI Compute Orchestration
Create and control your AI workloads on any compute infrastructure
🔥 Just Launched
Introducing the AI Playground — Your LLM Battleground to Test Powerful AI Models!
Overview of Workflows
The ability to process and understand different types of data is very useful. Think about it: what if you could take a picture of a sign and immediately have its text translated into another language? Or hear a voice recording and determine if the message is positive or negative? Sure, you can train complex models to do tasks like this, but a simpler way is just to chain models together where the output of one model is the input to the next. This is where Clarifai Community and Mesh, our workflow product, comes into play. It allows users to combine different tools, like image recognition and text translation, into one seamless multimodal system.
By creating these combined workflows, we can make computers more efficient and insightful. Clarifai Mesh offer a versatile framework for constructing your inference pipeline, and equips you with the fundamental components for sophisticated machine learning ensemble modeling and incorporating business logic. Clarifai simplifies the process of integrating diverse models, enabling you to execute intricate data operations and design solutions tailored to your precise business requirements.
One way to create workflows is using Clarifai Community's visual graph editor, however you might want to create them programmatically instead.
Creating a workflow with SDK
The Clarifai Python SDK empowers you to define and construct intricate workflows through a YAML configuration.
Installation
Install Clarifai Python SDK using the code snippet below.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| pip install -U clarifai |
Get started by retrieving the PAT token from the instructions here and setting up the PAT token as an environment variable. Signup here
To walk through the process of creating Workflows with YAML specifications let's consider two Tasks.
Task 1: Using a generative LLM model to perform text classification for Content moderation.
For this task, we would want to assemble the GPT 3.5 Turbo model (Explore Community models here. ) and create a prompt that performs text classification over an input.
The LLM model is a “text-to-text” model type within Clarifai and our current chosen model performs multiple text-based tasks in general. Here, we utilize the LLM to generate text.
To give more context on a prompter A prompt template serves as a pre-configured piece of text used to instruct a text-to-text model. It acts as a structured query or input that guides the model in generating the desired response.
Now, we are going to create a text sentiment classification prompter node,
Here is an example of a YAML specification for the task, stored as “prompter.yml”
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| workflow: | |
| id: wf-prompter-llm | |
| nodes: | |
| - id: prompter | |
| model: | |
| model_id: prompter | |
| model_type_id: prompter | |
| description: 'Prompter Model' | |
| output_info: | |
| params: | |
| prompt_template: 'Classify sentiment between postive and negative for the text {data.text.raw}' | |
| - id: llm | |
| model: | |
| user_id: openai | |
| model_id: GPT-3_5-turbo | |
| app_id: chat-completion | |
| node_inputs: | |
| - node_id: prompter |
Having specified the YAML, we can use the below SDK functionality to use the workflow created in the Clarifai platform.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from clarifai.client.user import User | |
| # Create a new app (replace USER_ID and APP_ID with your own) | |
| app = User(user_id="USER_ID").create_app(app_id="APP_ID") | |
| # Create Prompter+LLM workflow | |
| prompter_llm_workflow = app.create_workflow(config_filepath="configs/prompter_llm.yml") | |
| # Run workflow on a text input | |
| prompter_llm_workflow.predict_by_bytes(b"Good Morning! I think it's going to be a great day!", input_type="text") |
Try experimenting by creating (summarisation, translation, named entity recognition..etc)
Task 2: Face Sentiment Classification
Multi-model workflow that combines face detection and sentiment classification of 7 concepts: anger, disgust, fear, neutral, happiness, sadness, contempt, and
Workflow contains three nodes:
- Visual Detector - To detect faces
- Image Cropper - Crop faces from the image
- Visual Classifier - To classify the sentiment of the face
Here is an example of a YAML specification for the task, stored as “face_sentiment.yml”
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| workflow: | |
| id: Face-Sentiment | |
| nodes: | |
| - id: face-det | |
| model: | |
| model_id: face-detection | |
| model_version_id: 6dc7e46bc9124c5c8824be4822abe105 | |
| - id: image-crop | |
| model: | |
| model_id: margin-110-image-crop | |
| model_type_id: image-crop | |
| description: Custom crop model | |
| output_info: | |
| params: | |
| margin: 1.1 | |
| node_inputs: | |
| - node_id: face-det | |
| - id: face-sentiment | |
| model: | |
| model_id: face-sentiment-recognition | |
| model_version_id: a5d7776f0c064a41b48c3ce039049f65 | |
| node_inputs: | |
| - node_id: margin-110 |
After defining the YAML configuration, we can employ the following SDK features to utilize the workflow established on the Clarifai platform.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Create Face-Sentiment workflow | |
| face_sentiment_workflow = app.create_workflow(config_filepath="configs/face_sentiment.yml") | |
| # Predict workflow on an image input | |
| prediction = face_sentiment_workflow.predict_by_url("https://samples.clarifai.com/face-det.jpg", input_type="image") | |
| # Get workflow results | |
| print(prediction.results[0].outputs[-1].data) |
Jump into the Workflow Create notebook to explore a variety of workflows designed to help you kickstart your projects. These workflows include Audio Sentiment, Vector Search, Language Aware OCR, and Demographics.
Workflow Export
To begin or make rapid adjustments to existing Clarifai community workflows using an initial YAML configuration, the SDK offers an export feature.
An example of this pipeline is provided in the Clarifai/examples library.
What's next?
We are bringing more data utilities for converting annotation formats before uploading or exporting, text splitting, model training and evaluation interfaces, and vector search interfaces.
Also, let us know what functionality you would like to see in the SDK in our discord channel.
For more information on Python SDK, refer to our Docs here and for detailed examples, we constantly strive to add more notebooks here.
