Introduction

The Qwen1.5-32B model represents an exciting advancement in the Qwen1.5 language model series. The Qwen1.5-32B offers a significant reduction in memory usage and inference speed while still maintaining a high level of accuracy and capability. This model, featuring approximately 32 billion parameters, has been designed to be a "sweet spot" solution, particularly attractive for real-world applications where resource constraints are a consideration.

Qwen1.5-32B-Chat

The Qwen1.5-32B-Chat variant has been specifically enhanced for conversational AI, leveraging advancements in Reinforcement Learning from Human Feedback (RLHF) to offer improved dialogue interactions. The model achieves significant scores on established benchmarks such as MT-Bench and Alpaca-Eval 2.0, showing its effectiveness as a conversational agent.

Prompt Format

This format must be strictly respected for Chat Intruction Model variant, otherwise the model will generate sub-optimal outputs.

The template used to build a prompt for the Instruct model is defined as follows:

<|im_start|>system
{system prompt}<|im_end|>
<|im_start|>user
{prompt} <|im_end|>
<|im_start|>assistant

Run Qwen1.5 with an API

 Running the API with Clarifai's Python SDK

You can run the Qwen1.5 Model API using Clarifai’s Python SDK.

Export your PAT as an environment variable. Then, import and initialize the API Client.

Find your PAT in your security settings.

export CLARIFAI_PAT={your personal access token}

from clarifai.client.model import Model

prompt = 'What’s the future of AI?'

prompt_template = f"
<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
{prompt} <|im_end|>
<|im_start|>assistant
"

inference_params = dict(temperature=0.2, max_tokens=100, top_p=0.95, tok_k = 40)

# Model Predict
model_prediction = Model("https://clarifai.com/qwen/qwenLM/models/qwen1_5-32B-chat").predict_by_bytes(prompt.encode(), input_type="text", inference_params=inference_params)

print(model_prediction.outputs[0].data.text.raw)

You can also run Qwen1.5 API using other Clarifai Client Libraries like Java, cURL, NodeJS, PHP, etc here.

Aliases: qwen, qwen1.5, Qwen1.5

Use Cases

Qwen1.5-32B and its chat-specific variant are well-suited for a variety of applications, including:

• Customer service automation
• Personal assistants
• Multilingual translation services
• Educational bots
• Interactive entertainment systems.

Evaluation

The Qwen1.5-32B model has been rigorously evaluated across a variety of benchmarks to assess its performance in both general language understanding and specialized tasks. Below, we detail the performance of Qwen1.5-32B and compare it with other similar or larger models to provide a comprehensive overview of its capabilities.

General Language Understanding and Specialized Tasks

The following table showcases the performance of Qwen1.5-32B and other models across multiple benchmarks:

Model	MMLU (%)	C-Eval (%)	GSM8K (%)	MATH (%)	HumanEval (%)	MBPP (%)	BBH (%)	CMMLU (%)
Llama2-34B	62.6	-	42.2	6.2	22.6	33.0	44.1	-
Yi-34B	76.3	81.4	67.2	14.4	23.2	41.0	54.3	83.7
Mixtral-8x7B	70.6	-	74.4	28.4	40.2	60.7	-	-
Qwen1.5-72B	77.5	84.1	79.5	34.1	41.5	53.4	65.5	83.5
Qwen1.5-32B	73.4	83.5	77.4	36.1	37.2	49.4	66.8	82.3

Conversational Model Performance

The performance in conversational AI contexts has also been measured using MT-Bench and Alpaca-Eval 2.0. The results for these evaluations are detailed below:

Models	MT-Bench	AlpacaEval 2.0	Avg. Score	LC Win Rate (%)
Qwen1.5-72B-Chat	8.61	-	-	36.60
Qwen1.5-32B-Chat	8.30	-	-	27.49

Multilingual Capabilities

The Qwen1.5-32B model's performance across different languages is crucial for applications requiring multilingual support. Below are the results for a selection of language tasks:

Models	Exams (%)	Understanding (%)	Math (%)	Translation (%)	Average (%)
Mixtral-8x7B	56.08	70.70	45.00	29.78	50.39
Qwen1.5-72B	66.35	78.16	61.67	35.57	60.44
Qwen1.5-32B	61.57	76.48	56.13	33.46	56.91

Dataset

Qwen1.5-32B is trained on a diverse and expansive dataset compiled from various sources, including web text, books, and more specialized corpora covering scientific, technical, and multilingual content. This extensive training set ensures broad general knowledge and specialized understanding.

Advantages

• Strong Performance: Rivals larger models in specific benchmark tasks, providing a practical alternative without significant trade-offs in output quality.
• Versatility: Suitable for a broad range of applications from simple task automation to complex multilingual conversations.

Disclaimer

Please be advised that this model utilizes wrapped Artificial Intelligence (AI) provided by TogetherAI (the "Vendor"). These AI models may collect, process, and store data as part of their operations. By using our website and accessing these AI models, you hereby consent to the data practices of the Vendor. We do not have control over the data collection, processing, and storage practices of the Vendor. Therefore, we cannot be held responsible or liable for any data handling practices, data loss, or breaches that may occur. It is your responsibility to review the privacy policies and terms of service of the Vendor to understand their data practices. You can access the Vendor's privacy policy and terms of service at https://www.togetherai.com/legal/privacy-policy.

We disclaim all liability with respect to the actions or omissions of the Vendor, and we encourage you to exercise caution and to ensure that you are comfortable with these practices before utilizing the AI models hosted on our site.