Note

• Datatype: 4-bit AWQ quantised format
• Model Source
• License: tongyi-qianwen (other)

Introduction

Qwen1.5 is the latest iteration in our Qwen series, arriving just before the Chinese New Year. This version introduces a variety of models, including base and chat models across sizes ranging from 0.5B to 110B parameters, and a MoE (Mixture of Experts) model.

The Qwen1.5-7B-Chat model, a prominent model in this series, brings significant enhancements in alignment with human preferences and expanded multilingual capabilities. It supports a context length of up to 32,768 tokens, ensuring improved performance across a wide range of tasks.

Qwen1.5-7B-Chat LLM

Qwen1.5-7B-Chat is part of the Qwen1.5 series, featuring a chat-oriented model designed to align closely with human preferences. It supports extensive multilingual capabilities and long-context understanding up to 32,768 tokens. It integrates advanced techniques such as Direct Policy Optimization (DPO) and Proximal Policy Optimization (PPO) for enhanced instruction-following capabilities.

Run Qwen1.5 with an API

 Running the API with Clarifai's Python SDK

You can run the Qwen1.5 Model API using Clarifai’s Python SDK.

Export your PAT as an environment variable. Then, import and initialize the API Client.

Find your PAT in your security settings.

export CLARIFAI_PAT={your personal access token}

from clarifai.client.model import Model

prompt = "What’s the future of AI?"

inference_params = dict(temperature=0.2, max_tokens=100, top_p=0.95, tok_k = 40, num_beams=1)

# Model Predict
model_prediction = Model("https://clarifai.com/qwen/qwenLM/models/qwen1_5-7b-chat").predict_by_bytes(prompt.encode(), input_type="text", inference_params=inference_params)

print(model_prediction.outputs[0].data.text.raw)

You can also run Qwen1.5 API using other Clarifai Client Libraries like Java, cURL, NodeJS, PHP, etc here.

Aliases: qwen, qwen1.5, Qwen1.5

Use Cases

• Conversational AI: Qwen1.5-7B-Chat excels in generating human-like responses, making it suitable for chatbots and virtual assistants.
• Multilingual Applications: With its robust multilingual capabilities, this model is ideal for tasks involving translation, language understanding, and cross-lingual dialogue systems.
• Retrieval-Augmented Generation (RAG): The model's performance in RAG tasks makes it suitable for applications requiring real-time data retrieval and integration, such as dynamic knowledge bases or interactive systems.
• Coding Assistance: The model's proficiency in coding tasks makes it a useful tool for developers seeking AI assistance in code generation, debugging, and documentation.
• Long-Form Content Generation: With its support for extended context lengths, Qwen1.5-7B-Chat is well-suited for generating detailed reports, articles, and other long-form content.

Evaluation and Benchmark Results

Qwen1.5-7B-Chat has been rigorously evaluated across various benchmarks, demonstrating competitive performance against other leading models:

Dataset

The Qwen1.5-7B-Chat model was trained on a diverse dataset encompassing a wide range of text data, including multilingual corpora, code repositories, and conversational datasets. This comprehensive training data allows the model to generalize well across different tasks and languages.

Advantages

• Multilingual Capabilities: Strong performance across 12 languages, making it versatile for global applications.
• Long Context Support: Handles up to 32,768 tokens, suitable for tasks requiring extensive context.
• Human Preference Alignment: Optimized using techniques like Direct Policy Optimization (DPO) and Proximal Policy Optimization (PPO), ensuring responses align well with human preferences.
• Integration with External Systems: Effective in Retrieval-Augmented Generation and tool-use scenarios, enhancing its utility in real-world applications.

Limitations

• Trailing GPT-4: Despite strong performance, the model still lags behind GPT-4 in several benchmarks, particularly in code interpretation tasks.
• Variance in Evaluation: The model exhibits variance in MT-Bench evaluations, indicating room for improvement in consistency.
• Resource Intensive: The 7B parameter model, while powerful, may require substantial computational resources for deployment and fine-tuning.