Note

• Datatype: 4-bit bnb nf4 quantised format
• Model Source
• License: Apache-2.0

Introduction

The Qwen2-7B-Instruct model is part of the Qwen2 series, a new generation of large language models developed to achieve state-of-the-art performance across a range of tasks. The Qwen2 series includes both base and instruction-tuned models, with parameters ranging from 0.5 billion to 72 billion. 

The Qwen2-7B-Instruct model, with 7.07 billion parameters, is designed specifically for instruction-following tasks and is fine-tuned to enhance performance in natural language understanding, language generation, coding, mathematics, and multilingual capabilities.

Qwen2-7B-Instruct LLM

Qwen2-7B-Instruct is a powerful instruction-tuned model that has been optimized to handle a wide range of tasks, including complex reasoning, language translation, and code generation. The model supports an extended context length of up to 128,000 tokens, allowing it to process extensive inputs more effectively than most other models of its size.

Model Specifications:

• Parameters: 7.07 Billion
• Non-Embedding Parameters: 5.98 Billion
• GQA (Generalized Question Answering): Supported
• Context Length: Up to 128K tokens
• Languages: Trained on data in English, Chinese, and 27 additional languages

Run Qwen2 with an API

 Running the API with Clarifai's Python SDK

You can run the Qwen2 Model API using Clarifai’s Python SDK.

Export your PAT as an environment variable. Then, import and initialize the API Client.

Find your PAT in your security settings.

export CLARIFAI_PAT={your personal access token}

from clarifai.client.model import Model

prompt = "What’s the future of AI?"

system_prompt = "You are a AI expert."

inference_params = dict(temperature=0.2, max_tokens=100, top_p=0.95, tok_k = 40, num_beams=1, system_prompt=system_prompt)

# Model Predict
model_prediction = Model("https://clarifai.com/qwen/qwenLM/models/qwen2-7b-chat").predict_by_bytes(prompt.encode(), input_type="text", inference_params=inference_params)

print(model_prediction.outputs[0].data.text.raw)

You can also run Qwen2 API using other Clarifai Client Libraries like Java, cURL, NodeJS, PHP, etc here.

Aliases: qwen 2, qwen2, Qwen2

Use Cases

Qwen2-7B-Instruct is suitable for a variety of use cases, including but not limited to:

• Use Cases

• Language Understanding: Qwen2-7B-Instruct excels in tasks requiring comprehension of complex texts, such as reading comprehension and summarization.

• Language Generation: The model can generate coherent and contextually relevant text for applications like creative writing and dialogue systems.

• Multilingual Capability: With training data in multiple languages, the model supports multilingual applications, enhancing its usability in diverse linguistic contexts.

• Coding: Demonstrates improved performance in code generation and understanding, making it useful for programming assistance and code completion tools.

• Mathematics: Enhanced capabilities in solving mathematical problems and performing arithmetic tasks.

• Extended Context Processing: The model's ability to handle up to 128,000 tokens allows for processing extensive inputs, useful for large documents or long conversations.

Evaluation and Benchmark Results

Qwen2-7B-Instruct has demonstrated state-of-the-art performance across various benchmark evaluations, surpassing many open-source models and proving competitive against proprietary models. The model excels particularly in coding, mathematics, and multilingual tasks, with strong results in both English and Chinese benchmarks.

Key Evaluation Results:

Qwen2-7B-Instruct has shown state-of-the-art performance across various benchmarks:

• Language Understanding and Generation: Surpasses many open-source models and is competitive with proprietary models.
• Multilingual Performance: Excels in tasks involving multiple languages.
• Coding and Mathematics: Significant improvements in benchmarks related to coding and mathematics.
• Overall Competitiveness: Outperforms state-of-the-art models of similar or larger sizes, including the previous Qwen1.5 model.

Dataset

The Qwen2-7B-Instruct model was trained on a diverse and extensive dataset that includes multilingual text from various sources. The training data spans across 29 languages, including English, Chinese, and 27 others, covering a wide range of domains and genres. This diversity ensures that the model can handle various linguistic tasks and perform well in multilingual environments.

Advantages

• State-of-the-Art Performance: Qwen2-7B-Instruct surpasses most open-source models and remains competitive against proprietary models.
• Extended Context Length: Supports up to 128K tokens, making it ideal for processing long-form content and complex conversations.
• Multilingual Proficiency: Trained on data in 29 languages, offering robust multilingual capabilities.
• Improved Coding and Mathematics: Significant advancements in coding and mathematical reasoning make it a strong choice for technical applications.
• Open License: Released under the Apache 2.0 license, promoting openness and encouraging commercial use and community collaboration.

Limitations

• Model Size: While powerful, the 7.07 billion parameter size may require substantial computational resources for deployment.
• Data Bias: As with any large language model, Qwen2-7B-Instruct may exhibit biases present in its training data, which could affect the fairness and accuracy of its outputs.
• Language-Specific Performance: Although it performs well in 29 languages, performance may vary depending on the language and specific task.