Note

• Datatype: 4-bit AWQ quantized format
• Modet Source

Instruction format

In order to leverage instruction fine-tuning, your prompt should be surrounded by [INST] and [\INST] tokens. The Prompt should be as following:

<s>[INST] {prompt} [/INST]

Mistral 7B

Mistral 7B is a SOTA language model with a whopping 7.3 billion parameters. It represents a significant leap in natural language understanding and generation. The model is released under the Apache 2.0 license, allowing its unrestricted usage.

• Performance Superiority: Mistral-7B surpasses the performance of Llama2-13B on all benchmark tasks and excels on many benchmarks compared to Llama 34B. It also demonstrates competitive performance with CodeLlama-7B on code-related tasks while maintaining proficiency in English language tasks.
• Versatile Abilities: It excels not only in code-related tasks, approaching CodeLlama 7B performance, but also remains highly proficient in various English language tasks.
• Efficient Inference: Mistral-7B utilizes Grouped-query attention (GQA) to enable faster inference, making it suitable for real-time applications. Additionally, Sliding Window Attention (SWA) is employed to handle longer sequences efficiently and economically.

Mistral-7B-Instruct

Fine-Tuning for Chat

Mistral-7B Instruct demonstrates the model's generalization capabilities through fine-tuning on publicly available instruction datasets. It achieves remarkable performance, outperforming all other 7B models on MT-Bench and competing favorably with 13B chat models.

Run Mistral-7B Instruct with an API

You can run the Mistral-7B Instruct Model using Clarifai’s python SDK.

Check out the Code Below:

Export your PAT as an environment variable. Then, import and initialize the API Client.

export CLARIFAI_PAT={your personal access token}

from clarifai.client.model import Model

prompt = "<s>[INST] Write a tweet on future of AI [/INST]"

# Model Predict
model_prediction = Model("https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct").predict_by_bytes(prompt.encode(), "text")

You can also run Mistral 7B API using other Clarifai Client Libraries like Java, cURL, NodeJS, PHP, etc here.

Evaluation

Performance comparisons between Mistral-7B and different Llama models were conducted to provide insights into its capabilities. The benchmarks cover a wide range of tasks, including:

• Comparative Performance: Mistral-7B significantly outperforms Llama2-13B across a wide range of benchmarks, including commonsense reasoning, world knowledge, reading comprehension, and math-related tasks.
• Equivalent Model Size: In reasoning, comprehension, and STEM reasoning (MMLU), Mistral-7B demonstrates performance equivalent to a Llama 2 model more than three times its size. This indicates memory efficiency and improved throughput.
• Knowledge Benchmarks: Mistral-7B excels in most evaluations, it performs on par with Llama2-13B in knowledge benchmarks, possibly due to its limited parameter count.

Mistral-7B consistently outperforms Llama2-13B on all metrics and is competitive with Llama 34B. Notably, it excels in code and reasoning benchmarks.

Mistral 7B performs equivalently to a Llama 2 that would be more than 3x its size. This is as much saved in memory and gained in throughput.