Introduction

Mixtral 8x7B is a high-quality Sparse Mixture-of-Experts (SMoE) model developed by Mistral AI. Mixtral 8x7B represents a significant advancement in open models, offering novel capabilities and improved cost-performance trade-offs.

Mixtral 8x7B Model

Mixtral is a decoder-only model utilizing a sparse mixture-of-experts network. With a context handling capability of 32k tokens, it supports multiple languages, including English, French, Italian, German, and Spanish. The model excels in code generation and can be fine-tuned for instruction-following tasks, achieving an impressive score of 8.3 on MT-Bench.

Key Features:

• Sparse Mixture-of-Experts architecture
• Efficiently outperforms Llama 2 70B and GPT3.5 on various benchmarks
• Gracefully handles a context of 32k tokens
• Multilingual support: English, French, Italian, German, and Spanish
• Strong performance in code generation

Run Mixtral 8x7B with an API

Running the API with Clarifai's Python SDK

You can run the Mixtral 8x7B Model API using Clarifai’s Python SDK.

Export your PAT as an environment variable. Then, import and initialize the API Client.

Find your PAT in your security settings.

export CLARIFAI_PAT={your personal access token}

from clarifai.client.model import Model

prompt = "<s> [INST] What’s the future of AI? [/INST]"

inference_params = dict(temperature=0.7, max_tokens=200, top_k = 50, top_p= 0.95)

# Model Predict
model_prediction = Model("https://clarifai.com/mistralai/completion/models/mixtral-8x7B-Instruct-v0_1").predict_by_bytes(prompt.encode(), input_type="text", inference_params=inference_params)

print(model_prediction.outputs[0].data.text.raw)

You can also run Mixtral 8x7B API using other Clarifai Client Libraries like Java, cURL, NodeJS, PHP, etc here.

Aliases: Mixtral, mixtral-8*7b, mixtral-8x7b,

Prompt Format

This format must be strictly respected, otherwise the model will generate sub-optimal outputs.

The template used to build a prompt for the Instruct model is defined as follows:

<s> [INST] Instruction [/INST] Model answer</s> [INST] Follow-up instruction [/INST]

Note that <s> and </s> are special tokens for beginning of string (BOS) and end of string (EOS) while [INST] and [/INST] are regular strings.

Use Cases

Mixtral 8x7B excels in various applications, including but not limited to:

• Natural Language Processing tasks
• Multilingual applications
• Code generation
• Instruction-following models

Evaluation

Mixtral is evaluated against Llama 2 and GPT3.5 on various benchmarks, showcasing its superior performance in terms of quality versus inference budget tradeoff. It excels in TruthfulQA, BBQ, and BOLD benchmarks, demonstrating improved truthfulness and reduced biases compared to Llama 2.

Detailed Benchmarks**:**

• TruthfulQA: 73.9% accuracy
• MT-Bench: The instruction-following model that achieves a score of 8.3 on MT-Bench.
• BBQ Benchmark: Less bias compared to Llama 2
• BOLD Benchmark: More positive sentiments than Llama 2

Advantages

• Open-weight model with a permissive Apache 2.0 license
• 6x faster inference compared to Llama 2
• Best overall model in terms of cost/performance trade-offs
• Matches or outperforms GPT3.5 on standard benchmarks

Disclaimer

Please be advised that this model utilizes wrapped Artificial Intelligence (AI) provided by TogetherAI (the "Vendor"). These AI models may collect, process, and store data as part of their operations. By using our website and accessing these AI models, you hereby consent to the data practices of the Vendor. We do not have control over the data collection, processing, and storage practices of the Vendor. Therefore, we cannot be held responsible or liable for any data handling practices, data loss, or breaches that may occur. It is your responsibility to review the privacy policies and terms of service of the Vendor to understand their data practices. You can access the Vendor's privacy policy and terms of service at https://www.togetherai.com/legal/privacy-policy.

We disclaim all liability with respect to the actions or omissions of the Vendor, and we encourage you to exercise caution and to ensure that you are comfortable with these practices before utilizing the AI models hosted on our site.