Note

• Model Weight
• Framework: SgLang

Introduction

Llama-3.3-70B-Instruct is a multilingual large language model (LLM) developed by Meta. It is a pretrained and instruction-tuned generative model optimized for multilingual dialogue and text generation. The model achieves state-of-the-art performance across multiple industry benchmarks, surpassing many open-source and proprietary chat models.

Llama-3.3-70B-Instruct Model Details

• Model Developer: Meta
• Architecture: Auto-regressive transformer model with supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF)
• Parameters: 70 billion
• Input Modality: Multilingual text
• Output Modality: Multilingual text and code
• Context Length: 128k tokens
• Grouped-Query Attention (GQA): Yes
• Pretraining Data Size: 15+ trillion tokens
• Knowledge Cutoff: December 2023
• Release Date: December 6, 2024
• License: Llama 3.3 Community License Agreement
• Supported Languages: English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai

The model is a static release, trained on an offline dataset. Future iterations will integrate improved safety mechanisms based on community feedback.

Run Llama 3.3 with an API

Running the API with Clarifai's Python SDK

You can run the Llama 3.3 Model API using Clarifai’s Python SDK.

Export your PAT as an environment variable. Then, import and initialize the API Client.

Find your PAT in your security settings.

export CLARIFAI_PAT={your personal access token}

from clarifai.client.model import Model

prompt = "what's the future of AI?"
inference_params = dict(temperature=0.7, max_tokens=200, top_k = 50, top_p= 0.95)

# Model Predict
model_prediction = Model("https://clarifai.com/meta/Llama-3/models/llama-3_3-70b-instruct").predict_by_bytes(prompt.encode(), input_type="text", inference_params=inference_params)

print(model_prediction.outputs[0].data.text.raw)

Use Cases

Llama-3.3-70B-Instruct is designed for both commercial and research applications. Some primary use cases include:

• Conversational AI: Enhanced chatbot interactions across multiple languages
• Content Generation: Generating high-quality text for various domains
• Code Generation: Supporting developers in writing and debugging code
• Multilingual Assistance: Providing language-specific responses for different regions
• Synthetic Data Generation: Facilitating model distillation and fine-tuning
• Knowledge-based Question Answering: Answering domain-specific and general knowledge questions

Out-of-Scope Uses

• Applications violating laws, trade compliance regulations, or the Acceptable Use Policy
• Deployment in unsupported languages without additional fine-tuning

Evaluation and Benchmark Results

Llama-3.3-70B-Instruct demonstrates significant improvements in key benchmarks:

Dataset

Llama-3.3-70B-Instruct was trained on a new mix of publicly available online data.

• Pretraining Data: ~15 trillion tokens from publicly available sources
• Fine-tuning Data: Over 25 million synthetically generated instruction examples
• Data Freshness: Training data cutoff in December 2023

Advantages

• State-of-the-art performance on multilingual benchmarks
• Extended context length of 128k tokens for improved long-form reasoning
• Advanced instruction tuning using RLHF for better alignment with human intent
• Improved multilingual capabilities in 8 languages
• Optimized for dialogue and task-specific prompting
• Efficient inference with GQA for scalable deployments

Limitations

• Limited to 8 officially supported languages, though it may generate text in other languages with varying quality
• Potential for hallucination, especially on topics beyond its training data
• Not designed for real-time updating, as it is a static model with a fixed knowledge cutoff
• Requires external safeguards when integrated into production systems to mitigate risks
• Biases in training data may lead to unintended outputs, requiring careful evaluation before deployment