Note

Please use in accordance with Llama-3's license terms and Acceptable Use Policy.

Overview

Meta developed and released the Meta Llama 3 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. The Llama 3 instruction tuned models are optimized for dialogue use cases and outperform many of the available open source chat models on common industry benchmarks. Further, in developing these models, we took great care to optimize helpfulness and safety.

Model developers Meta

Input Models input text only.

Output Models generate text and code only.

Model Architecture Llama 3 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.

What's new with Llama-3?

Llama 3 brings significant enhancements over Llama 2, including a new tokenizer that increases the vocabulary size to 128,256 tokens (up from 32K tokens). This expanded vocabulary enhances text encoding efficiency, promoting stronger multilingual capabilities.

Moreover, Llama 3 models underwent extensive training on a diverse dataset comprising over 15 trillion tokens, approximately eight times more data than its predecessor. Specifically, Llama 3 Instruct, tailored for dialogue applications, was fine-tuned on a dataset of over 10 million human-annotated samples using a combination of techniques such as supervised fine-tuning, rejection sampling, proximal policy optimization, and direct policy optimization.

Llama-3-70B

Token counts refer to pretraining data only. Both the 8 and 70B versions use Grouped-Query Attention (GQA) for improved inference scalability.

Prompt template

The base models have no prompt format. Like other base models, they can be used to continue an input sequence with a plausible continuation or for zero-shot/few-shot inference. 

Run Llama 3-70B  with an API

Running the API with Clarifai's Python SDK

You can run the Llama 3-70B Model API using Clarifai’s Python SDK.

Export your PAT as an environment variable. Then, import and initialize the API Client.

Find your PAT in your security settings.

export CLARIFAI_PAT={your personal access token}

from clarifai.client.model import Model

prompt = 'What’s the future of AI?'

inference_params = dict(temperature=0.2, max_tokens=100, top_p=0.95, tok_k = 40)

# Model Predict
model_prediction = Model("https://clarifai.com/meta/Llama-3/models/llama-3-70B").predict_by_bytes(prompt.encode(), input_type="text", inference_params=inference_params)

print(model_prediction.outputs[0].data.text.raw)

You can also run Llama 3-70B API using other Clarifai Client Libraries like Java, cURL, NodeJS, PHP, etc here.

Aliases: llama3-70b, llama-3-70, llama 3, etc

Evaluation

The model has been evaluated against a set of 1,800 prompts covering 12 key scenarios, including advice, coding, and reasoning, among others. Human evaluators ranked Llama-3-Instruct favorably against competitors like Claude Sonnet and GPT-3.5, particularly in instruction following, coding tasks, and reasoning accuracy. 

Use Cases

Llama-3 is designed for a variety of applications:

• Advice and Guidance: Offering thoughtful suggestions based on complex user queries.
• Creative Writing: Assisting with the generation of narrative content.
• Code Generation: Writing and debugging software code efficiently.
• Data Extraction and Analysis: Pulling specific information from large datasets.
• Educational Support: Providing explanations and summaries for study materials.
• Customer Support: Automating responses in customer interaction scenarios.

Advantages

• State-of-the-Art Performance: Sets new benchmarks at the 8B and 70B scales for accuracy, efficiency, and adaptability.
• Enhanced Reasoning and Coding Abilities: Specific improvements in logical deduction and programming-related tasks.
• Scalable and Efficient: Benefits from architectural improvements and large-scale training, optimizing both performance and computational resource usage.
• Diverse Application Spectrum: Applicable across different domains and languages, preparing it for future multilingual and multimodal extensions.

Limitations

• Language Performance Variance: While it includes non-English data, performance in languages other than English may not be as robust.
• Continuous Development: As the model is still under refinement, certain functionalities may change, and performance may continue to improve or adjust over time.

Disclaimer

Please be advised that this model utilizes wrapped Artificial Intelligence (AI) provided by TogetherAI (the "Vendor"). These AI models may collect, process, and store data as part of their operations. By using our website and accessing these AI models, you hereby consent to the data practices of the Vendor. We do not have control over the data collection, processing, and storage practices of the Vendor. Therefore, we cannot be held responsible or liable for any data handling practices, data loss, or breaches that may occur. It is your responsibility to review the privacy policies and terms of service of the Vendor to understand their data practices. You can access the Vendor's privacy policy and terms of service at https://www.togetherai.com/legal/privacy-policy.

We disclaim all liability with respect to the actions or omissions of the Vendor, and we encourage you to exercise caution and to ensure that you are comfortable with these practices before utilizing the AI models hosted on our site.