The Llama 3 instruction tuned llm are optimized for dialogue use cases and outperform many of the available open source chat llm on common industry benchmarks




Model is trained and ready for deployment

llama-3-8b-instruct-4bit

The maximum number of tokens to generate. Shorter token lengths will provide faster performance.

A decimal number that determines the degree of randomness in the response

An alternative to sampling with temperature, where the model considers the results of the tokens with top_p probability mass.

The top-k parameter limits the model's predictions to the top k most probable tokens at each step of generation.

num_beams parameter is integral to a method called beam search, which impacts the quality and diversity of generated text

Setting do_sample to true increases creativity in the model's outputs, whereas false leads to more consistent text

setting this to true, returns input prompt along with generated output

Template for formatting the prompt. Can be an arbitrary string, but must contain the substring `{prompt}`.

A system prompt sets the behavior and context for an AI assistant in a conversation, such as modifying its personality.

The Llama 3 instruction tuned llm are optimized for dialogue use cases and outperform many of the available open source chat llm on common industry