Llama 3.1-8b-Instruct is a multilingual, highly capable llm optimized for extended context, instruction-following, and advanced applications

Model is trained and ready for deployment

llama-3_1-8b-instruct

The maximum number of tokens to generate. Shorter token lengths will provide faster performance.

A decimal number that determines the degree of randomness in the response

An alternative to sampling with temperature, where the model considers the results of the tokens with top_p probability mass.

The top-k parameter limits the model's predictions to the top k most probable tokens at each step of generation.

num_beams parameter is integral to a method called beam search, which impacts the quality and diversity of generated text

Setting do_sample to true increases creativity in the model's outputs, whereas false leads to more consistent text

setting this to true, returns input prompt along with generated output

Template for formatting the prompt. Can be an arbitrary string, but must contain the substring `{prompt}`.

A system prompt sets the behavior and context for an AI assistant in a conversation, such as modifying its personality.