Note

• Datatype: 4-bit AWQ quantise format
• Framework: vLLM LLM serving framework
• Model Source

Introduction

Gemma-1.1-7b-it is a lightweight and improved version of Gemma-7b-it. It belongs to the Gemma family of models, which are built using similar research and technology as the Gemini models. These models are designed for various text generation tasks, including question answering, summarization, and reasoning.

Gemma-1.1-7b-it

Gemma-1.1-7b-it is an instruction-tuned variant with 7 billion parameters, optimized for higher performance across multiple tasks without compromising on speed and resource efficiency. This model has been enhanced specifically to address issues in multi-turn conversations and incorporates novel Reinforcement Learning from Human Feedback (RLHF) techniques, which have shown substantial improvements in quality, coding capabilities, factuality, and instruction adherence..

Run Gemma with an API

Running the API with Clarifai's Python SDK

You can run the Gemma-7b Model API using Clarifai’s Python SDK.

Export your PAT as an environment variable. Then, import and initialize the API Client.

Find your PAT in your security settings.

export CLARIFAI_PAT={your personal access token}

from clarifai.client.model import Model

prompt = '''<bos><start_of_turn>user
Write a hello world program<end_of_turn>
<start_of_turn>model
'''

inference_params = dict(temperature=0.7, max_tokens=200, top_k = 50, top_p= 0.95)

# Model Predict
model_prediction = Model("https://clarifai.com/gcp/generate/models/gemma-1_1-7b-it").predict_by_bytes(prompt.encode(), input_type="text", inference_params=inference_params)

print(model_prediction.outputs[0].data.text.raw)

You can also run Gemma-7b API using other Clarifai Client Libraries like Java, cURL, NodeJS, PHP, etc here.

Aliases: Gemma-7b, gemma, gemma 7b,

Prompt Format

This format must be strictly respected, otherwise the model will generate sub-optimal outputs.

The template used to build a prompt for the Instruct model is defined as follows:

<bos><start_of_turn>user
{prompt}<end_of_turn>
<start_of_turn>model

Use Cases

The Gemma-1.1-7b-it is ideal for a variety of applications:

• Question Answering: Provides accurate answers to user queries.
• Summarization: Automatically generates concise summaries from longer texts.
• Code Generation: Offers assistance in writing and understanding code by understanding syntax and patterns.
• Mathematical Reasoning: Solves mathematical problems and provides explanations.
• Multi-turn Conversation: Engages in dialogues requiring context retention over multiple turns.

Evaluation

Benchmark Results: The pre-trained Gemma-1.1-7b-it model was evaluated against several benchmarks using different datasets and metrics. Here are the benchmark results:

Dataset

Training Dataset: The Gemma-1.1-7b-it model was trained on a dataset of text data totaling 6 trillion tokens. The training data consisted of the following key components:

• Web Documents: A diverse collection of web text primarily in English, exposing the model to various linguistic styles, topics, and vocabulary.
• Code: Text data related to programming languages to help the model learn syntax and patterns, enhancing its ability to generate and understand code.
• Mathematics: Text data containing mathematical content to facilitate learning logical reasoning, symbolic representation, and addressing mathematical queries.

Data Preprocessing: Several data cleaning and filtering methods were applied to the training data, including:

• Rigorous filtering for CSAM (Child Sexual Abuse Material) at multiple stages to exclude harmful and illegal content.
• Automated techniques to filter sensitive personal information and other sensitive data from training sets.
• Additional filtering based on content quality and safety in accordance with policies.

Advantages

• Lightweight and decoder-only, suitable for deployment in resource-constrained environments.
• Trained using a novel RLHF method, resulting in improvements in various quality aspects.
• Versatile, capable of handling various text generation tasks including question answering, summarization, and reasoning.
• Safe and reliable, with rigorous filtering for harmful and sensitive content during data preprocessing.

Limitations

• Limited to English-language text data.
• The model's performance may vary based on the specific task and dataset.
• Despite filtering efforts, there may still be instances where inappropriate or sensitive content is generated.
• The model's size may limit its applicability to certain tasks requiring larger models with more parameters.