Introduction

Granite-3.0-2B-Instruct is part of the Granite 3.0 family of lightweight, state-of-the-art language models developed by IBM. Designed to support a wide range of enterprise applications, this model is optimized for instruction-following tasks, making it suitable for various use cases in natural language processing. With a parameter count of 2 billion, it strikes a balance between performance and resource efficiency, allowing deployment in constrained environments.

Granite-3.0-2B-Instruct

The Granite-3.0-2B-Instruct SLM is a 2-billion parameter dense model specifically fine-tuned to follow instructions, making it highly suitable for interactive tasks such as customer support, virtual assistance, and question-answering. Built on the base 2B model, Granite-3.0-2B-Instruct SLM has undergone further training to enhance instruction adherence, contextual understanding, and alignment with human values. Key attributes include:

• Parameter Count: 2 billion
• Training Data: 12 trillion tokens from diverse domains, including natural language and code
• Tokenizer: Byte Pair Encoding (BPE) method, shared with StarCoder
• Parallelism: Optimized with 3D parallelism, combining Tensor, Pipeline, and Data Parallelism for efficient large-scale training

Run Granite-3.0 with an API

Running the API with Clarifai's Python SDK

You can run the Granite-3.0 Model API using Clarifai’s Python SDK.

Export your PAT as an environment variable. Then, import and initialize the API Client.

Find your PAT in your security settings.

export CLARIFAI_PAT={your personal access token}

from clarifai.client.model import Model

prompt = "what's the future of AI?"
inference_params = dict(temperature=0.7, max_tokens=200, top_k = 50, top_p= 0.95)

# Model Predict
model_prediction = Model("https://clarifai.com/ibm/granite/models/granite-3_0-2b-instruct").predict_by_bytes(prompt.encode(), input_type="text", inference_params=inference_params)

print(model_prediction.outputs[0].data.text.raw)

You can also run Granite-3.0 API using other Clarifai Client Libraries like Java, cURL, NodeJS, PHP, etc here.

Aliases: Granite-3.0, granite 3.0

Use Cases

The Granite-3.0-2B-Instruct SLM model is well-suited for applications that demand reliable instruction-following and contextual response generation, including:

• Customer Support Automation: Provides accurate responses and tailored assistance in multilingual contexts.
• Enterprise Virtual Assistance: Efficient for internal support, administrative tasks, and data retrieval.
• Code Generation and Debugging: Offers valuable assistance in generating code snippets, debugging, and completing functions.
• Multilingual Applications: Supports multiple languages, broadening its accessibility for global audiences.
• Cybersecurity and Compliance: Handles sensitive information retrieval and response generation with an emphasis on enterprise-specific compliance and security.

Evaluation and Benchmark Results

Granite-3.0-2B-Instruct has been evaluated against several benchmarks to assess its performance across different tasks. The model demonstrates strong capabilities in:

• Human Exams: Achieving competitive scores in benchmarks like MMLU and AGI-Eval.
• Commonsense Reasoning: Performing well on tasks such as WinoGrande and PIQA.
• Instruction Following: Excelling in tasks that require understanding and executing user instructions.

The model's performance metrics indicate that it consistently reaches state-of-the-art results for its size, making it a reliable choice for enterprise applications.

Dataset

The training of Granite-3.0-2B-Instruct involved a diverse dataset comprising 12 trillion tokens. The data curation process was designed to ensure high quality and relevance, with a focus on governance and compliance. 

Granite-3.0-2B-Instruct SLM is trained on a carefully curated dataset encompassing:

• Unstructured Natural Language and Code Data: Sourced from the Web, comprising text and code examples.
• Synthetic Datasets: Generated by IBM to enhance training diversity and data quality.
• Publicly Available High-Quality Datasets: Only datasets with permissive licenses were included.

Advantages

• Lightweight and Efficient: With only 2 billion parameters, the model is suitable for deployment in environments with limited computational resources.
• Strong Instruction Following: Specifically fine-tuned to understand and execute user instructions effectively.
• Enterprise-Ready: Model curation and training processes are GRC-compliant, emphasizing governance and data quality.
• Versatile Applications: Applicable across multiple domains, including customer support, content generation, and programming assistance.
• Open Access: Released under an Apache 2.0 license, allowing for both research and commercial use.

Limitations

• Context Length: As a smaller model, Granite-3.0-2B-Instruct may have limitations in handling very long context inputs compared to larger models.
• Domain-Specific Knowledge: While the model performs well across general tasks, it may not have specialized knowledge in niche domains without further fine-tuning.
• Potential Biases: Like all language models, it may reflect biases present in the training data, necessitating careful monitoring and evaluation in sensitive applications.