This model has 20 free requests then special pricing applied. Find out more

Introduction

The Dolly v2 model is a language model developed by Databricks, built upon the EleutherAI pythia model family. It is designed to follow instructions provided in natural language and generate responses accordingly. Unlike its predecessor, Dolly 2.0 is an open-source model, fine-tuned on a high-quality human-generated instruction dataset called "databricks-dolly-15k". The primary objective of this model is to exhibit human-like interactivity while adhering to provided instructions.

Dolly-v2-12b Model

• Model Type: Large Language Model (LLM)
• Parameter Count: 12 billion
• Architecture: Derived from EleutherAI's Pythia-12b
• Training Data: Fine-tuned on databricks-dolly-15k dataset
• Intended Use: Various applications including closed question-answering, summarization, and generation.

Use Cases

Dolly-v2-12b can be employed for a wide range of language-related applications, including but not limited to:

• Closed question-answering
• Summarization
• Content generation
• Information extraction
• Language-driven brainstorming

Dataset Information

The model has been fine-tuned using the "databricks-dolly-15k" dataset, which contains 15,000 high-quality human-generated prompt/response pairs. This dataset is the result of a contest among more than 5,000 Databricks employees, who contributed to tasks like open and closed question-answering, information extraction, summarization, and more. This dataset is licensed under the Creative Commons Attribution-ShareAlike 3.0 Unported License, allowing unrestricted use, modification, and commercial application.

Evaluation

While Dolly-v2-12b is not considered a state-of-the-art model, it performs better than certain other models such as EleutherAI/gpt-neox-20b and EleutherAI/pythia-6.9b. However, it might underperform its predecessor, Dolly-v1-6b, in certain evaluation benchmarks. The model's performance could be influenced by the composition and size of the fine-tuning datasets. 

Advantages

• Open-source: The model, training code, dataset, and weights are all open-source and suitable for commercial use.
• Human-generated dataset: Dolly-v2-12b is fine-tuned on a high-quality human-generated instruction dataset, making it tailored for instruction-following tasks.
• Customizability: Organizations can customize and utilize Dolly-v2-12b without the need for API access or sharing data with third parties.

Limitations:

• Performance: The model may not perform as well as state-of-the-art models on certain benchmarks or tasks.
• Complex prompts: Dolly-v2-12b might struggle with syntactically complex prompts, mathematical operations, and open-ended questions.
• Not competitive with newer models: It might not perform competitively against more modern model architectures with larger pretraining datasets.
• Lack of certain capabilities: Some capabilities present in the original model might be missing in Dolly-v2-12b, such as well-formatted letter writing.