Company
🔥 Just Launched
Introducing the AI Playground — Your LLM Battleground to Test Powerful AI Models!
Try now
Contact us
Contact us
Join the Discord
Join the Discord
November 23, 2023

Run Claude 2.1 with an API

Clarifai Computer Vision By Luv Bansal

Table of Contents:

Run Claude 2.1 with an API

Introduction

Claude 2.1 is the latest iteration of the anthropic language model. This model introduces significant enhancements over its predecessor, Claude 2.0, with a focus on expanding the context window and reducing hallucination rates.

Claude 2.1 introduces several advancements to address the evolving needs of enterprises. Key features include:

200K Context Window

Claude 2.1 provides 200K token context window, doubling the amount of information it can process. This enables users to interact with longer documents, including technical documentation, financial statements, and extensive literary works. The extended context window facilitates improved summarization, Q&A, trend forecasting, and more.

2x Decrease in Hallucination Rates

With a 2x decrease in false statements compared to Claude 2.0, Claude 2.1 demonstrates increased honesty and reliability. This reduction in hallucination rates enhances the trustworthiness of AI applications, making it suitable for solving concrete business problems and deploying AI across various operations.

Improvements in Comprehension and Summarization

Claude 2.1 exhibits a 30% reduction in incorrect answers and a 3-4x lower rate of mistakenly concluding a document supports a particular claim. These improvements are particularly beneficial for handling long, complex documents such as legal texts, financial reports, and technical specifications.

Run Claude 2.1 with an API

Claude 2.1 is on Clarifai and you can run the model with an API using Clarifai’s Python SDK.

Export your Personal Access Token(PAT) as an environment variable. Find your PAT in security settings.

export CLARIFAI_PAT={your personal access token}

from clarifai.client.model import Model
prompt = '''
Analyze the provided financial report and summarize key trends and insights.
<financial_report>
{{FINANCIAL_REPORT}}
</financial_report>
'''
anthropic_api_key = API_KEY
inference_params = dict(temperature=0.2, max_tokens=100, api_key = anthropic_api_key)
# Model Predict
model_prediction = Model("https://clarifai.com/anthropic/completion/models/claude-2_1").predict_by_bytes(prompt.encode(), input_type="text", inference_params=inference_params)
print(model_prediction.outputs[0].data.text.raw)
view raw claude_2.1.py hosted with ❤ by GitHub

You can also run Claude 2.1 API using other Clarifai Client Libraries like Javascript, Java, cURL, NodeJS, PHP, etc here.

Model Demo:

Try out the Claude 2.1 model in the Clarifai Platform here: clarifai.com/anthropic/completion/models/claude-2_1

Prompting Techniques for Claude 2.1

Prompting Claude 2.1 effectively with its expanded 200K context window involves considerations similar to those for the 100K context window, with a crucial distinction:

Input Types

Inputs for Claude 2.1 can take various forms, including but not limited to:

  • Natural Language Text: Prose, reports, articles, books, essays, etc.
  • Structured Documents: Forms, questionnaires, tables, lists.
  • Code Snippets
  • RAG Results: Chunked documents and search snippets.
  • Conversational Text: Transcripts, chat history (prompts and responses), questions, and answers.

Proper Prompt Structuring Examples

Bad Prompt:

Human: What do these academic papers say about the future of AI development?

<papers>

</papers>

Assistant:

Good Prompt:

Human: Here are some academic papers. Read these papers carefully, as you will be asked questions about them.

<papers>

</papers>

What do these papers say about the future of AI development?

Assistant:

Note

This structure helps ensure that your source documents & inputs appear in the prompt before your user query. This input-before-query ordering is vital to eliciting good performance from Claude 2.1.

Use Cases

Claude 2.1's advancements open up new use cases, including:

  • Longer-Form Content: With its expanded context window, Claude 2.1 excels in processing longer-form content, making it suitable for tasks involving extensive documents like codebases, financial statements, and literary works.
  • RAG (Retrieval-Augmented Generation): The increased context window enhances capabilities for retrieval-augmented generation, allowing for more sophisticated reasoning and content generation.
  • Handling more complex reasoning, conversation, and discourse over extended contexts.

Keep up to speed with AI

  • Follow us on Twitter X to get the latest from the LLMs

  • Join us in Discord to talk LLMs

Previous Return to Blog Menu Next