Company
🔥 Just Launched
Introducing the AI Playground — Your LLM Battleground to Test Powerful AI Models!
Try now
Contact us
Contact us
Join the Discord
Join the Discord
September 25, 2023

WizardCoder: Large Language Model for Code

Clarifai Computer Vision By Sumanth P

Table of Contents:

Run WizardCoder With An API

WizardCoder

WizardCoder is a Code Large Language Model (LLM) that has been fine-tuned on Llama2 and has demonstrated superior performance compared to other open-source and closed LLMs on prominent code generation benchmarks.

You can now try out wizardCoder-15B and wizardCoder-Python-34B in the Clarifai Platform and access it through the API.

Table of Contents

  1. Introduction
  2. Evol-Instruct
  3. Prompt Format
  4. Running WizardCoder with Python
  5. Best Use Cases
  6. Evaluation

Introduction

The world of coding has been revolutionized by the advent of large language models (LLMs) like GPT-4, StarCoder, and Code LLama. WizardCoder is taking things to a whole new level. WizardCoder is a specialized model that has been fine-tuned to follow complex coding instructions. It leverages the Evol-Instruct method to adapt to coding tasks, making it a powerful tool for developers.

Evol-Instruct

Evol-Instruct is an evolutionary algorithm that generates diverse and complex instruction data for Large-scale Language Models (LLMs). It is designed to enhance the performance of LLMs by providing them with high-quality instructions that are difficult to create manually.

Evol-Instruct works by generating a pool of initial instructions(52k instruction dataset of Alpaca), which are then evolved through a series of steps to create more complex and diverse instructions. Once the instruction pool is generated, it is used to fine-tune an LLM, resulting in a new model called WizardCoder. The fine-tuning process involves training the LLM on the instruction data to improve its ability to generate coherent and fluent text in response to various inputs.

Prompt Format

For WizardCoder, the Prompt should be as following:

Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{instruction}
### Response:
view raw prompt_format.txt hosted with ❤ by GitHub

Running WizardCoder model with Python

You can run the WizardCoder-15 B Model using Clarifai’s Python client.

Check out the Code Below:

######################################################################################################
# In this section, we set the user authentication, user and app ID, model details, and the URL of
# the text we want as an input. Change these strings to run your own example.
######################################################################################################
# Your PAT (Personal Access Token) can be found in the portal under Authentification
PAT = ''
# Specify the correct user_id/app_id pairings
# Since you're making inferences outside your app's scope
USER_ID = 'wizardlm'
APP_ID = 'generate'
# Change these to whatever model and text URL you want to use
MODEL_ID = 'wizardCoder-15B'
MODEL_VERSION_ID = '7a28224d1a9f406d98b7d0c4307e22d2'
RAW_TEXT = 'I love your product very much'
# To use a hosted text file, assign the url variable
# TEXT_FILE_URL = 'https://samples.clarifai.com/negative_sentence_12.txt'
# Or, to use a local text file, assign the url variable
# TEXT_FILE_LOCATION = 'YOUR_TEXT_FILE_LOCATION_HERE'
############################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
############################################################################
from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2
channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)
metadata = (('authorization', 'Key ' + PAT),)
userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)
# To use a local text file, uncomment the following lines
# with open(TEXT_FILE_LOCATION, "rb") as f:
# file_bytes = f.read()
post_model_outputs_response = stub.PostModelOutputs(
service_pb2.PostModelOutputsRequest(
user_app_id=userDataObject, # The userDataObject is created in the overview and is required when using a PAT
model_id=MODEL_ID,
version_id=MODEL_VERSION_ID, # This is optional. Defaults to the latest model version
inputs=[
resources_pb2.Input(
data=resources_pb2.Data(
text=resources_pb2.Text(
raw=RAW_TEXT
# url=TEXT_FILE_URL
# raw=file_bytes
)
)
)
]
),
metadata=metadata
)
if post_model_outputs_response.status.code != status_code_pb2.SUCCESS:
print(post_model_outputs_response.status)
raise Exception(f"Post model outputs failed, status: {post_model_outputs_response.status.description}")
# Since we have one input, one output will exist here
output = post_model_outputs_response.outputs[0]
print("Completion:\n")
print(output.data.text.raw)
view raw wizardcoder15b.py hosted with ❤ by GitHub

You can also run WizardCoder-15 B Model using other Clarifai Client Libraries like Javascript, Java, cURL, NodeJS, PHP, etc here

Model Demo in the Clarifai Platform:

Try out the WizardCoder-15B and WizardCoder-Python-34B models here:  https://clarifai.com/wizardlm/generate/models/wizardCoder-15B and https://clarifai.com/wizardlm/generate/models/wizardCoder-Python-34B

Best Use Cases

WizardCoder can be used for a variety of code-related tasks, including code generation, code completion, and code summarization. Here are some examples of input prompts that can be used with the model:

  • Code generation: Given a description of a programming task, generate the corresponding code. Example input: “Write a Python function that takes a list of integers as input and returns the sum of all even numbers in the list.”
  • Code completion: Given an incomplete code snippet, complete the code. Example input: “def multiply(a, b): \n return a * b _”
  • Code summarization: Given a long code snippet, generate a summary of the code. Example input: “Write a Python program that reads a CSV file and calculates the average of a specific column.”

The 34B model is not just a coding assistant; it’s a powerhouse capable of:

  1. Automating DevOps Scripts: Generate shell scripts or Python scripts for automating tasks.
  2. Data Analysis: Generate Python code for data preprocessing, analysis, and visualization.
  3. Machine Learning Pipelines: Generate end-to-end ML pipelines, from data collection to model deployment.
  4. Web Scraping: Generate code for web scraping tasks.
  5. API Development: Generate boilerplate code for RESTful APIs.
  6. Blockchain: Generate smart contracts for Ethereum or other blockchain platforms

Evaluation

WizardCoder beats all other open-source Code LLMs, attaining state-of-the-art (SOTA) performance, according to experimental findings from four code-generating benchmarks, including HumanEval, HumanEval+, MBPP, and DS-100.

WizardCoder-Python-34B has demonstrated exceptional performance on code-related tasks. The model has outperformed other open-source and closed LLMs on prominent code generation benchmarks, including HumanEval (73.2%), HumanEval+, and MBPP(61.2%).

WizardCoder-Python-34B-V1.0 attains the second position in HumanEval Benchmarks, surpassing GPT4 (2023/03/15, 73.2 vs. 67.0), ChatGPT-3.5 (73.2 vs. 72.5) and Claude2 (73.2 vs. 71.2).

WizardCoder-15B-v1.0 model achieves the 57.3 pass@1 on the HumanEval Benchmarks, which is 22.3 points higher than the SOTA open-source Code LLMs including StarCoder, CodeGen, CodeGee, and CodeT5+. Additionally, WizardCoder significantly outperforms all the open-source Code LLMs with instructions fine-tuning, including InstructCodeT5+, StarCoder-GPTeacher, and Instruct-Codegen-16B

Keep up to speed with AI

  • Follow us on X to get the latest from the LLMs

  • Join us in our Discord to talk LLMs

Previous Return to Blog Menu Next