Input

Prompt:

Press Ctrl + Enter to submit

Output

Submit a prompt for a response.

Notes

DistilGPT2

DistilGPT2 English language model pretrained with the supervision of GPT2 (the smallest version of GPT2) on OpenWebTextCorpus, a reproduction of OpenAI's WebText dataset. The model has 6 layers, 768 dimension and 12 heads, totalizing 82M parameters (compared to 124M parameters for GPT2). On average, DistilGPT2 is two times faster than GPT2.

On the WikiText-103 benchmark, GPT2 reaches a perplexity on the test set of 16.3 compared to 21.1 for DistilGPT2 (after fine-tuning on the train set).

We encourage to check GPT2 to know more about usage, limitations and potential biases.

ID
Model Type ID
Text To Text
Input Type
text
Output Type
text
Description
Text generation model for English
Last Updated
Jun 28, 2022
Privacy
PUBLIC
Toolkit
License
Share
Badge