metadata
language: en
tags:
- gpt
- text-generation
license: gpl
widget:
- text: 'Once upon a time, '
Keras Implementation of Text generation with a miniature GPT
This repo contains the model and the notebook to this Keras example on Text generation with a miniature GPT.
Full credits to: fchollet
Background Information
This example demonstrates how to implement text generation with a miniature GPT model. The model consists of a single Transformer block with causal masking in its attention layer.
Datasets
IMDB sentiment classification dataset for training. The model generates new movie reviews for a given prompt
How to use
You can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, I set seed for reproducibility:
>>> from transformers import pipeline, set_seed
>>> model = generation= pipeline('text-generation', model='keras-io/text-generation-miniature-gpt', tokenizer='bert-base-uncased')
>>> set_seed(20)
>>> generation("Once upon a time,", max_length=30, num_return_sequences=5)