Single character based miniGPT based on the tutorial by Andrej Karpathy. The miniGPT is trained on a dataset containing all the lyrics of the Dutch sing and song writers "Acda en de Munnik". A webscraper is added to the repository which is used to create the dataset. The dataset itself is also added. The model definition for the transformer can be found in in model.py. The script for training is also added to the repo.
This model is not meant as a perfect GPT for generating song lyrics. Rather, it is meant as a practice exercise, familiarizing yourself with the concept of generative transformers on a custom dataset.