Tokenizer
A tokenizer with a vocab size of 1k for Intro to Deep Learning Homework 4 on Language Modelling and Automatic Speech Recognition.
The tokenizer was trained on LibriSpeech LM text
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("alexgichamba/hw4_tokenizer_1k", dtype="auto")