Instructions to use NotShrirang/marathi-tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use NotShrirang/marathi-tokenizer with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("NotShrirang/marathi-tokenizer", dtype="auto") - Notebooks
- Google Colab
- Kaggle
| license: mit | |
| datasets: | |
| - samarthSonawane/marathi_news_articles | |
| language: | |
| - mr | |
| - en | |
| library_name: transformers | |
| # Marathi Tokenizer | |
| Vocab Size: 32768 |