Instructions to use universalml/Nepali_Tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use universalml/Nepali_Tokenizer with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("universalml/Nepali_Tokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
File size: 589 Bytes
1f8bc9d 175c634 1f8bc9d dabba95 937cfff 1f8bc9d 937cfff dabba95 1f8bc9d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | ---
library_name: transformers
license: mit
language:
- en
- ne
base_model:
- microsoft/Phi-3.5-mini-instruct
---
## Model Details
A tokenizer designed for the Nepali language, supporting both word and subword tokenization. Ideal for Nepali NLP tasks such as language modeling, text classification, and more, addressing the specific needs of Nepali script and grammar
### Model Description
- **Developed by:** Universalml Inc.
- **Funded by :** AWS , DigitalOcean
- **Language(s) (NLP):** Nepali
- **License:** MIT
- **Finetuned from model optional:** microsoft/Phi-3.5-mini-instruct
|