Instructions to use rapminerz/Kichtral-7B-v0.3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use rapminerz/Kichtral-7B-v0.3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="rapminerz/Kichtral-7B-v0.3")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("rapminerz/Kichtral-7B-v0.3")
model = AutoModelForCausalLM.from_pretrained("rapminerz/Kichtral-7B-v0.3")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use rapminerz/Kichtral-7B-v0.3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "rapminerz/Kichtral-7B-v0.3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rapminerz/Kichtral-7B-v0.3",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/rapminerz/Kichtral-7B-v0.3

SGLang

How to use rapminerz/Kichtral-7B-v0.3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "rapminerz/Kichtral-7B-v0.3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rapminerz/Kichtral-7B-v0.3",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "rapminerz/Kichtral-7B-v0.3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rapminerz/Kichtral-7B-v0.3",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use rapminerz/Kichtral-7B-v0.3 with Docker Model Runner:
```
docker model run hf.co/rapminerz/Kichtral-7B-v0.3
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Kichtral-7B-v0.3: a Mistral-7B Casual LM for French Rap Lyrics

Overview

Kichtral-7B-v0.3 is a Casual Language Model fine-tuned from the Mistral 7B model on french rap lyrics. The training dataset consists of cleaned French verses, with no repetitions, from songs that have at least 10k streams on Spotify. This dataset contains a total of 36M tokens.

This model aims to understand and generate french rap lyrics, making it a valuable tool for research in french slang and music lyrics generation.

Model Details

Kichtral-7B-v0.3 is based on the Mistral 7B v0.3 architecture and has been fine-tuned with the following hyperparameters:

Parameter	Value
Epochs	1
LoRA Rank	64
LoRA Alpha	128
LoRA Dropout	0.1
Learning Rate	1e-4
Learning Scheduler	Cosine

Versions

The model was trained using AWS SageMaker on a single ml.g5.2xlarge instance during 15 hours with the following software versions:

Requirement	Version
Transformers	4.28
PyTorch	2.0
Python	3.10

Installation

Install the required Python libraries:

pip install transformers

Loading the Model

from transformers import AutoTokenizer, AutoModelForCausalLM

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("rapminerz/Kichtral-7B-v0.3")
model = AutoModelForCausalLM.from_pretrained("rapminerz/Kichtral-7B-v0.3")

Using the Model

def generate_lyrics(prompt):
    inputs = tokenizer(prompt, return_tensors="pt")
    outputs = model.generate(
        inputs["input_ids"], 
        max_length=300, 
        num_return_sequences=1,
        top_k=10,
        top_p=0.95,
        temperature=1.0,
        repetition_penalty=1.2
    )
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

generate_lyrics("Okay ça fait")
"""
Okay ça fait un moment que tu m'appelles
Sans t'écouter, j'ai dû me tailler
Jusqu'à présent, je sais pas qui t'es mais je peux pas t'oublier
Tu m'as laissé des images dans l'crâne
Quand je repense à ce soir-là
"""

generate_lyrics("Je viens de là où")
"""
Je viens de là où ça tire
Je fais la loi je suis pas le roi
Et je sais que tu penses à moi quand t'as besoin d'aide
Quand y a trop d'ennemis autour de toi qui se mêlent
"""

Purpose and Disclaimer

This model is designed for academic and research purposes only. It is not intended for commercial use. The creators of this model do not endorse or promote any specific views or opinions that may be represented in the dataset.

Please mention @RapMinerz if you use our models

Improvements

This model doesn't totally capture rhymes, another method should be needed to prompt for example rhymes and topics

Contact

For any questions or issues, please contact the repository owner, RapMinerz, at rapminerz.contact@gmail.com.

Downloads last month: 3

Safetensors

Model size

7B params

Tensor type

F16

Model tree for rapminerz/Kichtral-7B-v0.3

Quantizations

1 model