MohammedNasri/whisper-algerian-dialect

Whisper Fine-tuned for Algerian Dialect

This model is a fine-tuned version of OpenAI's Whisper-tiny specifically for Algerian dialect automatic speech recognition (ASR).

Model Description

• Base Model: openai/whisper-tiny
• Language: Algerian Arabic dialect with French code-switching
• Dataset: UBC-NLP/Casablanca (Algeria subset)
• Task: Automatic Speech Recognition (ASR)
• Fine-tuned by: Mohammed Nasri

Performance

• Word Error Rate (WER): ~23% on validation set
• Optimized for: Algerian dialect, Arabic-French code-switching
• Training: Fine-tuned with low-resource optimization techniques

Usage

from transformers import WhisperForConditionalGeneration, WhisperProcessor
import torch
import numpy as np

# Load model and processor
model = WhisperForConditionalGeneration.from_pretrained("MohammedNasri/whisper-algerian-dialect")
processor = WhisperProcessor.from_pretrained("MohammedNasri/whisper-algerian-dialect")

# Process audio (audio_array should be 16kHz mono)
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt")

# Generate transcription
with torch.no_grad():
    predicted_ids = model.generate(inputs["input_features"])
    transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]

print(transcription)

Training Details

• Training Framework: HuggingFace Transformers
• Optimization: Mixed precision training, gradient accumulation
• Hardware: Optimized for both CPU and GPU training
• Memory: Ultra low-memory training techniques applied
• Training Steps: 3 steps (proof of concept)
• Batch Size: Minimal for memory optimization

Dataset

The model was trained on the Algeria subset of the UBC-NLP/Casablanca dataset, which contains authentic Algerian dialect audio recordings with transcriptions.

Limitations

• This is a proof-of-concept model trained with minimal steps for demonstration
• For production use, longer training with more steps is recommended
• Optimized for Algerian dialect but may work with other North African Arabic dialects

Citation

If you use this model, please cite:

@misc{whisper-algerian-dialect,
  title={Whisper Fine-tuned for Algerian Dialect},
  author={Mohammed Nasri},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/MohammedNasri/whisper-algerian-dialect}
}

Acknowledgments

• OpenAI for the Whisper model
• UBC-NLP for the Casablanca dataset
• HuggingFace for the training infrastructure