KelvinYang/harp-codec

HARP: Harmonic-Aware Residual Partitioning for Neural Audio Codecs

HARP is a neural audio codec that partitions residual vector quantization (RVQ) across harmonically meaningful frequency bands, giving high-quality, variable-bitrate audio compression from a single model. A harmonic-aware partitioning distributes codebook capacity across perceptually meaningful bands, so one model serves multiple bitrates by decoding a growing number of codebook groups.

• 📄 Paper: arXiv:2607.16657 (Interspeech 2026, Oral)
• 💻 Code: https://github.com/QiaoyuYang/harp-codec
• 🔊 Audio examples: https://qiaoyuyang.github.io/harp-codec/

Model details

• Architecture: convolutional encoder/decoder with grouped RVQ.
• Codebooks: 9 codebooks × 1024 entries (10 bits each), codebook dim 8.
• Band groups: 4 groups with a 3-2-2-2 codebook split, ordered by perceptual importance:

  Group	Band	Codebooks
  0	Bass (~0–1 kHz)	3
  1	Low-mid (~1–4 kHz)	2
  2	High-mid (~4–10 kHz)	2
  3	Treble (~10–22 kHz)	2

The bass group is always decoded; adding groups raises quality and bitrate.

Bitrate tiers

With 9 codebooks @ 1024 entries and a ~86 Hz frame rate:

Groups	Codebooks	Approx. bitrate
1	3	~2.6 kbps
2	5	~4.3 kbps
3	7	~6.0 kbps
4	9	~7.7 kbps (full)

Files

• harp.ckpt — slim, inference-ready weights (~293 MB, weights only).

Usage

Set up the GitHub repo (uv sync), download this checkpoint, and reconstruct audio:

uv pip install "huggingface_hub[cli]"
hf download KelvinYang/harp-codec harp.ckpt --local-dir checkpoints

python entry.py -i --input audio.wav --output recon.wav              # full rate
python entry.py -i --input audio.wav --n-groups 2 --output out.wav   # lower bitrate (1..4)

Each run reports SI-SDR, multi-scale mel loss, LSD, and SNR.

Citation

@inproceedings{harp2026,
  title     = {HARP: Harmonic-Aware Residual Partitioning for Neural Audio Codecs},
  author    = {Yang, Qiaoyu and He, Lixing and Deng, Binyue and Zhao, Weifeng},
  booktitle = {Interspeech},
  year      = {2026}
}

License

MIT