hacky/acchead-llama2-chat-7bx70b

The Acceptance Prediction Head for Llama-2-chat 7B and 70B model pair trained with weight_mismatch=6 and resnet_num_layers=3. It is recommended to be used with stop_threshold=0.7. See arxiv: 2405.19715 for more details.

Usage: GitHub

Citation

@article{huang2024specdec++,
  title={SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths},
  author={Huang, Kaixuan and Guo, Xudong and Wang, Mengdi},
  journal={arXiv preprint arXiv:2405.19715},
  year={2024}
}