FlashFormer: Whole-Model Kernels for Efficient Low-Batch Inference
Paper
• 2505.22758 • Published
AI & ML interests
None defined yet.
mayank-mishra
authored 2 papers 9 months ago
mayank-mishra
authored 3 papers about 1 year ago
Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence
Paper
• 2502.09927 • Published
Ladder-residual: parallelism-aware architecture for accelerating large model inference with communication overlapping
Paper
• 2501.06589 • Published
Selective Self-Rehearsal: A Fine-Tuning Approach to Improve Generalization in Large Language Models
Paper
• 2409.04787 • Published
• 1
mayank-mishra
authored 4 papers over 1 year ago
Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler
Paper
• 2408.13359 • Published
• 23
Enhancing Training Efficiency Using Packing with Flash Attention
Paper
• 2407.09105 • Published
• 17
Scaling Granite Code Models to 128K Context
Paper
• 2407.13739 • Published
• 21
The infrastructure powering IBM's Gen AI model development
Paper
• 2407.05467 • Published
• 3
mayank-mishra
posted an
update almost 2 years ago
Post
2111
New preprint out with colleagues from MIT and IBM Research
Reducing Transformer Key-Value Cache Size with Cross-Layer Attention (2405.12981)
We introduce a simple mechanism of sharing keys and values across layers, reducing the memory needed for KV cache during inference!!
Reducing Transformer Key-Value Cache Size with Cross-Layer Attention (2405.12981)
We introduce a simple mechanism of sharing keys and values across layers, reducing the memory needed for KV cache during inference!!
- 1 reply
mayank-mishra
authored 5 papers almost 2 years ago
Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
Paper
• 2405.12981 • Published
• 33
Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization
Paper
• 2404.03605 • Published
• 1
Granite Code Models: A Family of Open Foundation Models for Code Intelligence
Paper
• 2405.04324 • Published
• 26
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Paper
• 2211.05100 • Published
• 37
Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models
Paper
• 2404.05567 • Published
• 10
mayank-mishra
posted an
update almost 2 years ago
Post
2677
Thrilled to unveil DS-MoE: a dense training and sparse inference scheme for enhanced computational and memory efficiency in your MoE models! 🚀🚀🚀
Discover more in our blog: https://huggingface.co/blog/bpan/ds-moe and dive into the details with our paper: Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models (2404.05567)
Discover more in our blog: https://huggingface.co/blog/bpan/ds-moe and dive into the details with our paper: Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models (2404.05567)
- 1 reply
mayank-mishra
authored a paper almost 2 years ago
mayank-mishra
posted an
update almost 2 years ago
Post
1911
Current LLMs are very susceptible to generating toxic, harmful and even dangerous content. They can also generate outputs with gender or racial biases.
The Biden-Harris Executive Order (https://www.federalregister.gov/documents/2023/11/01/2023-24283/safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence) sets forth guidelines on what is considered a safe AI system.
Following up on these guidelines, we present the world's first open source Biden-Harris Executive Order Red teamed Multilingual Language Model: Aurora-M.
The model is trained on 5 languages: English, Hindi, Japanese, Vietnamese and Finnish.
Blog: https://huggingface.co/blog/mayank-mishra/aurora
Paper coming out soon.
Base model: aurora-m/aurora-m-base (not safety tuned)
Instruct model: aurora-m/aurora-m-instruct (not safety tuned)
Red teamed model: aurora-m/aurora-m-biden-harris-redteamed (safety tuned according to the order mentioned above)
The Biden-Harris Executive Order (https://www.federalregister.gov/documents/2023/11/01/2023-24283/safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence) sets forth guidelines on what is considered a safe AI system.
Following up on these guidelines, we present the world's first open source Biden-Harris Executive Order Red teamed Multilingual Language Model: Aurora-M.
The model is trained on 5 languages: English, Hindi, Japanese, Vietnamese and Finnish.
Blog: https://huggingface.co/blog/mayank-mishra/aurora
Paper coming out soon.
Base model: aurora-m/aurora-m-base (not safety tuned)
Instruct model: aurora-m/aurora-m-instruct (not safety tuned)
Red teamed model: aurora-m/aurora-m-biden-harris-redteamed (safety tuned according to the order mentioned above)
mayank-mishra
authored a paper about 2 years ago
mayank-mishra
posted an
update about 2 years ago
Post
I have just published my first blog post.
While FlashAttention has been readily integrated into HuggingFace transformers, there are much higher gains to be had (at least theoretically) for finetuning models with examples of variable sequence lengths in a batch.
For a deeper dive, please go through my blog at https://huggingface.co/blog/mayank-mishra/padding-free-transformer.
While FlashAttention has been readily integrated into HuggingFace transformers, there are much higher gains to be had (at least theoretically) for finetuning models with examples of variable sequence lengths in a batch.
For a deeper dive, please go through my blog at https://huggingface.co/blog/mayank-mishra/padding-free-transformer.
-
- 10 replies