A Survey on Data Selection for LLM Instruction Tuning
Paper
• 2402.05123 • Published
• 3
LLM Data
-
Data-Juicer Sandbox: A Comprehensive Suite for Multimodal Data-Model Co-development
Paper • 2407.11784 • Published • 4
Data Management For Large Language Models: A Survey
Paper • 2312.01700 • PublishedNote 综述
Datasets for Large Language Models: A Comprehensive Survey
Paper • 2402.18041 • Published • 2Note 其他相关文章： The Art of Data Selection: A Survey on Data Selection for Fine-Tuning Large Language Models https://openreview.net/pdf?id=hTBD3LYoqd A Survey on Data Quality Dimensions and Tools for Machine Learning https://arxiv.org/abs/2406.19614
Beyond Scale: the Diversity Coefficient as a Data Quality Metric Demonstrates LLMs are Pre-trained on Formally Diverse Data
Paper • 2306.13840 • Published • 12Note Stanford. 数据多样性
-
From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models
Paper • 2406.16838 • Published • 2 -
Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks
Paper • 2309.17002 • Published • 1
A Survey on Data Selection for Language Models
Paper • 2402.16827 • Published • 4Note **
-
BaichuanSEED: Sharing the Potential of ExtensivE Data Collection and Deduplication by Introducing a Competitive Large Language Model Baseline
Paper • 2408.15079 • Published • 54 -
LLM360/TxT360
Updated • 15.7k • 248
HuggingFaceFW/fineweb-edu
Viewer • Updated • 3.5B • 228k • 967Note https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu-score-2 https://huggingface.co/HuggingFaceFW/fineweb-edu-classifier
-
D4: Improving LLM Pretraining via Document De-Duplication and Diversification
Paper • 2308.12284 • Published