Manifest — 4.0B tokens
5f448f1 verified - 149 MB coder_combined — 74,567,978 tokens (0.1GB)
- 16 GB english_cx — 8,000,000,000 tokens (14.9GB)
- 10 GB fineweb_edu — 5,000,000,000 tokens (9.3GB)
- 93.3 MB langchain_docs — 46,653,933 tokens (0.1GB)
- 417 Bytes Manifest — 4.0B tokens
- 8 GB math — 4,000,000,000 tokens (7.5GB)
- 16.4 kB ot — 8,192 tokens (0.0GB)
- 16 GB sinhala_cx — 8,000,000,000 tokens (14.9GB)
- 4 GB sinhalen_gz — 2,000,000,000 tokens (3.7GB)
