Tanishq Mathew Abraham, Ph.D. avatar

Tanishq Mathew Abraham, Ph.D.

@iScienceLuvr

5/7/2025, 10:02:32 AM

Rewriting Pre-Training Data Boosts LLM Performance in Math and Code

Introduces two openly licensed datasets:
1. SwallowCode (≈16.1 billion tokens) refines Python snippets from The-Stack-v2
2. SwallowMath (≈2.3 billion tokens) enhances Finemath-4+ by removing boilerplate, restoring context, and reformatting solutions into concise, step-by-step explanations
abs: https://arxiv.org/abs/2505.02881
datasets: https://huggingface.co/datasets/tokyotech-llm/swallow-code
https://huggingface.co/datasets/tokyotech-llm/swallow-math
Share
Explore

TwitterXDownload

v1.1.5

The fastest and most reliable Twitter video downloader. Free to use, no registration required.

© 2024 TwitterXDownload All rights reserved.