nanochat-jp Bundles Japanese Training And Evaluation Into One Repository
nanochat-jp, a Japanese fork of Andrej Karpathy's nanochat, carries tokenizer training, pretraining, supervised fine-tuning, reinforcement learning, evaluation and inference in a single repository.
1 source 1h ago
Reporting from 1 source: GIGAZINE.
Why it matters
A complete Japanese pipeline in one repository lowers the setup cost for anyone testing how a small model handles Japanese, and the released v4 and v3 checkpoints let others check the developer's benchmark claims without training from scratch.
Key Facts
- nanochat-jp is a Japanese fork of nanochat, the project released by Andrej Karpathy in October 2025.
- nanochat-jp adds a SentencePiece Unigram tokenizer, a Japanese pretraining corpus, Japanese SFT and RL data, Japanese CORE and ChatCORE benchmarks, and a Hugging Face Transformers converter.
- Training runs from speedrun.sh in the order of tokenizer training and evaluation, pretraining, SFT and training report generation.
- In developer benchmarks, nanochat-jp-v4 rose from 41.4 to 52.3 on YOMI-Gen and passed Qwen3.5-2B on YOMI-Gen and PFGen.
- Checkpoints for nanochat-jp-v4 and nanochat-jp-v3, an SFT model in Hugging Face format, and a vLLM plugin are released.
It adds a SentencePiece Unigram tokenizer, a Japanese pretraining corpus, Japanese SFT and RL data, Japanese CORE and ChatCORE benchmarks, and a converter to Hugging Face Transformers format. Training runs from speedrun.sh; base_eval.sh and chat_eval.sh handle evaluation.
The fork swaps in a SentencePiece Unigram tokenizer and publishes its Japanese text data on Hugging Face, including a pretraining corpus plus SFT and RL sets. The standard script runs tokenizer training and evaluation, pretraining, SFT and report generation in that order; reinforcement learning is optional in it.
In the developer's benchmarks, the model trained with standard settings scored comparably to Qwen3.5-0.8B on Japanese ability, and nanochat-jp-v4 improved on four of five metrics against v3. On YOMI-Gen it rose from 41.4 to 52.3, and on YOMI-Gen and PFGen it passed Qwen3.5-2B; the Qwen3.5 models were evaluated with thinking disabled. Checkpoints for v4 and v3, an SFT model in Hugging Face format, and a vLLM plugin are released.
- SentencePiece Unigram tokenizer: processes Japanese text
- Japanese pretraining corpus: published on Hugging Face
- Japanese SFT data: question and answer examples for conversational ability
- Japanese RL data: for improving responses
- CORE and ChatCORE benchmarks: measure Japanese ability
- Hugging Face Transformers format converter
Written with AI by Yomimono from the 1 cited source below, including Japanese-language reporting where cited. See our editorial policy.
Add Yomimono as a preferred source on Google