CorX1.5
A ~158M parameter decoder-only transformer, built entirely from scratch — architecture, tokenizer, pretraining and fine-tuning, all written and trained solo on a single GPU. Not a fine-tune of an existing base model.
- Parameters
- 157.8M
- Layers
- 10
- Context
- 1,024 tokens
- Vocabulary
- 32,768 BPE
- Pretraining
- ~3B tokens
- Licence
- Apache 2.0