SiQ-VL: A Curriculum for Small VLMs When Compute Is the Hard Constraint
SiQ-VL: A Curriculum for Small VLMs When Compute Is the Hard Constraint Most VLM writeups assume a cluster. SiQ-VL started from the opposite constraint: one (or few) GPUs, and the question was which design choices still buy capability when you cannot buy FLOPs. This post is the consolidated field guide for that project — architecture, token economics, staged training, and offline Chain-of-Thought (CoT) distillation — replacing three earlier notes that said the same thing three ways. Kernel-level throughput (how we pushed Stage-1 from ~15K to ~100K real tokens/s on Blackwell) lives in the companion post: Optimizing VLM Training on One GPU. ...