VLM

Large vision-language models describe images well but are vague about where things are. Perceptio makes the model emit segmentation and depth tokens first, then answer. Short note on the idea, the training tricks and the results.

Perceptio explores perception-enhanced vision-language modeling through spatial token generation for complex 2D and 3D spatial reasoning.