Across Kimi K3, Inkling, and Qwen3-Omni, a single linear boundary separates image vectors from text vectors at 100% in-sample accuracy. The separation comes from coordinate conventions alone: each modality occupies its own offset and basis, so a classifier can label a vector by modality without using its paired content. Perfect separability shows only that the coordinate systems differ. Whether image and text share content is a separate question, which the rest of this note measures.
Whitening each modality and applying one rank-32 orthogonal Procrustes rotation, fitted on the training pairs, aligns the image basis with the text basis. After alignment, each held-out image is compared with the candidate captions and matched to the nearest one. Its own caption is the nearest match far more often than random pairing would give: 9.9× as often in Kimi K3, 3.2× in Inkling, and 9.2× in Qwen3-Omni. The rotation closes part of the basis mismatch, and how much it closes depends on the architecture.
Part of the gap between image and text is just a coordinate mismatch, and a rotation removes it. How much it removes varies by architecture. The modality does not set it.
What was measured
For each model, we use paired image and text vectors from the decoder boundary, where projected image features and text embeddings enter the same residual stream. Kimi K3 contributes 128 caption pairs. Inkling and Qwen3-Omni contribute 96 each. Every figure below draws the same vectors. Only the coordinate treatment changes.
Three quantities are reported per model:
- Raw separability is the accuracy of a linear classifier on the native coordinates.
- Cone half-angle is the median angle between a sample and its own modality mean direction.
- Paired rotation is the angle between an image vector and the text vector of the same pair.
Native coordinates separate trivially
A single linear boundary separates the modalities in all three models. The classifier reaches 100% in-sample accuracy because each modality has a different offset and basis. It identifies the modality from a vector without using the paired content.
Dispersion differs by architecture
Each encoder-projector model has a narrow cone for one modality and a wide cone for the other, in opposite assignments:
- Kimi K3 has a 1.6° median text cone.
- Qwen3-Omni has a 2.0° median image cone.
- The encoder-free model has wide cones for both modalities.
Held-out retrieval after rank-32 alignment
We whiten each modality on training pairs and fit a rank-32 orthogonal Procrustes map from the image basis to the text basis. A held-out image then retrieves its caption by nearest neighbor. The lift over chance is 9.9× for Kimi K3, 3.2× for Inkling, and 9.2× for Qwen3-Omni.
Whiten and truncate image vectors in their own training basis.
Whiten and truncate text vectors independently.
Cross-covariance from the 96 training pairs only.
Discard Σ and retain only rotation R. The map has no scaling or translation.
Apply R to held-out image vectors, then rank text vectors by cosine similarity over 32 candidates.
unmodified decoder-boundary vectors
L2-normalized, cosine geometry only
joint image-text mean subtracted
Limits of the projection
Whitening rescales each modality by its own training standard deviation, so both clusters end up the same size in the plot. That equal size comes from the rescaling and does not describe the raw vectors. Radius, area, and distance from the origin no longer reflect real activation magnitude.
Figure 4 shows three dimensions of a 32-dimensional aligned space. Points that look close here are not always matched pairs. The real evidence is the held-out retrieval, measured in the full space.
@misc{modalitytransport2026,
author = {Korolev, Konstantin},
title = {Image and text do not share a coordinate system},
year = {2026},
note = {White Circle},
url = {https://advprop.ai/notes/modality-transport/}
}This research is sponsored by White Circle.