Cover most of a photograph. Ask a network what the hidden parts would mean — never what they would look like. No cropping, no colour jitter, no decoder.
↓ SCROLL
01 — The landscape
Every self-supervised method is trying to give compatible inputs a low score and incompatible ones a high score. They differ in one place only: where the comparison happens.
invariance-based
Two views of the same photo should land in the same place.
Crop it, recolour it, flip it — then push the two embeddings together.
the catch
You have to hand-design the augmentations, and they bake in assumptions. Classification and segmentation do not want the same invariances, and none of it transfers to audio.
collapse
Can collapse: nothing stops the encoder outputting the same vector for everything.
DINO · iBOT · SimCLR · BYOL · MSN
reconstruction-based
Hide part of the photo and draw it back in.
Mask patches, run a decoder, compare pixel by pixel.
the catch
Spends its capacity on detail nobody could predict — the exact grain of the grass, the noise in the sky. What it learns sits at a low level of abstraction.
collapse
Cannot collapse, as long as the mask tokens carry little information.
MAE · BEiT · SimMIM
this paper
Hide part of the photo and predict what it would mean, not what it would look like.
Encode the visible part, predict the target encoder's output for the hidden parts, compare in embedding space.
the catch
No decoder, no augmentations. The comparison happens after the detail has already been thrown away.
collapse
Could collapse — prevented by an asymmetric design: the target encoder is an exponential moving average of the context encoder, not a copy being trained.
I-JEPA
the one-line difference
A generative model puts its loss on pixels. A joint-embedding model puts it on two views you had to invent. I-JEPA puts it on representations of parts of the same image — so it needs no augmentations and never has to draw anything.
02 — The argument
Cover a patch of grass. Nobody — no model, no person — can say where each blade will fall. A model trained to draw it back is spending its capacity on noise.
predicting pixels
Every one of these fillings is equally correct. The loss forces the model to pick one, and punishes it for the detail it could never have known.
×which blade goes where
×the exact grain
×sensor noise
×the blur of the background
predicting the representation
The target encoder has already thrown that detail away. What is left is what the patch is — and that is genuinely predictable from the rest of the image.
✓grass
✓in shade
✓below the horizon
✓in front of the fence
and it is measurable — Table 7
Same architecture, same masking. Only the space the loss lives in changes.
loss in representation space
500 epochs
loss in pixel space
800 epochs
Twenty-six points, in favour of the version that trains for fewer epochs. This single row is the paper's central claim.
The trade this makes
Pixel reconstruction gives you something you can look at, which is how you check a generative model is working. I-JEPA gives up that check entirely — you cannot eyeball an embedding. In exchange it stops paying for detail it was never going to get right.
03 — The design choice that matters
The architecture is not what makes this work. The masking is. Predict several medium, object-sized blocks from one large, spatially spread context — get any part of that wrong and the whole thing falls apart.
Each covering 15–20% of the image, with an aspect ratio between 0.75 and 1.5. They are allowed to overlap each other.
Square, covering 85–100% of the image. Almost everything.
Any part of the context that overlaps a target is removed, so the model can never just copy the answer. What survives averages about a quarter of the image.
Try it — every slider is an axis the paper ablated
the paper's choice
the paper's choice
the paper's choice
what this setting scored
54.2
top-1 on 1% ImageNet, ViT-B/16, 300 epochs
default setting · Tables 8–10
This is the paper's sampler, implemented exactly as described. When your settings match one of the ablation rows, the score it actually achieved appears on the right. Move two sliders at once and it goes blank — because that combination was never run.
04 — The architecture
The context encoder is trained. The target encoder is a slow-moving average of it. That asymmetry is the only thing stopping the model from collapsing to a constant.
the objective, in full
That is the entire loss. No contrastive term, no negative pairs, no redundancy penalty, no clustering objective — just squared distance between two sets of vectors.
three asymmetries doing the work
Only one side gets gradients. The target encoder is updated by moving average, never by backprop. It cannot chase the predictor into a degenerate solution.
Only one side sees everything. The target encoder reads the whole image; the context encoder reads about a quarter of it.
The predictor is narrower than either. 384 channels against the encoder's 1024. The prediction has to squeeze through a bottleneck.
Pretraining settings
Appendix A.1
| Optimiser | AdamW | context encoder and predictor only |
| Batch size | 2048 | default |
| Learning rate | 1e-4 → 1e-3 → 1e-6 | warm up over 15 epochs, then cosine decay |
| Weight decay | 0.04 → 0.4 | increased linearly through pretraining |
| EMA momentum | 0.996 → 1.0 | target encoder, increased linearly |
| Target blocks | 4 | scale (0.15, 0.2), aspect ratio (0.75, 1.5) |
| Context block | 1 | scale (0.85, 1.0), unit aspect ratio |
| Predictor width | 384 | narrow, regardless of encoder width |
| Predictor depth | 6 / 12 / 16 | ViT-B / ViT-L, H / ViT-G |
| [cls] token | none | evaluation uses average-pooled patches |
05 — The algorithm
Thirteen lines. Every one of them is a decision the ablations later put a number on.
I-JEPA · one step
line 1
One image. No second view, no augmented copy — this is why it is cheap.
what is active right now
trained by gradient descent
updated by moving average
predicts, never draws
What is missing is the point. There is no augmentation pipeline, no negative sampling, no queue of past embeddings, no decoder, no tokeniser, no [cls] token. A step is: encode everything once with the slow network, encode a quarter of it with the fast one, predict, take a squared distance, and nudge the slow network a fraction of the way toward the fast one.
06 — Results
Among methods that use no hand-crafted augmentations, I-JEPA leads everywhere. Against methods that do use them, it closes most of the gap on semantic tasks and wins outright on low-level ones.
ImageNet-1K · linear evaluation
Frozen encoder, linear classifier on the full training set (Table 1)
| no hand-crafted augmentations | ||||
| data2vec | ViT-L/16 | 1600 | 77.3 | |
| MAE | ViT-B/16 | 1600 | 68.0 | |
| MAE | ViT-L/16 | 1600 | 76.0 | |
| MAE | ViT-H/14 | 1600 | 77.2 | |
| CAE | ViT-B/16 | 1600 | 70.4 | |
| CAE | ViT-L/16 | 1600 | 78.1 | |
| I-JEPA | ViT-B/16 | 600 | 72.9 | |
| I-JEPA | ViT-L/16 | 600 | 77.5 | |
| I-JEPA | ViT-H/14 | 300 | 79.3 | |
| I-JEPA | ViT-H/16₄₄₈ | 300 | 81.1 | |
| using extra view augmentations | ||||
| SimCLR v2 | RN152 (2×) | 800 | 79.1 | |
| DINO | ViT-B/8 | 300 | 80.1 | |
| iBOT | ViT-L/16 | 250 | 81.0 | |
ImageNet-1K · only 1% of labels
About 12 images per class (Table 2)
| no hand-crafted augmentations | ||||
| data2vec | ViT-L/16 | 1600 | 73.3 | |
| MAE | ViT-L/16 | 1600 | 67.1 | |
| MAE | ViT-H/14 | 1600 | 71.5 | |
| I-JEPA | ViT-L/16 | 600 | 69.4 | |
| I-JEPA | ViT-H/14 | 300 | 73.3 | |
| I-JEPA | ViT-H/16₄₄₈ | 300 | 77.3 | |
| using extra view augmentations | ||||
| iBOT | ViT-B/16 | 400 | 69.7 | |
| DINO | ViT-B/8 | 300 | 70.0 | |
| SimCLR v2 | RN151 (2×) | 800 | 70.2 | |
| BYOL | RN200 (2×) | 800 | 71.2 | |
| MSN | ViT-B/4 | 300 | 75.7 | |
pretraining cost against what you get for it
Down and to the right is expensive. Up and to the left is what you want. Hover a point.
A huge I-JEPA model costs less to pretrain than the smallest iBOT model. Computing targets in representation space adds about 7% per iteration, but convergence takes roughly 5× fewer iterations, so it comes out far ahead.
Transfer · semantic tasks
Linear probe on a frozen encoder (Table 3)
| CIFAR100 | Places205 | iNat18 | |
|---|---|---|---|
| data2vecViT-L/16 | 81.6 | 54.6 | 28.1 |
| MAEViT-H/14 | 77.3 | 55 | 32.9 |
| I-JEPAViT-H/14 | 87.5 | 58.4 | 47.6 |
| DINOViT-B/8+aug | 84.9 | 57.9 | 55.9 |
| iBOTViT-L/16+aug | 88.3 | 60.4 | 57.3 |
Transfer · low-level tasks
Object counting and depth prediction (Table 4)
| Clevr/Count | Clevr/Dist | |
|---|---|---|
| data2vecViT-L/16 | 85.3 | 71.3 |
| MAEViT-H/14 | 90.5 | 72.4 |
| I-JEPAViT-H/14 | 86.7 | 72.4 |
| DINOViT-B/8+aug | 86.6 | 53.4 |
| iBOTViT-L/16+aug | 85.7 | 62.8 |
the result that surprises people
On depth prediction, I-JEPA scores 72.4 against DINO's 53.4 — a nineteen point gap. Methods trained to be invariant to crops and colour have, understandably, thrown away exactly the information those tasks need. Predicting representations of specific locations keeps it.
Scaling data and model
Table 5 — transfer after more pretraining
| Pretrain | Arch | CIFAR100 | Places205 | iNat18 | Clevr/Count | Clevr/Dist |
|---|---|---|---|---|---|---|
| IN1k | ViT-H/14 | 87.5 | 58.4 | 47.6 | 86.7 | 72.4 |
| IN22k | ViT-H/14 | 89.5 | 57.8 | 50.5 | 88.6 | 75 |
| IN22k | ViT-G/16 | 89.5 | 59.1 | 55.3 | 86.7 | 73 |
A bigger model on a bigger dataset helps semantic tasks — but the ViT-G/16 uses larger patches, and that hurts the local tasks. The paper points this out rather than averaging it away.
07 — Ablations
This is where the paper earns its claims. Almost every design choice turns out to be load-bearing, and several of them are worth more than the architecture itself.
Table 7 · ViT-L/16 · linear probe on 1% ImageNet-1K
What if we just put the loss back in pixel space?
Top-1 · higher is better
what it shows
This is the whole paper in one row. Move the loss to pixels and the linear probe drops by 26 points — even with 300 more epochs of training. The target encoder is what makes the targets abstract.
spread across this axis
26.2
points between the best and worst setting tested
jump to another
1 of 9
all nine, side by side
How far apart the best and worst tested settings are on each axis. The masking choices dominate — which is the paper's own conclusion about where the work actually is.
08 — What the predictor learns
The authors froze a pretrained model and trained a separate decoder to map the predictor's output back to pixels, purely to look at it. Sampling that decoder repeatedly shows what the representation actually contains.
four samples from one predicted representation
What stays the same across samples is information the representation holds. What changes is information it discarded.
sample 1
sample 2
sample 3
sample 4
kept — the same in every sample
The object's identity and its pose. The back of a bird, curved the right way. The top of a car, at the right angle. The predictor was told only where to look, and it worked out what should be there.
discarded — different every time
Exact colours, background, fine texture, precise low-level detail. None of it survives into the representation, which is the entire design intent rather than a shortcoming.
The predictor is conditioned only on position. It cannot know exactly what is there — and the samples show it representing that uncertainty rather than committing to one guess.
The decoder is trained after the fact, on a frozen model, purely for visualisation. It plays no part in pretraining and cannot flatter the representation it is reading.
These are qualitative samples on chosen images. They illustrate the mechanism; the ablations are what actually establish it.
The trade-off, one last time
You cannot look at what I-JEPA learned without building a separate decoder to look for you. That is the price of never learning to draw. In exchange the model reaches a ViT-Huge on sixteen GPUs in under three days, beats every comparable method on a frozen linear probe, and is better at judging depth than models built specifically to be invariant.