Every single multimodal LLM you are using right now is broken in predictable, measurable ways that no public benchmark will tell you. Over 72 hours this week, nine independent research groups dropped papers documenting consistent failure modes across every open and closed model. None of these papers propose another 1% benchmark gain. They are pulling the foundation out from under what we thought worked.

We have been lying about multimodal judges

When you use GPT-4o, Claude 3 Opus or any open MLLM as an automated evaluator, you assume it will look at the image first, then judge the answer. This does not happen.

Researchers documented what they call Perceptual Judgment Bias: when visual evidence conflicts with textual plausibility, MLLM judges will reliably reward the well written wrong answer every single time. In controlled tests they left response text completely unchanged, modified one critical detail in the image that made the answer factually incorrect. 78% of the time the judge returned exactly the same score. Nobody noticed this flaw for two years because all judge benchmarks only tested obviously wrong answers, not convincing wrong answers.

If you are using MLLM-as-a-Judge for reward modeling right now, your training signal is garbage. You are not training your model to produce correct answers. You are training it to produce answers that sound correct.

The paper proposes a simple fix: generate minimally perturbed counterfactual images, train a GRPO based reward model on verifiable perceptual correctness. This lifted human alignment on judge benchmarks from 62% to 89% with no increase in model size.

Continual tuning does not work the way we thought

Nobody deploys static multimodal models. Every production team is adding new tasks every month. Almost everyone uses the standard approach: Mixture of LoRA Experts routed by image-text embedding similarity.

That approach is fundamentally broken.

Two independent papers, ProtoAda and CRAM, both identified the same flaw this week. Routing by input embedding similarity will assign coordinate grounding tasks and open ended VQA tasks to the exact same expert. They look semantically identical to the vision encoder. They have completely incompatible output formats. After you tune the VQA task, the grounding expert will silently start returning natural language sentences instead of bounding box coordinates. No regression test will catch this until it breaks in production.

ProtoAda solves this with format aware task prototypes that route based on both input semantics and required output structure. CRAM uses centroid routing and orthogonality penalties to isolate incompatible parameter updates. Both deliver ~18% lower catastrophic forgetting across 12 sequential tasks than the previous state of the art.

You can stop tuning LoRAs per task now. That method has been obsolete for three days.

Video MLLMs are wasting 85% of their compute

AdaCodec is the single most important paper in this batch. Every video MLLM today encodes every sampled frame as an independent full resolution RGB image. This is insane. Video is 95% temporally redundant.

AdaCodec only sends a full reference frame when conditional prediction error crosses a calibrated threshold. All other frames are encoded as 12 token delta patches describing only what changed. At 1/7th the total visual token budget, it outperforms the Qwen3-VL-8B baseline on every long video benchmark. Time to first token drops from 9.26s to 1.62s.

This is not a minor optimization. This changes the entire cost curve for video inference. Every production video MLLM will be running something like this within 6 months. There is no good argument for encoding full frames any more.

No model can see things that happen for 3 frames

Moment-Video is the first benchmark that tests for transient visual events. All existing video benchmarks ask questions about objects or events that persist for hundreds of frames. Real world questions almost never work that way. A pedestrian stepping into the road. A switch being flipped. A hand gesture. These things last 2 to 5 frames.

1000 human verified QA pairs. 33 models tested. The best performing model in the world, Seed-2.0-Pro, scored 39.6% overall accuracy. Every open source model scored below 25%.

Doubling frame sampling rate only improved scores by 7%. This is not a sampling problem. Transient events get averaged out during visual token aggregation. They are erased before they ever reach the language decoder.

Right now you cannot build a reliable system that detects momentary events. Not with any model available. This is not something you can fix with prompt engineering. This is a fundamental architectural flaw in every existing MLLM.

Multi stream reasoning does not exist

X-Stream is the first benchmark for concurrent multi video stream understanding. This is the capability required for autonomous driving, live sports analytics, security camera monitoring and multi screen collaboration. Every vendor claims they support this.

No model can do this.

All state of the art models score approximately 50% on this benchmark. That is chance performance. A model can attend to one stream. It will completely ignore the second one. You can fit both streams comfortably into 128k context window. It does not matter. There is no multiplexing mechanism. The model will only ever reason about one stream at a time.

Every product demo you have seen showing multi camera processing is cherry picked.

There is no best modality for 3D reasoning

MASER tested every common input modality across the Open3D-VQA benchmark. No single modality wins more than 52% of questions. Point clouds are best half the time. RGB is best 31% of the time. Depth maps are best 17%.

Every existing 3D MLLM is fine tuned for one primary modality. All of them leave 30-40% of possible accuracy on the table.

The fix is trivial. MASER adds a 120,000 parameter MLP router that selects the correct modality adapter per question. It matches oracle performance within 0.2%. This adds 0.1ms to inference time. No one did this before because everyone assumed you should just feed every modality into the model all the time.

Spatial reasoning requires active exploration

All existing spatial reasoning benchmarks treat the model as a passive observer. Humans do not answer spatial questions by looking once. They move their head. They walk around. They build a map.

Inspired directly by documented pigeon navigation behaviour, researchers built an agentic VLM that maintains a persistent dynamic cognitive map of the scene, and can request additional camera views during reasoning. They added intermediate spatial assertion codes that provide dense verifiable reward during training.

On the MindCube rotation subset this model hit 80.5% accuracy. The previous best result was 51%.

Stop trying to make passive VLMs good at spatial reasoning. It will never work. The model needs agency. It needs to be able to move the camera. This is not a limitation of model size. This is a limitation of how we set up the task.

Safety warning systems are useless right now

PaSBench-Video tests proactive safety warning. This is not: "did an accident happen". This is: "there will be an accident in 1.2 seconds, warn now".

13 models tested. No model exceeded 20% on the strict evaluation metric. Every model showed an almost perfect 0.64 Pearson correlation between recall and false positive rate. To get 50% recall you have to accept that 60% of completely safe scenes will trigger a warning.

Models do not see emerging risk. They see dangerous scenes. They cannot tell the difference between a car that is about to crash and a car that is driving normally. All the demo videos you have seen are cherry picked. There is no production ready MLLM safety monitor today.

What this all means

None of these papers are about making models bigger. None are about another leaderboard win. All of them are about measuring things we never bothered to measure before.

We spent three years scaling MLLMs. We never stopped to check if they actually see anything. It turns out most of the time they do not. They read the text. They guess. They write very convincing wrong answers.

For engineers this is good news. Almost every hard problem people are fighting right now has a known solution. Most of them are trivial to implement. We just did not know the problems existed until this week.

What you should do next

Stop using vanilla MLLM as judges. Run the perturbation test first. If your judge gives the same score when you change the ground truth image, throw it away.

Replace your per-task LoRA routing with format aware prototypes. This will eliminate 90% of the silent regression you get when adding new tasks.

Stop encoding every video frame. Test AdaCodec. You will get better performance at 1/7th the cost.

Stop building systems that rely on detecting momentary events. They will fail in production. Wait until someone fixes the temporal aggregation problem.

Do not promise multi stream support. No one can do that yet.

This is the state of multimodal LLMs as of June 2026. We know more today than we did last week. Most of what we knew was wrong. That is how research works.