An image tells you what is visible. What would it take to understand the world behind it—and use that understanding in a new situation?

An image tells you what is visible. What would it take to understand the world behind it—and use that understanding in a new situation?

Hirokatsu Kataoka and 20 coauthors

21 pages. Ten perspectives. An open research agenda, with no single agreed route to general intelligence. [1]

Start with an ambiguity. These three mechanisms look the same when closed. Can you choose an observation that tells them apart?

One image. Three possible worlds.

1. Choose a scene hypothesis

How does the blue panel move? Select a possible hidden mechanism.

Move the slider, then compare your prediction with a new observation.

At 0%, every hypothesis explains the same view. Try opening the panel to reveal how its hidden structure changes what you see.

Explore a geometric thought experiment. Positions are generated from simple geometry, not a trained model or paper measurements. Testing reads a simulated hidden mechanism. Inspired by §§2.7–2.8. [3, 4]

One world, several ways to understand it.

Kataoka and colleagues propose that three learning objectives may complement one another. This is a hypothesis about a path toward visual intelligence, not a demonstrated recipe for AGI. [2]

Reconstruct

Infer hidden structure from incomplete observations.

In this lab: ask which mechanism could have produced the observed panel position. A single closed view leaves three explanations.

Predict

Use inferred structure to anticipate what comes next.

In this lab: change the opening command. Each assumed mechanism makes a different prediction about the panel’s next position.

Imagine

Explore possible alternatives while preserving their internal structure.

In this lab: overlay other mechanisms to compare counterfactual outcomes. Enumerating three coded alternatives is only a tiny illustration of open-ended generation.

Does predicting pixels teach physics?

The contributors share questions, not a single architecture. Two perspectives show why a convincing output and evidence of physical understanding need to be evaluated carefully.

Scale generative video models.

Geirhos argues that large-scale video generation can support general visual capabilities: generating a scene demands accounting for much more than a recognition shortcut. The section cites earlier work reporting transfer to tasks without task-specific training. [5]

Test the structure behind the image.

Wu and Wu argue that plausible pixels alone do not establish physical understanding. They emphasize entities, intrinsic properties, extrinsic conditions, relations, and dynamics that can support editing, simulation, and action. [4]

Still open: will useful physical structure emerge from pixel prediction alone, or should it be an explicit learning target? This toy lab cannot settle that question. [4]

Choose what to see next.

Du describes action as a way to obtain evidence, resolve ambiguity, and revise a world model (§2.7). Liu asks whether a visual system knows when to look, where to look, and how much detail it needs (§2.9). [3] [6]

The paper’s discussion broadens evaluation to transfer, continual adaptation, persistent knowledge, creativity, physical consistency, and efficiency. A static image score cannot cover all of these proposed dimensions. [7]

Try it: test the closed panel, then a partly open panel. The second observation separates the mechanisms in this idealized scene. More observations help only if they contain useful distinctions.

Where the demonstration stops

The true mechanism is always a left hinge. Reset clears evidence; it does not sample a new world.

The camera, panel shape, and motion rules are known. The simulator has no noise, friction, occlusion, or actuator error.

The three hypotheses are supplied by hand. The program compares geometry; it does not learn, perceive images, or discover structure.

Ruling out alternatives here is not proof of general intelligence, and does not establish the panel’s material, mass, or behavior under other forces.

The opening command u ranges from 0 to 1. For hinges, rotation θ = u × 80°. Panel width w = 92 drawing units and height h = 110. With a left hinge, the moving edge is (w cos θ, w sin θ); with a right hinge it is (w − w cos θ, w sin θ). A slider translates by 0.9wu. Depth projects vertically with factor 0.42 in an orthographic view. These are illustrative choices, not fitted paper parameters.

A candidate is retained when every corner matches the simulated observation within 0.01 drawing units. All candidates are retained at u = 0; any nonzero slider step separates them. Evidence accumulates until reset. This is exact geometric filtering, not a probability estimate.

It assembles ten perspectives and a research agenda, citing prior systems and results. It does not present an integrated VGI system, a single new benchmark result, or proof of a path to AGI. Its conclusion explicitly leaves the route unresolved (§4, pp. 16–17). The broader agenda also includes creativity, lifelong learning, multimodality, spatial memory, scientific validation, and efficiency (§3.1, Table 1, pp. 14–15). [1]

Visual General Intelligence: A White Paper

Canonical arXiv record ↗ · Full paper PDF ↗ · Extracted full text ↓

Authors: Hirokatsu Kataoka, Yoshihiro Fukuhara, Yonglong Tian, Shangzhe Wu, Oishi Deb, Ryousuke Yamada, Christian Rupprecht, Jianyuan Wang, Kohsuke Ide, Koichi Namekata, Xianzheng Ma, Yiming Chen, Robert Geirhos, Aditi Raghunathan, Yuki M. Asano, Deva Ramanan, David Fouhey, Andrew J. Davison, Yilun Du, Jiajun Wu, and Zhuang Liu.

[1] Scope and plurality. §1, p. 2; §3.1 and Table 1, pp. 14–15; §4, pp. 16–17. Research perspectives and unresolved routes.

[2] Three complementary objectives. §2.10, pp. 13–14. Sequential prediction, open-ended generation, and reconstruction.

[3] Active observation. §2.7, pp. 9–10; especially “Active Visual Learning,” p. 10. Acting to obtain informative evidence.

[4] Recovering physical structure. §2.8, pp. 10–12; especially “What structure means” and “Structure at scale,” p. 11.

[5] Generative video perspective. §2.1, p. 3. Geirhos’s argument and discussion of earlier zero-shot task results.

[6] When and where to look. §2.9, pp. 12–13; especially the closing evaluation proposal, p. 13.

[7] Evaluation beyond static tasks. §3.2, p. 16, “Continual learning and evaluation.” Proposed dimensions of VGI evaluation.

Read from the complete 21-page arXiv v1 PDF (26 August 2026). Citations use PDF page numbers. All diagrams are original. No source figures are reproduced. Educational interpretation; scientific review is still required.