Cross-Scene 3D Semantic Occupancy Prediction

OccAnyScene

Towards Unified Indoor-Outdoor 3D Occupancy Prediction

Junjie Liu1,* Wanshui Gan2,* Zitong Dai3 Guiping Cao4 Yan Li4
Ke Chen4 Dongmei Jiang4 Xiangyuan Lan4 Jianguo Zhang1

1Southern University of Science and Technology 2Shanghai Artificial Intelligence Laboratory 3Harbin Institute of Technology, Shenzhen 4Pengcheng Laboratory

* Equal contribution.

arXiv Code

One shared model predicts semantic occupancy across room-scale indoor scenes and street-scale outdoor environments with heterogeneous cameras, voxel grids, and semantic taxonomies.

Demo

From indoor rooms to outdoor streets

A single OccAnyScene model handles heterogeneous indoor and outdoor occupancy protocols without switching models.

Overview

One representation for different scene scales

Existing occupancy models are usually specialized to one camera setup, spatial range, voxel resolution, and taxonomy. OccAnyScene instead uses the pixel frustum as a scene-adaptive geometric unit.

Abstract

3D semantic occupancy prediction is fundamental to scene understanding, yet existing methods are typically specialized to fixed scene types and occupancy protocols. Such specialization is poorly suited to autonomous systems operating across environments with different spatial demands: open roads require long-range, coarse-grained perception, whereas confined indoor spaces demand short-range, fine-grained understanding. Maintaining and switching between separate models for each setting complicates deployment and scales poorly as operating environments diversify. We therefore introduce Cross-Scene 3D Semantic Occupancy Prediction, a task that requires one model to handle heterogeneous indoor and outdoor scenes with different camera configurations, spatial ranges, voxel specifications, and semantic taxonomies. To address the challenge of metric-consistent yet scene-adaptive image-to-3D lifting, OccAnyScene builds a continuous Gaussian representation upon a pretrained depth foundation model, combining Pixel-Aligned Frustum Feature Aggregation with Frustum-Parameterized Gaussian Construction.

OccAnyScene unifies indoor room-scale and outdoor street-scale occupancy prediction.
One shared model supports both fine-grained room-scale prediction and large-range street-scale prediction.
01

Cross-scene learning

Jointly learns indoor and outdoor datasets under their native protocols.

02

Camera-aware aggregation

Builds a context-rich frustum query for every feature pixel.

03

Scene-adaptive Gaussians

Adapts Gaussian positions and scales to camera geometry and scene depth.

Method

A pixel-frustum-centered Gaussian framework

PFFA aggregates geometry and visual context into frustum queries. FPGC decodes those queries into Gaussian primitives whose centers and scales are expressed relative to the corresponding pixel frusta.

Architecture of OccAnyScene, including PFFA, FPGC, and Gaussian-to-voxel splatting.
OccAnyScene aggregates pixel-aligned features into frustum queries, constructs scene-adaptive Gaussian primitives, and maps them to each dataset's taxonomy.

Results

Competitive accuracy with one shared model

With DAv3 as the image encoder, one shared cross-scene model closely matches separately trained scene-specific models on both indoor and outdoor benchmarks, with only 0.41 and 0.19 mIoU gaps, respectively.

Indoor benchmark

Occ-ScanNet

Monocular
Scene-specific 59.92% mIoU IoU 68.34%
Cross-scene One shared model 59.51% mIoU IoU 67.96%

Only -0.41 mIoU with joint indoor-outdoor training

Outdoor benchmark

SurroundOcc-nuScenes

6 cameras
Scene-specific 23.06% mIoU IoU 35.97%
Cross-scene One shared model 22.87% mIoU IoU 35.83%

Only -0.19 mIoU with joint indoor-outdoor training

Qualitative comparisons on Occ-ScanNet and SurroundOcc-nuScenes.
Qualitative comparison across Occ-ScanNet and SurroundOcc-nuScenes. The shared OccAnyScene model preserves both indoor structures and outdoor layouts.

Occlusion reasoning

Gaussians extend beyond visible surfaces

Surface-relative depth increments move selected Gaussians behind foreground occluders, enabling the model to represent both visible and inferred scene content.

Outdoor multi-camera depth increments and semantic occupancy prediction.

Occlusion-aware Gaussian positioning on SurroundOcc-nuScenes. The six columns correspond to the six camera views. The rows show (a) input images, (b) predicted surface depth, (c) the depth of the closest-to-camera Gaussian among the K = 3 Gaussians decoded from each pixel frustum, and (d) the corresponding surface-relative depth increment. Row (c) is obtained by adding (d) to (b), while the right panel shows the final semantic occupancy prediction. Red boxes highlight representative foreground occlusions, where spatially alternating large increments move Gaussians behind visible surfaces.

Efficiency

A practical accuracy–efficiency trade-off

OccAnyScene-DAv2 combines the smallest parameter count among the compared methods with low inference latency and memory usage.

Parameter count, inference time, and memory usage comparison.

Citation

Cite OccAnyScene

If you find this work useful, please consider citing the paper.

@article{chen2026occanyscene,
  title   = {OccAnyScene: Towards Unified Indoor-Outdoor 3D Occupancy Prediction},
  author  = {Junjie, Liu and Wanshui, Gan and Zitong, Dai and Guiping Cao and Yan Li and Ke Chen and Dongmei Jiang and Xiangyuan Lan and Jianguo Zhang},
  journal = {arXiv preprint},
  year    = {2026}
}