Back to my projects

Hierarchical Incremental Beta Splatting
for Online Reconstruction

Accepted at WACV 2027

Linus Franke

University of Würzburg

University of WürzburgVisual Computing Group
Paper Video Code BibTeX
top viewcurrent render
Garden (MipNeRF360 dataset), reconstructed online from the raw image stream in about a minute on one GPU, without SfM preprocessing. Left: the growing model and camera trajectory from above. Right: the model rendered from the current camera.
+0.8 dBPSNR over i3DGS on MipNeRF360 (otf)
−40%reconstruction time vs. i3DGS on MipNeRF360 (otf)
1.0–1.5 sfrom the first streamed frame to the first rendered image
+0.8 dBPSNR over i3DGS on MipNeRF360 (otf)

Abstract

Online 3D Gaussian Splatting reconstructs a scene during capture: each keyframe spawns primitives once and refines them for only a few dozen iterations, leaving little chance to fix poor initialization or missing detail. Gaussians are also inherently soft, needing many overlaps for solid surfaces.

We present Hierarchical Incremental Beta Splatting (HIBS), which brings beta kernels to incremental reconstruction. Their shape parameter interpolates between a soft, Gaussian-like profile and a flat-topped, near-opaque surfel, acting as a confidence dial for geometry. Our confidence-aware spawning uses geometric observability and multi-view depth agreement to initialize both opacity and shape, producing solid primitives where geometry is verifiable and soft, correctable primitives where it is not. As online densification, error-driven relocation recycles stale primitives into photometric-error hotspots. Bounded beta kernel support enables exact tile culling and faster rasterization, and a closed-form, shape-preserving merging formulation provides a level-of-detail (LOD) hierarchy for fast optimization and real-time rendering. On Mip-NeRF 360 in the on-the-fly protocol, HIBS improves PSNR by up to 0.8 dB while reducing optimization time by up to 40%.

Video

Method

Each frame is registered by online pose estimation. Each new keyframe then passes through four stages: confidence-aware spawning, error-driven relocation, a 30-iteration optimization burst, and periodic hierarchy merging. All primitives are beta kernels, α = o (1 − r²/9)β, with bounded support and a shape parameter β from soft (β = 5) to solid and flat-topped (β = 1).

Pipeline: input stream with online pose estimation, confidence spawn, error relocate, optimize, hierarchy; the middle three stages run once per keyframe
Stage 1

Confidence-aware spawning

New primitives are spawned where the model under-explains image detail. Two geometric cues set their confidence: depth observability cz, from the triangulation angle of the widest covisible baseline, and depth agreement cmvs between guided multi-view stereo and an aligned monocular prior. Confident spawns are solid, uncertain ones soft, so a wrong depth does not block gradients behind it.

c = (cz · cmvs)4  →  β: 5 → 1,   o: 0.1 → 0.3
Observability rises as the baseline widens; agreement rises as the stereo depth enters the monocular prior's window. The spawn changes from soft (yellow) to solid (purple).
University2 keyframe with spawn candidates coloured by observability, MVS agreement, their product, and the resulting spawn opacity and shape
A University keyframe: distant trees have low observability, dense foliage low depth agreement. Only well-constrained surfaces spawn solid.
Stage 2

Error-driven relocation

Online reconstruction has no time for clone-and-split densification. Instead, every keyframe moves near-dead primitives onto live primitives in the top 10% of the current view's photometric error.

Opacity and scale of the copies are adjusted so the render is unchanged; the following burst separates them.

A Garden keyframe: render, error map, relocation from dead sources (rings) onto the hotspots, and the error after the burst.
Stage 3

A fused 30-iteration burst

Each keyframe gets one 30-iteration burst. Bounded support gives exact tile culling, and a fused rasterizer (as in Faster-GS) applies sparse Adam inside the backward pass, so per-primitive gradients are never written to memory.

A lifetime controller hardens primitives that stay accurate across views and softens those with view-inconsistent error, so β tracks confidence over time.

The keyframe loop on Garden: confident regions spawn solid (purple), uncertain ones soft (yellow), followed by relocation, the burst, and two levels of hierarchy merging.

Per-keyframe cost on Garden (track = 480 ms). Fusing the optimizer reduces the burst from 287 to 170 ms. Over the nine otf scenes, beta kernels take 100 s, Gaussians 130 s.

Stage 4

Shape-preserving level of detail

For large captures, groups of four sub-pixel primitives are merged into a parent and the fine levels are moved to host memory. The KL-divergence merge of Gaussian hierarchies does not apply: for bounded kernels, KL is infinite once child mass lies outside the parent.

Beta kernels have closed-form mass, covariance and kurtosis, and kurtosis is invertible in β. Moment matching thus gives the parent's center, extent, opacity and shape in closed form.

Gaussiansbeta, KL-stylebeta, ours
Four children merged into one parent. KL-style merging of betas gives an undersized, too solid parent; moment matching preserves the footprint.

Live reconstruction with a confidence field

Left: the render during online reconstruction. Right: the shape field β as a confidence map. New regions spawn soft and solidify as views accumulate; reflective and poorly observed regions stay soft.

Scene
renderconfidence fieldreal time
β = 5 · soft, uncertainβ = 1 · solid, confident
render
The finished Garden model along a novel orbit.
confidence field
Its shape field along the same orbit. Most of the scene is solid; the reflective table centre stays soft.

Better throughout the capture

Mean PSNR over the test views registered so far, during reconstruction. HIBS leads i3DGS from shortly after bootstrap to the end, and earlier test views keep improving.

Mean test PSNR versus fraction of keyframes registered on MipNeRF360, TUM and StaticHikes; ours above i3DGS throughout

Comparisons

The same novel camera path, rendered from each method's reconstruction of the same input. Ours is on the left. Online: i3DGS and On-the-fly NVS. Offline references: GLOMAP poses with Beta-Splatting or Faster-GS (7K iterations).

Scene
Baseline
Ours
Paths are mapped into each method's world frame; On-the-fly NVS drifts slightly, so its viewpoint can differ.

Held-out test views

Drag to compare. Test views from the paper's comparison figures, with ground truth.

Protocol
Scene
Compare with
Oursi3DGS

City-scale, online

The hierarchy keeps kilometre-scale captures within GPU memory during reconstruction, with inactive levels in host memory. On SmallCity, a driving capture, the active model grows from 0.85 M to 1.32 M primitives at about 10 GB peak GPU memory; HIBS reaches 24.45 dB in under 4 minutes (i3DGS 23.56, offline Octree-GS 24.18 after about an hour).

CityWalk, rendered from the 31 M-node hierarchy along part of the capture trajectory.

CityWalk

A long walking capture, reconstructed online in 43 minutes.

HIBS (ours)
25.81 dB · 43 min
i3DGS
22.03 dB · 45 min
On-the-fly NVS
21.71 dB · 30 min
Octree-GS (offline)
17.33 dB · ~2.5 h
H3DGS (offline)
11.78 dB · ~20 h

A hierarchy that degrades gracefully

full detail · 12.1 M activecoarse cut · 5.6 M active
CityWalk, novel path: a coarser cut renders with 54% fewer active primitives and no visible change.
moment merge (ours)KL-style merge
The cut swept coarse and back at equal primitive budgets. KL-style parents smear into solid blobs; moment-matched parents keep footprint and shape.
moment merge (ours)KL-style merge
A fixed street view during training. Each merge event degrades the KL-style model; the moment-matched model is unaffected.

Without the moment merge, PSNR drops by 0.67 dB on SmallCity and 2.27 dB on CityWalk, which needs more hierarchy levels. Two further components keep late, low-resolution views from degrading regions that are already well reconstructed:

  • Resolution-aware weighting. Views that see a primitive at under a quarter of its finest observed resolution spawn and update it at reduced weight.
  • Error-based repair. A final 1000-iteration pass on the training views whose error grew most counters recency bias.

Results

Ours is averaged over five runs. Times are wall-clock on one RTX 5090, including pose estimation.

MethodTUMMipNeRF360 (otf)StaticHikes
PSNR↑SSIM↑LPIPS↓Time↓PSNR↑SSIM↑LPIPS↓Time↓PSNR↑SSIM↑LPIPS↓Time↓
GLOMAP + Beta-Splatting (7K)22.730.8420.2141:5027.750.8630.2324:3420.360.5380.47059:22
GLOMAP + Faster-GS (7K)23.080.8450.2071:4227.810.8760.2104:2220.570.5470.45959:38
MonoGS†16.600.6820.381~15 min14.460.4360.663~4 min15.460.3010.659~9 min
S3PO-GS w/ refinement†21.880.7770.309~1 h19.820.5530.576~20 min18.090.3680.611~50 min
LongSplat†26.510.8750.176~1.5 h23.630.6570.415~2 h20.050.4610.481~20 h
On-the-fly NVS23.370.8240.2411:0022.510.6710.3701:4521.140.6190.3382:08
i3DGS25.360.8580.1881:2426.660.8350.2242:0922.170.6990.2692:53
HIBS (ours)25.940.8680.1751:4427.470.8520.2041:1822.490.7100.2592:02
best pose-freesecond best pose-freeGrey rows: offline references (times include GLOMAP)

† Numbers from i3DGS, times roughly scaled to our setup. Reduced-resolution and 448×448 protocols (DROID-Splat, CF-3DGS, ARTDECO, AnySplat, DepthAnything3) are under Other protocols.

References

BibTeX

@inproceedings{franke2027hibs,
  title     = {Hierarchical Incremental Beta Splatting for Online Reconstruction},
  author    = {Franke, Linus},
  booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  year      = {2027}
}