Incoherent Deformation, Not Capacity

Abstract

Dynamic 3D Gaussian Splatting achieves impressive novel-view synthesis on monocular video by coupling a deformable point cloud with Adaptive Density Control (ADC), but exhibits a severe train–test generalization gap. On the D-NeRF benchmark (8 synthetic scenes) we measure an average gap of 6.18 dB (up to 11 dB per scene) and, through a systematic ablation of every ADC sub-operation (split, clone, prune, frequency, threshold, schedule), identify splitting as the bottleneck of the overfitting cascade — disabling split eliminates the gap (1.15 dB) but also collapses test PSNR by 9.93 dB, so it is not a viable mitigation. Split is the operation through which the cascade flows, not a knob one can simply turn off.

Our central finding is that a local smoothness penalty on the per-Gaussian deformation field — we use a k-NN strain prior we call EER — breaks the count–gap correlation observed across ablations: it reduces the gap by 40.8% while growing the cloud by 85%. This reframes overfitting from a capacity problem to an incoherent deformation problem. A controlled ablation against E-D3DGS-style per-embedding smoothness and an SC-GS-style ARAP residual shows the three normalized variants achieve statistically tied gap reductions (47.5% / 46.1% / 40.4%); dropping the canonical-distance normalization disables the prior entirely (+2.2%). The substantive contribution is therefore not a new method but the diagnostic finding plus the identification that the canonical-distance normalization is the load-bearing element of these priors. Combined with GAD (a loss-rate-aware densification threshold), the recommended configuration GAD+EER closes 48.2% of the gap; adding PTDrop (jitter-weighted dropout) and a soft cloud cap reaches 57.4% at larger quality cost.

Findings are validated on D-NeRF (8 synthetic scenes), Deformable-3DGS (cross-architecture), and HyperNeRF (5 real-world scenes; +16.1% gap reduction on the high-baseline-gap subset, neutral on low-baseline scenes). EER's k-NN cost scales with cloud size; approximate-NN structures are needed to scale beyond ~100K Gaussians on consumer hardware.

Method Taxonomy

Two small drop-in methods, each one hyperparameter and about 20 lines of code:

We also tried spectral-gated densification, temporal Sobolev smoothness, SH-coefficient penalties, and opacity-entropy maximization (SGD / STSR / ChromReg / OEM). At our scale none moves the gap by more than 10%, so the v2 paper documents them as negative results rather than first-class methods; the v1 paper (PDF, companion main_v1.tex) has the full taxonomy for reference.

GAD: a BIC-motivated threshold schedule

We adapt the per-iteration gradient threshold as

where K(t) is the current count, N is the number of training pixels, and Δℓ_ema is an EMA of the per-iteration loss improvement. λ is the single tunable knob. The mapping from BIC to this formula is a heuristic (see paper, §6.2); the empirical diminishing-returns exponent we measure (α ≈ 0.04) is too mild to justify the often-quoted O((N/λ)^1/4) growth bound, so we present the bound qualitatively as "sublinear in N".

EER: k-NN elastic strain energy

For a subset of Gaussians i and their k=8 canonical neighbors j, we penalize

where u(x, t) is the deformation offset at time t. This is the discrete elastic strain — physically the correct choice for linear elasticity (Hooke's law penalizes ∂u/∂x, not ∂u). In canonical space the k-NN graph is stable; we rebuild it every 500 iterations and apply a cosine ramp from iteration 3K to 10K.

Cross-Architecture Validation (Deformable-3DGS)

Main experiments are on 4DGS (HexPlane deformation). We ported EER and GAD to Deformable-3DGS (MLP deformation) and ran baseline + EER on three D-NeRF scenes for 20K iterations.

Phase 1: direct-transfer test at D-NeRF-tuned λ=0.05

Direct transfer at λ=0.05 is poor (mean −6% reduction). Why? Deformable-3DGS trains with L1+0.2·(1−SSIM) vs.\ 4DGS's pure L1 — the loss magnitude is roughly 3× larger and λ=0.05 is therefore under-regularized. Our dimensional-analysis note (paper §6.2) predicts the correct λ for Deformable-3DGS is ≈ 0.15–0.30. Testing this directly:

Phase 2: λ sweep on Deformable-3DGS Lego (dimensional-analysis test)

Cross-scene confirmation at λ=0.30

To confirm the sweep is not Lego-specific, we replicated λ=0.30 on Hellwarrior:

The coherence mechanism transfers across deformation architectures and across scenes; the hyperparameter requires per-architecture (and to a lesser extent per-scene) calibration, exactly as the dimensional-analysis note predicted. On Hellwarrior the quality cost is larger at λ=0.30 (−2.44 dB); a smaller λ like 0.05 already gives +5.2% gap reduction at only −0.22 dB.

BibTeX

@article{droby2026monodygs,
  author  = {Ahmad Droby},
  title   = {Incoherent Deformation, Not Capacity: Diagnosing and
             Mitigating Overfitting in Dynamic Gaussian Splatting},
  journal = {arXiv preprint},
  year    = {2026}
}