Learning Scene-Level Signed Directional Distance Function with Ellipsoidal Priors and Neural Residuals

Zhirui Dai1, Hojoon Shin1, Yulun Tian1, Ki Myung Brian Lee1, Nikolay Atanasov1
1Existential Robotics Lab, University of California San Diego
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2026.
Paper arXiv BibTeX Code
Teaser Figure

Comparison of our method vs existing works for predicting Signed Directional Distance Function (SDDF). (a), (c): We present a method to learn scene-level signed directional distance function (SDDF). (a), (b), (d): Our method uses ellipsoids as an initial coarse approximation of the shapes of objects in the environment. (e), (f): The ellipsoid prior is refined by a latent feature network and a shared decoder to predict the surface reconstruction residual. (f), (g): Our SDDF learning method offers single-query differentiable novel distance image synthesis without RGB supervision as an alternative to Gaussian Splat distance rendering (e.g., RaDe-GS) or signed distance function sphere tracing (e.g., InstantNGP).

Abstract: Dense differentiable environment representations are critical for navigation and exploration by aerial robots. In this work, we explore a novel implicit scene representation, the Signed Directional Distance Function (SDDF), to enhance geometry modeling and differentiable trajectory optimization. Unlike the standard signed distance function (SDF) and similar to neural radiance fields (NeRF), SDDF takes both a position and a viewing direction as input. However, unlike NeRF, SDDF directly provides the distance to the observed surface along the viewing direction. This formulation allows for efficient view synthesis without the computationally expensive iterative ray marching required by other implicit representations. To learn and predict scene-level SDDF efficiently, we develop a differentiable hybrid representation that combines explicit ellipsoid priors with implicit neural residuals. This approach allows the model to effectively handle large distance discontinuities around obstacle boundaries while preserving the ability for dense, high-fidelity prediction. We demonstrate that SDDF is competitive with state-of-the-art neural implicit scene models in terms of reconstruction accuracy and rendering efficiency, while enabling differentiable view prediction for robot trajectory optimization.

Methodology

Signed Directional Distance Function

Traditional SDFs map a spatial coordinate to the distance of the nearest surface, requiring sphere tracing to render views. We introduce SDDF, which maps a coordinate and a ray direction directly to the distance along that ray. This eliminates the need for iterative stepping during rendering, significantly accelerating the process for real-time aerial applications.

2D Scene Example 2D SDDF Example

Hybrid Representation

Hybrid Representation

Method overview. Given a query ray from position $\mathbf{p}\in\mathbb{R}^3$ in direction $\mathbf{v}\in\mathbb{S}^2$, an ellipsoid-based Prior network $P$ uses $M$ ellipsoids $\{\mathbf{\xi}_i,\mathbf{r}_i\}_{i=1}^M$ to learn the rough shape of the environment such that it can determine the closest ellipsoid intersected by the ray and predict an SDDF prior. Then, with the intersection point $\mathbf{q}\in\mathbb{R}^3$ and ray direction $\mathbf{v}'\in\mathbb{S}^2$ in the ellipsoid's local frame, a Latent network $L$ generates a latent feature $\mathbf{z}\in\mathbb{R}^m$, which is decoded by the Residual decoder $R$ into residual predictions $\left(\delta_i,\delta_s,\delta_f\right)$, i.e. the difference between the groundtruth and the prior. Finally, we compose the SDDF prediction as $\hat{f}=f+\delta_f$. Blue arrows show the data flow in the forward pass, while red arrows represent the backward pass.

To model complex scenes, we employ a hybrid architecture. Explicit ellipsoid priors capture the coarse geometry and large discontinuities of obstacles. Implicit neural residuals are then learned to refine the geometry and capture high-fidelity surface details. This combination ensures both robustness to boundaries and high precision.

SDDF Prediction by Our Method

SDDF Prediction Example

Qualitative comparison of SDDF predictions. Row 1: Replica Hotel (synthesized). Row 2: Gibson Allensville (synthesized). Row 3: ScanNet scene 0000-00 (real). In areas with limited sensor measurements, RaDe-GS fails to learn the geometry, with RGB (c) or without RGB (d), yielding artifacts. Nerfacto in e) shows significant artifacts and large distance prediction error. SDF-Instant-NGP in f) tends to learn a smoother approximation, missing sharp boundaries

Differentiable Trajectory Optimization

A key advantage of our representation is its differentiability. Because SDDF allows for direct distance querying, we can compute gradients of the view rendering loss with respect to the robot's state. This enables differentiable trajectory optimization, allowing an aerial robot to plan paths that maximize information gain or minimize collision risk based on the predicted scene geometry.

View Optimization Example

More Examples of Trajectory Optimization:

Gibson Scene Traj Optimization Replica Hotel Scene Traj Optimization Replica Office3 Scene Traj Optimization

BibTeX

Our TPAMI Paper:

@misc{dai2025sddf,
        title = {{Learning Scene-Level Signed Directional Distance Function with Ellipsoidal Priors and Neural Residuals}},
        author = {Zhirui Dai and Hojoon Shin and Yulun Tian and Ki Myung Brian Lee and Nikolay Atanasov},
        year = {2025},
        eprint = {2503.20066},
        archivePrefix = {arXiv},
        primaryClass = {cs.RO},
        url = {https://arxiv.org/abs/2503.20066},
}

Our Workshop Paper at RSS 2025 (Best Paper Award):

@inproceedings{Dai_SDDF_RSS25,
        author = {Zhirui Dai and Hojoon Shin and Yulun Tian and Ki Myung Brian Lee and Nikolay Atanasov},
        title = {{Learning Scene-Level Signed Directional Distance Function for Aerial Autonomy}},
        booktitle = {Workshop on Leveraging Implicit Methods for Aerial Autonomy at RSS},
        year = {2025}
        url = {https://daizhirui.github.io/assets/pdfs/ref/SDDF_RSS25.pdf}
}