Terrain Mesh Reconstruction

This project addresses outdoor terrain mapping using overhead images obtained from an unmanned aerial vehicle. This paper develops a joint 2D-3D learning approach to reconstruct local meshes at each camera keyframe, which can be assembled into a global environment model. Each local mesh is initialized from sparse depth measurements. We associate image features with the mesh vertices through camera projection and apply graph convolution to refine the mesh vertices based on joint 2-D reprojected depth and 3-D mesh supervision.

• Q. Feng and N. Atanasov, "Mesh Reconstruction from Aerial Images for Outdoor Terrain Mapping Using Joint 2D-3D Learning", IEEE International Conference on Robotics and Automation (ICRA), 2021
• Q. Feng and N. Atanasov, "TerrainMesh: Metric-Semantic Terrain Reconstruction from Aerial Images Using Joint 2D-3D Learning", Submitted to IEEE Transactions on Robotics (T-RO)

Close Project

Metric-Semantic Mapping for Multi-Agent Systems

We develop an online probabilistic metric-semantic mapping approach for mobile robot teams relying on streaming RGB-D observations. The generated maps contain full continuous distributional information about the geometric surfaces and semantic labels (e.g., chair, table, wall). Our approach is based on online Gaussian Process (GP) training and inference, and avoids the complexity of GP classification by regressing a truncated signed distance function (TSDF) of the regions occupied by different semantic classes. Online regression is enabled through a sparse pseudo-point approximation of the GP posterior. To scale to large environments, we further consider spatial domain partitioning via an octree data structure with overlapping leaves. An extension to the multi-robot setting is developed by having each robot execute its own online measurement update and then combine its posterior parameters via local weighted geometric averaging with those of its neighbors. This yields a distributed information processing architecture in which the GP map estimates of all robots converge to a common map of the environment while relying only on local one-hop communication.

• E. Zobeidi, A. Koppel and N. Atanasov, "Dense Incremental Metric-Semantic Mapping via Sparse Gaussian Process Regression", IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), 2020
• E. Zobeidi, A. Koppel and N. Atanasov, "Dense incremental metric-semantic mapping for multi-agent systems via sparse Gaussian process regression", Submitted to IEEE Transactions on Robotics (T-RO).

We focus on probabilistic variants of mapping due to its potential utility in down-stream tasks such as uncertaintyaware path-planning. A critical question to enabling this capability is how to process and aggregate incrementally observed local information among individual platforms, especially when their ability to communicate is intermittent. We put forth an Incremental Sparse Gaussian Process (GP) methodology for multi-robot mapping, where the regression is over a truncated signed-distance field (TSDF). Doing so permits each robot in the network to track a local estimate of a pseudo-point approximation GP posterior and perform weighted averaging of its parameters with those of its (possibly time-varying) set of neighbors. We establish conditions on the pseudo-point representation, as well as communications protocol, such that robots’ local GPs converge to the one with globally aggregated information.

• J. Di, E. Zobeidi, A. Koppel and N. Atanasov, "Distributed Gaussian Process Mapping for Robot Teams with Time-Varying Communication", American Control Conference (ACC), 2022.

Close Project

Trajectory Planning and Optimization for Minimizing Uncertainty

Many recent works in planning periodic trajectories for persistent monitoring assume that all points of interest are observable for a given periodic UAV trajectory, but we found that as the number of interest points increases (e.g., in the range of 20-80) this assumption is rarely met. The simplification of the UAV motion to a simple maximum velocity constraint ignores important, real-world constraints on the UAV dynamics. We developed a multi-objective optimization that minimizes the UAV trajectory jerk and the maximum uncertainty of the points of interest. Our algorithm leads to dynamically feasible UAV trajectories with substantially higher velocities and comparable observability to existing methods, which in turn results in decreases in the times between successive measurements and lower uncertainty of the estimates of the environmental phenomenon.

• M. Ostertag, N. Atanasov and T. Rosing, "Robust Velocity Control for Minimum Steady State Uncertainty in Persistent Monitoring Applications", American Control Conference (ACC), 2019
• M. Ostertag, N. Atanasov and T. Rosing, "Trajectory Planning and Optimization for Minimizing Uncertainty in Persistent Monitoring Applications", Springer Journal of Intelligent and Robotic Systems, 2022.

Close Project

UAV and Mobile Recharging Vehicle Rendezvous

Small UAVs equipped with sensors offer an effective way to perform high-resolution environmental monitoring in remote areas but suffer from limited battery life, requiring multiple discharge cycles to survey regions of practical importance. Our proposed method, CAR-Diff, plans trajectories for the UAV and rendezvous locations along an existing road network to minimize the time required to sense all POIs within the environment. Existing works suffer from the large computation time of genetic algorithms, which have difficulty in finding pathing solutions for 100s to 1000 POIs, or from inherent assumptions about the availability of nearby rendezvous locations, which are not valid in remote sensing applications. CAR-Diff divides a region into independent subregions aligned to existing road networks and balances the POIs within subregions using a new diffusion heuristic. The algorithm outperforms existing state-of-the-art planners for remote sensing applications, creating more fuel efficient paths that better align with the MRV travel.

• M. Ostertag, J. Ma and T. Rosing, "Remote Sensing with UAV and Mobile Recharging Vehicle Rendezvous", Initiative on Autonomous Systems Design (ASD), Design, Automation and Test in Europe Conference (DATE), 2022.

Close Project

Frequency-aware Trajectory and Power Control

This interference coordination problem in deployments of multiple unmanned aerial vehicles (UAVs) is approached through either frequency reuse or splitting the spectrum into sections reserved for individual UAVs. However, these methods are not always spectrum efficient and often do not consider other objectives such as trajectory control. Recent advancements in multi-UAV control have started to focus on coordinated trajectory and radio transmission power control or trajectory and radio frequency control, which is important in the deployment of large-scale multi-UAV sensing networks, especially when having reliable real-time connections are necessary or data intensive applications are in use. By utilizing alternating optimization of successive convex approximation and a novel adaptive band graph coloring algorithm, we achieve 27% increased network capacity over state-of-the-art UAV frequency reuse algorithms, 152% increased network capacity over state of the art trajectory and power controllers, and 135% faster control overall.

• J. Ma, M. Ostertag, D. Bharadia and T. Rosing, "Frequency-aware Trajectory and Power Control for Multi-UAV Systems", DroneCom Workshop, IEEE Conference on Computer Communications (INFOCOM), 2021.

Close Project

Sampling-Based Planning Using Graph Neural Networks

Collision checking is the major computational bottleneck in sampling-based motion planning. We developed new learning-based methods for reducing collision checking to accelerate motion planning by training graph neural networks (GNNs) that perform path exploration and path smoothing. Given random geometric graphs (RGGs) generated from batch sampling, the path exploration component iteratively predicts collision-free edges to prioritize their exploration. The path smoothing component then optimizes paths obtained from the exploration stage. The methods benefit from the ability of GNNs for capturing geometric patterns from RGGs through batch sampling and generalize better to unseen environments. Experimental results show that the learned components can significantly reduce collision checking and improve overall planning efficiency in challenging high-dimensional motion planning tasks.

• C. Yu and S. Gao, "Reducing Collision Checking for Sampling-Based Motion Planning Using Graph Neural Networks", Conference on Neural Information Processing Systems (NeurIPS), 2021.

Close Project

Neural Lyapunov Control with Stability Guarantee

We propose new methods for learning control policies and neural network Lyapunov functions for nonlinear control problems, with provable guarantee of stability. The framework consists of a learner that attempts to find the control and Lyapunov functions, and a falsifier that finds counterexamples to quickly guide the learner towards solutions. The procedure terminates when no counterexample is found by the falsifier, in which case the controlled nonlinear system is provably stable. The approach significantly simplifies the process of Lyapunov control design, provides end-to-end correctness guarantee, and can obtain much larger regions of attraction than existing methods such as LQR and SOS/SDP.

• Y. Chang, N. Roohi and S. Gao, "Neural Lyapunov Control", Conference on Neural Information Processing Systems (NeurIPS), 2019

The lack of stability guarantee restricts the practical use of learning-based methods in core control problems in robotics. We develop new methods for learning neural control policies and neural Lyapunov critic functions in the model-free reinforcement learning (RL) setting. We use sample-based approaches and the Almost Lyapunov function conditions to estimate the region of attraction and invariance properties through the learned Lyapunov critic functions. The methods enhance stability of neural controllers for various nonlinear systems including automobile and quadrotor control.

• Y. Chang and S. Gao, "Stabilizing Neural Control Using Self-Learned Almost Lyapunov Critics", IEEE International Conference on Robotics and Automation (ICRA), 2021

Close Project

Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functions

Safety and stability are common requirements for robotic control systems; however, designing safe, stable controllers remains difficult for nonlinear and uncertain models. We develop a model-based learning approach to synthesize robust feedback controllers with safety and stability guarantees. We take inspiration from robust convex optimization and Lyapunov theory to define robust control Lyapunov barrier functions that generalize despite model uncertainty. We demonstrate our approach in simulation on problems including car trajectory tracking, nonlinear control with obstacle avoidance, satellite rendezvous with safety constraints, and flight control with a learned ground effect model. Simulation results show that our approach yields controllers that match or exceed the capabilities of robust MPC while reducing computational costs by an order of magnitude.

• C. Dawson, Z. Qin, S. Gao, and C. Fan, "Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functions", Conference on Robot Learning (CoRL), 2021

Close Project