ECCV 2026

FeDepth: Federated Learning for Depth Estimation under Robot Heterogeneity

Ganghyeon Lee*, Inha Lee*, Junhee Lee, Jeongeon Lee, Sungwhan Yoon, Kyungdon Joo†
Ulsan National Institute of Science and Technology, Ulsan, Korea
*Equal contribution. †Corresponding author.
Paper Coming soon Code Coming soon BibTeX Pending
FeDepth soft clustering illustration for federated depth estimation
FeDepth models complex inter-client relationships with soft cluster assignments, allowing related robot clients to contribute to multiple cluster models.

Abstract

Federated learning enables distributed training without transferring raw robot perception data, but depth estimation under real robotic deployments faces severe domain shifts across clients. In practice, client distributions often overlap across platforms, environments, and sensing conditions, which breaks the clearly separated client-domain assumption used by conventional clustered federated learning.

FeDepth is a descriptor-based clustered federated learning framework for monocular depth estimation. It models client relationships through soft clustering, allowing clients to participate in multiple clusters and capture continuous, ambiguous domain transitions observed in heterogeneous robotic environments.

Motivation

Centralized learning requires collecting high-bandwidth sensory data at a server, creating communication, scalability, and privacy challenges for real-world robot fleets. Standard federated learning avoids raw-data transfer, but a single global model struggles when clients differ by robot platform, viewpoint, motion dynamics, environment, camera parameters, and depth range.

41
Training clients in the HPE scenario.
11
Unseen HPE test sequences.
32
BMR clients: 15 NYUv2 and 17 KITTI.

Robot Heterogeneity Scenarios

HPE and BMR robot heterogeneity scenario overview
HPE introduces shifts across robot platforms and environments using M3ED. BMR reflects bi-modal depth distributions between indoor NYUv2 and outdoor KITTI clients.
Depth distribution and UMAP visualization for FeDepth scenarios
The proposed scenarios expose both overlapping client relationships and strongly separated depth-range distributions.

FeDepth Overview

Overall framework of FeDepth
FeDepth first extracts client descriptors and builds a soft cluster map, then performs cluster-wise model updates and assigns unseen sequences to relevant cluster models at inference.

Descriptor-Based Soft Clustering

FeDepth descriptor extractor
Keyframes are encoded by a frozen backbone and aggregated with GeM pooling. The descriptor mean and variance summarize each client's local data distribution.
0.264
Hard clustering Abs Rel on HPE with NeWCRFs.
0.249
Soft clustering Abs Rel under the same setup.
20%
Best reported overlap ratio in the HPE ablation.

Quantitative Results

Scenario Model FedAvg PACFL† FeDepth
HPE NeWCRFs 0.366 0.318 0.249
HPE DCDepth 0.351 0.321 0.293
BMR DCDepth 0.158 0.082 0.082

Values report Abs Rel, where lower is better. PACFL† denotes the descriptor-based PACFL variant used in the paper.

Qualitative Results

Qualitative depth prediction results on the HPE scenario
Qualitative DCDepth results on the HPE scenario compare input, ground truth, predicted depth maps, and RMSE error maps for FeDepth and FL/CFL baselines.

Cluster Map and Convergence

Cluster map and convergence graph for FeDepth
Cluster maps visualize client assignments in HPE and BMR, and convergence curves report RMSE log over communication rounds for NeWCRFs.

Citation

Publication pending. BibTeX will be added after the paper is public.