Optimal Federated Learning for Nonparametric Regression with Heterogeneous Distributed Differential Privacy Constraints

Abstract

This paper investigates federated learning for nonparametric regression under heterogeneous differential privacy (DP) constraints. Data is distributed across multiple servers with varying sample sizes and privacy budgets, with each server communicating privatized summaries to a central aggregator. Our work operates within the Federated Differential Privacy (FDP) framework, which generalizes both local and central DP frameworks.We establish the minimax optimal rates of convergence over Besov spaces and develop optimal distributed, wavelet-based estimators for both global and pointwise risk. Our estimators handle heterogeneous data distributions by addressing covariate shift through a two-stage procedure that includes private density estimation. We also derive minimax rates for density estimation under FDP and extend our regression estimators to unknown noise levels using adaptive procedures. Simulations and real data, including a heart disease dataset from four hospitals, illustrate these findings.Our results highlight the trade-offs between statistical accuracy and privacy, quantifying the compromise based on privacy budgets and the inherent loss from distributing data within this privacy framework. This analysis captures the common understanding that privacy is easier to retain in larger samples and explores the nuanced differences between pointwise and global estimation under distributed privacy constraints.

The authors' abstract, as published at the source. Journal of the American Statistical Association, 2026 · DOI ↗