designing-federated-learning-architecture

Designing Federated Learning Architecture

Overview

Federated Learning (FL) enables multiple parties to collaboratively train a machine learning model without sharing their raw data. Each participant trains a local model on their own data and shares only model updates (gradients or parameters) with a central aggregator. The aggregated model benefits from all participants' data without any single party accessing another's dataset.

FL directly supports GDPR Article 5(1)(c) data minimization (only model updates are shared, not personal data), Article 25(1) data protection by design (privacy is built into the architecture), and can reduce the need for cross-border data transfers under Chapter V (data stays in its jurisdiction of origin).

FL Architectures

Horizontal Federated Learning

Participants share the same feature space but have different data samples. Each participant has a complete record for their subjects but covers different subjects.

┌──────────────────────────────────────────────────────────────┐
│                    Central Aggregator                          │
│               (aggregates model updates only)                 │
│                                                                │
│  Global Model = Aggregate(Local_1, Local_2, ..., Local_K)    │
└─────────────┬──────────────┬──────────────┬──────────────────┘
              │              │              │
    ┌─────────▼────┐ ┌──────▼──────┐ ┌────▼──────────┐
    │ Hospital A   │ │ Hospital B  │ │ Hospital C    │
    │ 5,000 pts    │ │ 8,000 pts   │ │ 3,000 pts     │
    │ Same features│ │ Same features│ │ Same features │
    │ Train locally│ │ Train locally│ │ Train locally │
    └──────────────┘ └─────────────┘ └───────────────┘

Use case: Multiple hospitals training a diagnostic model; each hospital has complete patient records but different patients.

Vertical Federated Learning

Participants share the same data subjects but have different features. Features are distributed across participants.

┌──────────────────────────────────────────────────────────────┐
│                    Coordination Server                         │
│         (aligns features without exposing raw data)           │
└─────────────┬──────────────┬──────────────┬──────────────────┘
              │              │              │
    ┌─────────▼────┐ ┌──────▼──────┐ ┌────▼──────────┐
    │ Bank         │ │ Insurer     │ │ Retailer      │
    │ Financial    │ │ Claims      │ │ Purchase      │
    │ features     │ │ features    │ │ features      │
    │ Same users   │ │ Same users  │ │ Same users    │
    └──────────────┘ └─────────────┘ └───────────────┘

Use case: Credit scoring where a bank has financial data, an insurer has claims history, and a retailer has purchase behavior — all for overlapping customers.

Aggregation Strategies

FedAvg (Federated Averaging)

The foundational FL algorithm (McMahan et al., 2017). Each participant trains for multiple local epochs, then the aggregator averages the model weights proportional to each participant's dataset size.

For each communication round t:
  1. Server sends global model w_t to selected participants
  2. Each participant k trains locally for E epochs on their data:
     w_k = LocalSGD(w_t, data_k, E epochs, learning_rate η)
  3. Server aggregates:
     w_{t+1} = Σ (n_k / n) * w_k
     where n_k = participant k's sample count, n = total samples

FedProx (Federated Proximal)

Extension of FedAvg for heterogeneous settings. Adds a proximal term to the local objective to limit how far local models drift from the global model. Suitable when participants have non-IID (non-independently and identically distributed) data.

Local objective for participant k:
  minimize F_k(w) + (μ/2) * ||w - w_t||^2

  where μ controls the strength of the proximal regularization

Privacy Guarantees

Threat Model

Secure Aggregation

Secure aggregation ensures the server sees only the sum of all participants' updates, not individual updates. Implemented using:

1. Pairwise masking (Bonawitz et al., 2017) — Each pair of participants generates a shared random mask. Masks cancel out in the sum, but individual updates remain hidden.
2. Threshold secret sharing — Each participant secret-shares their update; the server reconstructs only the aggregate.
3. Homomorphic encryption — Participants encrypt updates; server aggregates ciphertexts.

Differential Privacy in FL

Adding calibrated noise to model updates before sharing provides a formal privacy guarantee:

DP-FedAvg:
  1. Each participant clips gradients: ||g_k|| ≤ C (sensitivity bounding)
  2. Each participant adds Gaussian noise: g_k' = g_k + N(0, σ²·C²·I)
  3. Server aggregates noised updates: g_agg = (1/K) Σ g_k'
  4. Privacy accounting: track cumulative (ε, δ) via Rényi DP composition

Privacy budget for Prism Data Systems AG FL deployment:

Communication Efficiency

Implementation Workflow

1. Define the ML task — Specify the model architecture, training objective, and evaluation metrics.

1. Assess data distribution — Evaluate the degree of non-IID across participants. This determines the aggregation strategy (FedAvg for IID, FedProx for non-IID).

1. Select privacy mechanisms — Choose secure aggregation for gradient confidentiality and differential privacy for formal guarantees. Set epsilon budget.

1. Configure communication — Select gradient compression technique based on bandwidth constraints. Set the communication round budget.

1. Implement local training — Deploy the local training pipeline at each participant. Implement gradient clipping and noise injection for DP.

1. Deploy aggregation server — Deploy the central aggregator with secure aggregation protocol. Implement Byzantine-robust aggregation if untrusted participants exist.

1. Monitor and evaluate — Track model convergence, per-participant contribution quality, privacy budget consumption, and communication overhead.

Key Regulatory References

• GDPR Article 5(1)(c) — Data minimization (only model updates shared)
• GDPR Article 25(1) — Data protection by design
• GDPR Article 26 — Joint controller arrangements (FL consortium governance)
• GDPR Chapter V — Cross-border transfers (FL reduces transfer need)
• GDPR Article 35 — DPIA for FL involving special category data
• EDPB Guidelines 4/2019 on Article 25 Data Protection by Design and by Default