Federated learning on the hardware you already own — with data that never leaves it.
FusionNet trains one shared model across cloud MI300X GPUs, gaming RX cards, Steam Decks, and CPU-only office laptops — coordinated natively on AMD ROCm, with AFLoRA adapters, DP-SGD, and zero-knowledge update verification stacked on FedAvg.
- Runs natively on AMD ROCm
- Verified zero-raw-data protocol
- Formal (ε, δ)-differential privacy
- Built for AMD Developer Hackathon ACT II
Eight things every federated framework should do — and one that only FusionNet does.
A tight system, not a stack of buzzwords. Each capability below is a real component, wired end-to-end and demonstrable on TinyLlama-1.1B today.
AFLoRA federated adapters
Clients train and exchange only low-rank LoRA A matrices. B and Lambda personalization stay local — that's what makes CPU-only participation and low-bandwidth sync possible.
Differential privacy by default
Per-sample gradient clipping and calibrated Gaussian noise, targeting (ε ≤ 1.0, δ ≤ 1e-5) per round. A persistent PrivacyAccountant tracks spend and hard-stops training when the budget is exhausted.
Zero-knowledge update verification
Clients send a cryptographic commitment before upload and reveal the nonce after. The coordinator verifies the hash and the clipping bound without ever seeing the raw gradient.
Hardware-aware contribution weighting
Every device is auto-tiered — Cloud MI300X, RX 7900 XTX, Steam Deck, CPU-only — and weighted from 2.0× down to 0.1× in aggregation so weaker devices contribute signal without destabilizing the model.
Serverless coordination via Hugging Face Hub
A private HF Hub dataset repo is the parameter server. No custom infrastructure to run. Every round is versioned, auditable, and independently verifiable through Hub logs.
Non-IID data realism
A Dirichlet-skewed partitioner gives every simulated client a genuinely different data shard — mirroring one clinic or one branch holding a rare slice. Every node prints a live ASCII partition report at startup.
LAN auto-discovery via mDNS
Edge devices find the coordinator automatically — no manual IP configuration, no static hosts file, no reverse tunnels. Bring the laptop into the room, it joins the round.
Real-time telemetry dashboard
A FastAPI + WebSocket backend streams live round status, loss curves, and privacy-budget spend to a React dashboard. Fire-and-forget: training never blocks on the dashboard being reachable.
A verifiable federated round, without a single custom server.
A private Hugging Face Hub dataset repo is the parameter server. Every round is versioned, every update is committed before it's revealed, and no raw data — labels, text, or gradients — ever crosses the network.
- STEP 01→
Train locally
Each client fine-tunes AFLoRA adapters on data that never leaves the device. Rank, batch size, and contribution weight adapt to the detected hardware tier.
- STEP 02→
Clip & noise (DP-SGD)
Per-sample gradients are clipped to norm C; calibrated Gaussian noise is added. The PrivacyAccountant records the round's ε spend on disk.
- STEP 03→
Commit (ZK)
The client sends a cryptographic hash of the update plus the clipping bound before uploading anything — the commitment.
- STEP 04→
Upload adapters
Only the LoRA A matrices are serialized to a .pt file and pushed to a private Hugging Face Hub dataset repo. B and Lambda stay local.
- STEP 05→
Reveal & verify
The client reveals the nonce. The coordinator verifies the hash and the clipping bound — proof of integrity without ever seeing the raw gradient.
- STEP 06→
Weighted FedAvg
Once enough clients have submitted, updates are aggregated by hardware tier weight (2.0× cloud → 0.1× CPU) into a new global adapter.
- STEP 07
Publish & pull
The new global adapter is published back to the Hub. Every client pulls it and the next round begins. Every round is versioned and auditable.
Why this works with zero custom servers
A private HF Hub dataset repo gives us versioned storage, auth, audit logs, and global availability — for free. The coordinator is a small polling process; the "server" is a git repo. Every artifact is signed, timestamped, and independently verifiable by anyone with read access.
Train together. Keep your data home.
The point isn't federated learning as a feature — it's what federated learning finally makes possible when it works on real hardware, under real constraints, with real proofs.
Your rarest data never has to leave your building
A clinic's rare-case scans, a law firm's client memos, a compliance team's flagged transactions — the data stays where it is. Only mathematically-noised adapter deltas are exchanged.
Every device you already own can participate
Cloud MI300X, RX 7900 XTX, Steam Deck, and CPU-only office laptops train the same shared model in the same round. Hardware tier is detected automatically at join.
Privacy that's provable, not promised
Formal (ε ≤ 1.0, δ ≤ 1e-5) differential privacy per round, cumulative budget tracked across restarts, and a zero-knowledge commit-reveal on every gradient upload.
Fault-tolerant by design
A dropped client never blocks a round. Telemetry is fire-and-forget — training keeps running even if the dashboard is unreachable. mDNS re-discovers coordinators without manual reconfiguration.
Auditable by default
Every round's global update is a versioned artifact on a private HF Hub repo. Anyone with read access can reconstruct the training history and verify commitment hashes after the fact.
Four privacy layers on top of FedAvg — and every one of them is checkable.
Federated learning alone isn't enough. FusionNet stacks differential privacy, cryptographic verification, private-hub coordination, and RCCL-ready secure aggregation so that privacy is a proof, not a promise.
Formal (ε, δ)-differential privacy
In plain language: no single record in a client's dataset can be detected from the model's behavior. FusionNet targets (ε ≤ 1.0, δ ≤ 1e-5) per round — a strong, quantitative privacy guarantee, not a marketing adjective.
Gaussian mechanism, clipped per sample
Every per-sample gradient is clipped to norm C, then calibrated Gaussian noise is added. This is the standard DP-SGD construction — and FusionNet ships a dual engine (Opacus primary, custom fallback for 4-bit quantized modules Opacus hooks can't attach to on ROCm).
Persistent privacy accounting
A PrivacyAccountant tracks cumulative ε across rounds and survives restarts on disk. When the budget is exhausted, training hard-stops. There is no way to silently overspend.
Zero-knowledge commit-then-reveal
Before upload, a client commits a hash of the update and the clipping bound. After upload, it reveals the nonce. The coordinator verifies both — proving the update wasn't tampered with post-commit, without ever seeing the gradient.
Only serialized tensors cross the network
No text, no labels, no raw gradients. What leaves the device is a .pt file of noised LoRA A matrices. Independently verifiable via HF Hub audit logs or packet capture.
Written for engineers who read the source.
Small surfaces, honest contracts, and interfaces that stay put while the transport underneath swaps out. The kind of API you can extend without asking permission.
Stable message contracts across every transport
ClientUpdate and GlobalUpdate stay the same from local simulation → HTTP → HF Hub → RCCL. Swap transports without touching training code.
Clean ownership boundaries
Comms, backend, frontend, and ML layers are independently ownable. Teams can move in parallel without stepping on each other's contracts.
Structured logging, fail-fast config
Pydantic-validated config with startup checks that surface misconfigured runs before they burn a GPU-hour. Structured logs, never bare prints.
Dual-engine differential privacy
Opacus is the primary DP engine; a custom fallback engine handles 4-bit quantized modules Opacus hooks can't attach to on ROCm — no silent gaps in the guarantee.
# start a round on any transportround_id = comms.start_round(config=cfg, global_state=state)# each client submits its noised adapter deltacomms.submit_update( round_id, ClientUpdate( client_id=me, adapter=noised_lora_A, # only A matrices leave commitment=commit_hash, # zero-knowledge pre-commit clip_norm=C, eps_spent=accountant.step(), ),)# coordinator aggregates by hardware tier weightglobal_update = comms.aggregate( round_id, weights=tier_weights, # 2.0× MI300X → 0.1× CPU)Precise commitments. Not inflated ones.
Every figure below traces back to an actual component in the framework — nothing rounded up, nothing invented.
Formal differential privacy target per round
with δ ≤ 1e-5, tracked cumulatively
Device tiers supported end-to-end
Cloud MI300X → RX 7900 XTX → Steam Deck → CPU-only
Bytes of raw data transmitted per round
only noised LoRA A matrices ever leave the device
Privacy layers stacked on FedAvg
AFLoRA · DP-SGD · ZK proofs · MPC-ready RCCL
Answers judges (and skeptical engineers) actually ask.
Because the pitch is 'any device, any office, participates.' A Llama-3-8B demo that only runs on GPU nodes would contradict that. TinyLlama shares the same Llama transformer architecture, so all AFLoRA injection and aggregation code is identical — the exact same system scales to larger models on GPU-only cohorts with zero code changes.
Not in the same round — federated averaging requires a shared base model. What clients do differ on is LoRA rank, batch size, and contribution weight, all derived from the detected hardware tier. That's how a Steam Deck and an MI300X train the same TinyLlama-1.1B together.
That's exactly what the contribution-weighting system is for. A CPU-only laptop's update is weighted 0.1× in aggregation while a cloud MI300X is 2.0×. Weaker devices still contribute their rare, non-transferable data signal, but they can't destabilize the global model.
Only serialized .pt files containing noised LoRA A matrices, plus a small commitment hash. No text, no labels, no raw gradients, no B or Lambda personalization matrices. You can verify this from an HF Hub audit trail or a packet capture.
FusionNet targets (ε ≤ 1.0, δ ≤ 1e-5) per round using DP-SGD — per-sample gradient clipping plus calibrated Gaussian noise. A persistent PrivacyAccountant composes ε across rounds and hard-stops training if the budget is exhausted. The guarantee is a formal one from the differential privacy literature, not a policy.
Three layers. Gradient clipping bounds any single update's influence. The zero-knowledge commit-then-reveal proves an update wasn't altered post-commit. Contribution weighting caps low-trust hardware. RCCL-based secure aggregation is the next step for cryptographic protection against a compromised coordinator.
Train together. Keep your data home.
FusionNet is open source, ROCm-native, and built to be extended. Read the architecture, run a round on your own hardware, and see for yourself.