Exchange codebooks to make federated communication smaller
Send the shared values more often than their positions. FedCode usually exchanges only weight-cluster codebooks. Clients and server map their existing weights to received centers, while periodic transfers of weight indices recalibrate the models.
Codebook exchange: omit repeated weight-index transmission in ordinary upstream and downstream rounds.
Nearest-center updates: reuse stored model structure when receiving codebooks.
Periodic calibration: restore weight assignments by occasionally sending centers plus indices.
Federated Learning (FL) is a distributed machine learning paradigm that enables learning models from decentralized local data. While FL offers appealing properties for clients' data privacy, it imposes high communication burdens for exchanging model weights between a server and the clients. Existing approaches rely on model compression techniques, such as pruning and weight clustering to tackle this. However, transmitting the entire set of weight updates at each federated round, even in a compressed format, limits the potential for a substantial reduction in communication volume. We propose FedCode where clients transmit only codebooks, i.e., the cluster centers of updated model weight values. To ensure a smooth learning curve and proper calibration of clusters between the server and the clients, FedCode periodically transfers model weights after multiple rounds of solely communicating codebooks. This results in a significant reduction in communication volume between clients and the server in both directions, without imposing significant computational overhead on the clients or leading to major performance degradation of the models. We evaluate the effectiveness of FedCode using various publicly available datasets with ResNet-20 and MobileNet backbone model architectures. Our evaluations demonstrate a 12.2-fold data transmission reduction on average while maintaining a comparable model performance with an average accuracy loss of 1.3% compared to FedAvg. Further validation of FedCode performance under non-IID data distributions showcased an average accuracy loss of 2.0% compared to FedAvg while achieving approximately a 12.7-fold data transmission reduction.
Abstract and overview follow the public preprint; the citation identifies the published version.
Read more ↗With P weights, K centers and b bits per center, ordinary clustering still sends one index per weight. FedCode targets that repeated index cost.
Equation 3, PDF page 2. b abbreviates the source’s wordlength; this is a representation-size expression.
Download a sorted codebook; map each local weight to its nearest center; train locally. One K-means run over the entire local model then produces the upload codebook.
Section 4.1 and Algorithm 2, PDF pages 5–6. Downstream mapping is O(P log K); K-means is O(PKI), with I iterations. These are complexity claims, not device timings.
Concatenate received client codebooks and map each existing global weight to its nearest received center. This operation changes the ordinary server update; it does not average complete client tensors.
Section 4.1 and Algorithm 1, PDF page 5. Global weights are clustered again to prepare the next downlink.
Send centers plus indices to reconstruct the sender’s clustered model. The server uses FedAvg after upstream reconstruction. Calibration frequency controls communication and accuracy together.
Section 4.2, PDF page 5. The v1 preprint assigns F1/F2 to directions inconsistently, so the result panel retains those source labels without relabeling them.
CIFAR-100 · ResNet-20 · 10 clients · full participation · 100 rounds
CIFAR-100 · ResNet-20 · 64 centers · upstream / downstream calibration
| Calibration F1 / F2 | Accuracy Δ (pp) | Joint DTR (×) |
|---|---|---|
| 1 / 1 | -0.86 | 5.3 |
| 0.5 / 0.5 | -3.46 | 10.4 |
| 0.2 / 0.5 | -3.15 | 14.6 |
| 0.33 / 0.33 | -5.94 | 15.2 |
| 0.1 / 0.1 | -21.64 | 44.3 |
| 0 / 0 | -46.88 | 264.7 |
@inproceedings{khalilian2024fedcode,
title = {{FedCode}: Communication-Efficient Federated Learning via Transferring Codebooks},
author = {Khalilian, Saeed and Tsouvalas, Vasileios and Ozcelebi, Tanir and Meratnia, Nirvana},
booktitle = {2024 IEEE International Conference on Edge Computing and Communications (EDGE)},
year = {2024},
pages = {99--109},
publisher = {IEEE},
doi = {10.1109/EDGE62653.2024.00022},
url = {https://doi.org/10.1109/EDGE62653.2024.00022}
}