Exchange codebooks to make federated communication smaller
Send the shared values more often than their positions. FedCode usually exchanges only weight-cluster codebooks. Clients and server map their existing weights to received centers, while periodic transfers of weight indices recalibrate the models.
Codebook exchange: omit repeated weight-index transmission in ordinary upstream and downstream rounds.
Nearest-center updates: reuse stored model structure when receiving codebooks.
Periodic calibration: restore weight assignments by occasionally sending centers plus indices.
Federated Learning (FL) is a distributed machine learning paradigm that enables learning models from decentralized local data. While FL offers appealing properties for clients' data privacy, it imposes high communication burdens for exchanging model weights between a server and the clients. Existing approaches rely on model compression techniques, such as pruning and weight clustering to tackle this. However, transmitting the entire set of weight updates at each federated round, even in a compressed format, limits the potential for a substantial reduction in communication volume. We propose FedCode where clients transmit only codebooks, i.e., the cluster centers of updated model weight values. To ensure a smooth learning curve and proper calibration of clusters between the server and the clients, FedCode periodically transfers model weights after multiple rounds of solely communicating codebooks. This results in a significant reduction in communication volume between clients and the server in both directions, without imposing significant computational overhead on the clients or leading to major performance degradation of the models. We evaluate the effectiveness of FedCode using various publicly available datasets with ResNet-20 and MobileNet backbone model architectures. Our evaluations demonstrate a 12.2-fold data transmission reduction on average while maintaining a comparable model performance with an average accuracy loss of 1.3% compared to FedAvg. Further validation of FedCode performance under non-IID data distributions showcased an average accuracy loss of 2.0% compared to FedAvg while achieving approximately a 12.7-fold data transmission reduction.
Abstract and overview follow the public preprint; the citation identifies the published version.
Read more ↗With P weights, K centers and b bits per center, ordinary clustering still sends one index per weight. FedCode targets that repeated index cost.
Equation 3, PDF page 2. b abbreviates the source’s wordlength; this is a representation-size expression.
Download a sorted codebook; map each local weight to its nearest center; train locally. One K-means run over the entire local model then produces the upload codebook.
Section 4.1 and Algorithm 2, PDF pages 5–6. Downstream mapping is O(P log K); K-means is O(PKI), with I iterations. These are complexity claims, not device timings.
Concatenate received client codebooks and map each existing global weight to its nearest received center. This operation changes the ordinary server update; it does not average complete client tensors.
Section 4.1 and Algorithm 1, PDF page 5. Global weights are clustered again to prepare the next downlink.
Send centers plus indices to reconstruct the sender’s clustered model. The server uses FedAvg after upstream reconstruction. Calibration frequency controls communication and accuracy together.
Section 4.2, PDF page 5. The v1 preprint assigns F1/F2 to directions inconsistently, so the result panel retains those source labels without relabeling them.
Results from arXiv:2311.09270v1 ↗
FedCode reports 14.6× joint reduction at 58.07% accuracy. FedZip’s 19.8× upload reduction becomes 1.9× when the download cost is included.
CIFAR-100, ResNet-20, 10 clients, full participation, 100 rounds, 4 local epochs. FedCode uses 64 centers and calibration rates 0.2/0.5. Accuracy (%) and data transmission reduction (DTR, × FedAvg) are higher-is-better; no uncertainty reported.
Dashes reproduce unreported entries in the source table, not zero communication. These results use one workload and the paper’s configured baselines; MUCSC was not reproduced.
| Method | Accuracy (%) | Downstream DTR (×) | Upstream DTR (×) | Joint DTR (×) |
|---|---|---|---|---|
| FedAvg | 61.22 | — | — | — |
| FedAvg + weight clustering | 58.78 | 5.3 | 5.3 | 5.3 |
| FedZip | 57.98 | — | 19.8 | 1.9 |
| FedCode | 58.07 | 24.2 | 10.4 | 14.6 |
Less frequent calibration saves traffic but can sharply reduce accuracy: 264.7× reduction costs 46.88 percentage points in the codebook-only setting.
CIFAR-100, ResNet-20, 64 centers, 10 clients, full participation, 100 rounds, 4 local epochs. F1/F2 are the source table’s calibration-rate labels. Accuracy change is in percentage points versus FedAvg; data volume is reported in MB.
No uncertainty is reported. The preprint inconsistently assigns F1/F2 to upstream and downstream in different sections, so the original labels are retained without relabeling their direction.
| Calibration F1 / F2 | Accuracy Δ (pp) | Joint DTR (×) | Data volume (MB) |
|---|---|---|---|
| 1 / 1 | -0.86 | 5.3 | 394.21 |
| 0.5 / 0.5 | -3.46 | 10.4 | 201.08 |
| 0.2 / 0.5 | -3.15 | 14.6 | 143.92 |
| 0.33 / 0.33 | -5.94 | 15.2 | 138.01 |
| 0.1 / 0.1 | -21.64 | 44.3 | 47.35 |
| 0 / 0 | -46.88 | 264.7 | 7.9 |
With 10% participation, all four selected comparisons show larger accuracy losses under non-IID data. Training budgets and some cluster counts also differ.
10 clients, 10% participation; IID Dir(10) versus non-IID Dir(0.1), with label and quantity skew. Accuracy changes are percentage points versus each FedAvg baseline; joint DTR is a multiplicative reduction. No uncertainty reported.
Appendix B uses longer runs for non-IID data and partial participation; MobileNet also uses more clusters for non-IID data. These rows are not controlled one-variable comparisons.
| Model / dataset | IID accuracy Δ (pp) | Non-IID accuracy Δ (pp) | Joint DTR (×) |
|---|---|---|---|
| ResNet-20 / CIFAR-10 | -1.51 | -1.67 | 15 |
| ResNet-20 / CIFAR-100 | -3.04 | -3.74 | 15.1 |
| MobileNet / CIFAR-10 | -0.33 | -0.81 | 10.9 |
| MobileNet / CIFAR-100 | -1.93 | -2.91 | 10.9 |
@inproceedings{khalilian2024fedcode,
title = {{FedCode}: Communication-Efficient Federated Learning via Transferring Codebooks},
author = {Khalilian, Saeed and Tsouvalas, Vasileios and Ozcelebi, Tanir and Meratnia, Nirvana},
booktitle = {2024 IEEE International Conference on Edge Computing and Communications (EDGE)},
year = {2024},
pages = {99--109},
publisher = {IEEE},
doi = {10.1109/EDGE62653.2024.00022},
url = {https://doi.org/10.1109/EDGE62653.2024.00022}
}