Many local tasks. One shared task vector.
Clients with several tasks produce updates that can interfere. MaTU sends one unified task vector plus per-task masks and scalars, preserving task differences while enabling server aggregation within tasks and transfer between related tasks.
Compact task representation: elect a shared vector and retain lightweight task-specific modulators.
Selective transfer: combine same-task updates with sign-aligned related tasks.
No retained client weights: the server computes task-level updates without storing client-specific models across rounds.
Federated Learning (FL) traditionally assumes homogeneous client tasks; however, in real-world scenarios, clients often specialize in diverse tasks, introducing task heterogeneity. To address this challenge, Many-Task FL (MaT-FL) has emerged, enabling clients to collaborate effectively despite task diversity. Existing MaT-FL approaches rely on client grouping or personalized layers, requiring the server to manage individual models and failing to account for clients handling multiple tasks. We propose MaTU, a MaT-FL approach that enables joint learning of task vectors across clients, eliminating the need for clustering or client-specific weight storage at the server. MaTU constructs a single “unified” task vector per client and leverages lightweight task-specific modulators — binary masks and scalar rescalers — to disentangle and adapt per-task behavior. This design ensures stateless server aggregation and naturally scales to large task sets with minimal communication overhead. Evaluated across 30 datasets, MaTU achieves superior performance over state-of-the-art MaT-FL approaches, with results comparable to per-task fine-tuning, while delivering significant communication savings.
Read more ↗Each client fine-tunes its assigned tasks separately. A task vector is the change from the common pretrained weights. The experiments use ViT-B/32 with rank-16 LoRA on vision classification tasks.
Section 3.1 · p. 2. θt* denotes fine-tuned task weights and θp the shared pretrained origin.
At each coordinate, take the sign of the sum of local task vectors. Among entries agreeing with that sign, retain the largest magnitude. This election preserves a common direction without averaging away its amplitude.
Equation (2) · p. 3. σ is the elected sign, μ the largest aligned magnitude; ⊙ denotes elementwise multiplication.
Each task keeps a binary mask for entries aligned with its own signs and a scalar matching its original total absolute magnitude. These modulators produce approximate task updates from the unified vector.
Section 3.2, “Forming clients updates” · p. 4. Client and round indices are omitted consistently; mt is a task mask and λt its scalar.
The server uses task agreement masks to combine same-task clients, then transfers sign-aligned updates from up to κ related tasks with similarity above 0.5. It reunifies the updated vectors for each client and returns new masks and scalars for the next local-training round.
Equations (3)–(7) · p. 4. “Stateless” means no client-specific weights retained across rounds; task-level aggregation still occurs. Cross-task transfer is not uniformly beneficial in the ablations.
8 vision tasks · 30 clients · 100 rounds (MaT-FL: 300)
8 vision tasks · 30 clients · 100 rounds (MaT-FL: 300) · selected references
| Method | Accuracy (%) | M bits/task/round |
|---|---|---|
| Individual (centralized) | 90.21 | — |
| FedPer | 82.11 | 5.36 |
| MaT-FL | 76.66 | 6.32 |
| MaTU | 84.32 | 6.32 |
@inproceedings{tsouvalas2025matu,
title = {Many-Task Federated Fine-Tuning via Unified Task Vectors},
author = {Vasileios Tsouvalas and Tanir Ozcelebi and Nirvana Meratnia},
booktitle = {FedGenAI Workshop at IJCAI 2025},
year = {2025},
url = {https://arxiv.org/abs/2502.06376},
eprint = {2502.06376},
archivePrefix = {arXiv}
}