AutoEncoder-Compressed Parallel Split Learning for Pre-Trained Model Fine-Tuning

Bas Meuwissen; Vasileios Tsouvalas; Nirvana Meratnia

AE-PSLWAFL @ ECML-PKDD · 2026

AutoEncoder-Compressed Parallel Split Learning for Pre-Trained Model Fine-Tuning

Align first. Compress every training step.

Bas Meuwissen Vasileios Tsouvalas Nirvana Meratnia

Paper Code Poster

At a glance

Split learning still sends activations and gradients at every training step. AE-PSL aligns an autoencoder before fine-tuning, then freezes it to compress patch features in both directions while preserving compatibility with pretrained representations.

Inside AE-PSL

Align with pretrained featuresGeneral Alignment trains the autoencoder to reconstruct public split-layer features. The pretrained client partition stays frozen; the evaluated preparation uses ImageNet100 for 30 epochs.

Adapt locally, average decodersClient-Specific Alignment adapts each autoencoder for one local epoch using reconstruction loss. Clients retain their own encoders and send only decoder weights for averaging into one shared decoder.

Compress patches, pass CLS directlyFine-tuning begins with both autoencoder halves frozen. Patch activations cross the network as low-dimensional latents, while the classification token (CLS) bypasses compression.

Return gradients through the frozen AEPredictions return for local loss computation; the server aggregates losses. Backpropagation sends latent and CLS gradients to clients. Gradients pass through the frozen encoder to update client-side LoRA parameters.

Separate the two inference settingsGlobal tests average client model weights and bypass the autoencoder. Local FEMNIST tests use each client’s own model with the autoencoder active. These settings answer different questions about compressed training and deployment.

What this adds

01
Two-stage alignment: learn public feature reconstruction, then adapt the compressor to each client.
02
Shared decoding: keep local encoders and average only decoders into one server module.
03
Bidirectional compression: exchange compact patch activations and gradients while leaving CLS uncompressed.

Abstract

Distributed Fine-Tuning (DFT) of large-scale Foundation Models (FMs) on resource-constrained edge devices is limited by local compute constraints and communication overhead. Parallel Split Learning (PSL) reduces client-side computation by keeping few model layers on each client and offloading the remaining computation to the server; however, clients must exchange intermediate activations and gradients with the server at every training step. Existing SL communication-compression methods mainly rely on task-agnostic heuristics, such as sparsification and quantization. While learnable SL compressors can better adapt to intermediate representations, they require co-training with the target model. Therefore, directly inserting them into off-the-shelf FMs introduces feature-distribution misalignment and degrades DFT performance. To address this, we propose AE-PSL, a communication-efficient PSL framework that compresses intermediate activations and gradients using a lightweight AutoEncoder (AE) placed at the split layer. To ensure compatibility of AE compression with pre-trained FMs, AE-PSL introduces a novel two-stage alignment mechanism, which adapts the AE to the pre-trained model’s feature manifold and client-specific feature distributions before DFT. We evaluate AE-PSL across four vision datasets and various communication reduction settings, showing that it preserves downstream accuracy at 10.2× reduction; at the same communication budget, the strongest evaluated heuristic baseline trails AE-PSL by 5.2 percent point. This demonstrates that AE-PSL offers a stronger accuracy–communication trade-off than heuristic SL compression.

AE-PSL

General Alignment

Train the autoencoder on public features with the pretrained client partition frozen. The reported default uses ImageNet100 for 30 epochs. The reconstruction objective is reused during local alignment.

ℒ_{rec} = \frac{1}{| 𝒟 |} \sum_{x \in 𝒟} {‖ H (x) - D_{ψ} (E_{ϕ} (H (x))) ‖}_{2}^{2}

Equation (1), p. 5, compactly written as a loss. H is the frozen split-layer feature; E/D are encoder/decoder; 𝒟 is public or local alignment data.

Client-Specific Alignment

Each client adapts both autoencoder halves for one local epoch while keeping the feature extractor frozen. It retains its encoder and sends only decoder weights for a simple mean at the server.

ψ^{global} = \frac{1}{N} \sum_{n \in 𝒩} ψ_{n}

Section 4.1.2 · p. 5. N is the number of clients; ψn denotes client n’s decoder weights.

Fine-tune through frozen compressors

With the AE weights fixed, clients send patch-token latents plus the original CLS activation. The server reconstructs patches, predicts, aggregates local losses and returns latent and CLS gradients. Backpropagation through the frozen encoder still updates the client model.

Section 4.2 · p. 6. Default: ViT-B/32, split layer 5, LoRA rank 16; four supervised vision datasets.

Match the inference setting to the claim

Global evaluation aggregates client weights and bypasses the AE; local FEMNIST evaluation keeps each client model and the AE active. Alignment is particularly important when the compressor is removed after training.

Section 5.2, Table 2 and Appendix A.3 · pp. 7, 11 and 21. An unaligned AE does not show the same collapse in local tests with the AE retained.

Method details in the paper ↗

Original paper figures & sources

AE-PSL pipeline: general autoencoder alignment on public data, local client-specific alignment with decoder averaging, and distributed fine-tuning using compressed activations and gradients. — Two alignment stages prepare the compressor before fine-tuning. Each client keeps a local encoder, while the server shares one decoder; both stay frozen during the final training stage. Figure 1 in the public preprint

Evaluation

Results

Results from arXiv:2607.17913v1 ↗

Global tests · AE bypassed

At this reduction and client count, AE-PSL stays within 0.5 percentage points of the uncompressed reference on all three global benchmarks.

Global accuracy (%)

Higher is better

CIFAR-100 (%)Food101 (%)SUN397 (%)

0100

No compression · R=1

82.7 ± 0.2

69.1 ± 0.9

67.0 ± 0.6

C3-SL

50.9 ± 1.2

40.3 ± 1.2

41.3 ± 2.1

Rand-Top-K

76.2 ± 0.4

60.2 ± 0.2

62.9 ± 1.0

ADC

68.4 ± 0.4

54.6 ± 1.6

52.9 ± 1.1

AE-PSL

82.7 ± 0.2

69.6 ± 1.2

66.7 ± 0.3

Experiment details

ViT-B/32, split layer 5, five clients, Dirichlet α=0.5; top-1 accuracy (%), higher is better. Mean ± standard deviation over three seeds. Compressed methods use R≈10; the reference uses R=1.

These are global tests after client-weight aggregation with the autoencoder bypassed. They do not measure compressed inference accuracy or full preparation cost.

Global accuracy at about 10× reduction. ViT-B/32, split layer 5, five clients, Dirichlet α=0.5; top-1 accuracy (%), higher is better. Mean ± standard deviation over three seeds. Compressed methods use R≈10; the reference uses R=1.
Method	CIFAR-100 (%)	Food101 (%)	SUN397 (%)
No compression · R=1	82.7 ± 0.2	69.1 ± 0.9	67.0 ± 0.6
C3-SL	50.9 ± 1.2	40.3 ± 1.2	41.3 ± 2.1
Rand-Top-K	76.2 ± 0.4	60.2 ± 0.2	62.9 ± 1.0
ADC	68.4 ± 0.4	54.6 ± 1.6	52.9 ± 1.1
AE-PSL	82.7 ± 0.2	69.6 ± 1.2	66.7 ± 0.3

Table 4 · PDF page 17 ↗

Local tests · AE active

AE-PSL has the highest reported accuracy among the compressed methods, 1.2 percentage points below the uncompressed reference.

Local accuracy (%)

Higher is better

0100

No compression · R=1

85.8 ± 0.6

C3-SL

6.9 ± 1.8

Rand-Top-K

82.2 ± 0.3

ADC

73.8 ± 5.4

AE-PSL

84.6 ± 1.0

Experiment details

FEMNIST, ViT-B/32, 25 clients, writer-based partitions; local accuracy (%) averaged across clients, higher is better. Mean ± standard deviation over three seeds. Compressed methods use R≈20; reference R=1.

FEMNIST uses 10% subsamples per writer and local tests. Its scores are not interchangeable with the centralized global tests above.

Local inference under stronger compression. FEMNIST, ViT-B/32, 25 clients, writer-based partitions; local accuracy (%) averaged across clients, higher is better. Mean ± standard deviation over three seeds. Compressed methods use R≈20; reference R=1.
Method	Local accuracy (%)
No compression · R=1	85.8 ± 0.6
C3-SL	6.9 ± 1.8
Rand-Top-K	82.2 ± 0.3
ADC	73.8 ± 5.4
AE-PSL	84.6 ± 1.0

Table 4 · PDF page 17 ↗

Alignment and freezing ablation

Alignment and freezing restore compatibility with uncompressed global inference. An unaligned AE does not cause the same collapse when it remains active locally.

Accuracy (%)

Higher is better

CIFAR-100 · global (%)FEMNIST · local (%)

0100

No compression

82.7 ± 0.4

87.6 ± 0.7

AE only

1.4 ± 0.3

86.9 ± 2.0

AE + GA

75.2 ± 0.4

87.1 ± 1.1

AE + GA + FZ

79.6 ± 0.9

88.0 ± 0.6

AE + GA + CSA + FZ

82.2 ± 0.7

88.0 ± 0.7

Experiment details

ViT-B/32, five clients; R≈15 for all AE variants, R=1 for the reference. Accuracy (%) with reported ± terms over three seeds. GA: general alignment; CSA: client alignment; FZ: frozen autoencoder.

This is a separate ablation at R≈15. Its baseline values and reported spreads differ from Table 4, so the two tables should not be combined into one estimate.

Why alignment and freezing matter. ViT-B/32, five clients; R≈15 for all AE variants, R=1 for the reference. Accuracy (%) with reported ± terms over three seeds. GA: general alignment; CSA: client alignment; FZ: frozen autoencoder.
Configuration	CIFAR-100 · global (%)	FEMNIST · local (%)
No compression	82.7 ± 0.4	87.6 ± 0.7
AE only	1.4 ± 0.3	86.9 ± 2.0
AE + GA	75.2 ± 0.4	87.1 ± 1.1
AE + GA + FZ	79.6 ± 0.9	88.0 ± 0.6
AE + GA + CSA + FZ	82.2 ± 0.7	88.0 ± 0.7

Table 2 · PDF page 11 ↗

Study scope & sources

Supervised vision fine-tuning with ViT-B/32; language and multimodal adaptation are not evaluated.
Public-data preparation and client warm-up are required; uncompressed CLS imposes a communication floor.
Global and local tests use different inference paths and must not be pooled.

Data: CIFAR-100, Food101, SUN397 and FEMNIST; public ImageNet100 for general alignment.
Model: ImageNet-pretrained ViT-B/32, default split layer 5, two-layer MLP autoencoder; LoRA rank 16.
Training: 5 or 25 clients; 10 fine-tuning epochs, global batch 125; 30 general-alignment epochs and one local alignment epoch.
Compared with: Uncompressed training, Rand-Top-K, C3-SL and ADC, adapted to the same aggregated parallel split framework.

Scope & limitations

The evaluated downstream tasks are vision classification; non-vision, language-model and multimodal adaptation are not tested.
The design relies on an uncompressed classification token. Compressing it damages accuracy in the appendix ablation.
The principal global tests bypass the AE at inference; local FEMNIST uses a different evaluation protocol with it active.
Client GFLOP comparisons include local warm-up, but do not establish total system latency, energy savings or the amortized cost of 30-epoch public alignment.

arXiv:2607.17913v1, 20 July 2026; 28 pages including Appendix A.1–A.8 ↗

Reference

Cite this work

BibTeX

Download .bib

@inproceedings{meuwissen2026aepsl,
  title = {AutoEncoder-Compressed Parallel Split Learning for Pre-Trained Model Fine-Tuning},
  author = {Bas Meuwissen and Vasileios Tsouvalas and Nirvana Meratnia},
  booktitle = {4th Workshop on Advancements in Federated Learning (WAFL), ECML-PKDD 2026},
  year = {2026},
  url = {https://arxiv.org/abs/2607.17913},
  eprint = {2607.17913},
  archivePrefix = {arXiv}
}

Keep exploring

MPSL

Multimodal fine-tuning with lightweight edge clients.

DeltaMask

Adapt foundation models by communicating sparse mask changes.