This is the implementation of the paper Federated Multimodal Learning with Dual Adapters and Selective Pruning for Communication and Computational Efficiency (arxiv).

Linux x86_64

With the installed conda, we can run this platform in a conda virtual environment called dlp.

conda env create -f env.yaml # for Linux or Windows with CUDA >= 12.1
conda activate dlp

We provide 4 datasets: Oxford-IIIT Pet (Pets), Describable Textures Dataset (DTD), Oxford 102 Flower (Flowers), and FGVC-Aircraft (Aircraft). They can be easy split into IID and non-IID version. For non-IID, we have practical setting (with hyperpameter for Dirichlet distribution $\beta$) and pathological setting (few-shot scenario).

• Total 10 clients, iid and balance scenario

  cd ./dataset
  python generate_pets.py 10 iid balance - - - - pfl
  

• Total 10 clients, practical noniid scenario with $\beta = 0.1$

  cd ./dataset
  python generate_pets.py 10 noniid - dir 0.1 - - pfl
  

• Total 10 clients, practical noniid scenario with $\beta = 0.01$

  cd ./dataset
  python generate_pets.py 10 noniid - dir 0.01 - - pfl
  
  
  
  

After generating and partitioning dataset for clients, we can run the training and evaluation. All codes corresponding to FedDLP and other baselines: Local, FedCLIP, FLoRA, and FedDAT are stored in ./scripts. Different folder corresponds with that specific dataset. From the FedDLP folder, run the following, the results will be saved in the logs folder.

bash ./scripts/dtd/localora_text.sh         # for text encoder
bash ./scripts/dtd/localora_image.sh        # for image encoder

bash ./scripts/pets/fedclip_text.sh         # for text encoder
bash ./scripts/pets/fedclip_image.sh        # for image encoder

bash ./scripts/dtd/flora_text.sh            # for text encoder
bash ./scripts/dtd/flora_image.sh           # for image encoder

bash ./scripts/flowers/feddat_text.sh       # for text encoder
bash ./scripts/flowers/feddat_image.sh      # for image encoder

bash ./scripts/aircraft/feddlp_text.sh      # for text encoder
bash ./scripts/aircraft/feddlp_image.sh     # for image encoder

Feel free to change parameters to your desired experiments. If you use the same setting as our papers, you can simply adopt the hyperparameters reported in our paper.

This code is heavily inspired from the popoular federated learning project PFLlib. Big shout out for their wonderful work!

• FedDLP

  @inproceedings{nguyen2025feddlp,
      title={Adaptive configuration for heterogeneous participants in decentralized federated learning},
      author={Nguyen, Duy Phuong and Muñoz, J. Pablo and Roosta, Tanya and Jannesari, Ali},
      booktitle={IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid)},
      year={2025}
  }