Two-Way Knowledge Distillation for Federated Learning

The workflow of the Fed2KD framework. Red arrows indicate the knowledge distillation direction.


Invention Summary:

Current contact tracing reporting platforms for infectious diseases are limited by their reactive nature, centralized approach, use of GPS, and privacy risk due to exchange of user data. Existing federated learning (FL) frameworks that can be used for alternative disease-tracking are limited to homogenous models and ignore user preferences and varying device properties.

Rutgers researchers have developed a novel, 2-way knowledge distillation-based FL framework (“Fed2KD”) which can provide a real-time, scalable, privacy-preserving indoor risk assessment tool. Fed2KD learns from global and local models with a conditional variational autoencoder (CVAE) to generate a proxy, common dataset – thus sparing the use of any local, private data. Fed2KD has been shown to outperform the existing FedAvg model by 18-30% using heterogenous, non-independent, identically distributed data. Fed2KD has also been evaluated on a pandemic risk assessment task mobile app to accurately determine indoor risk protection.

Marketing Applications:

•   Financial/economic assessments
•   Real-time monitoring applications
•   Pandemic/disease risk assessments

Advantages:

•    Privacy-preserving
•    Scalable
•    Proactive
•    Can provide indoor infectious disease risk assessments
•    Can incorporate heterogeneity in Federated Learning

Intellectual Property & Development Status: Provisional patent application filed, patent pending. Available for licensing and/or research collaboration. For any business development and other collaborative partnerships contact marketingbd@research.rutgers.edu.

Publications: C. Sun, T. Jiang, S. Zonouz and D. Pompili, "Fed2KD: Heterogeneous Federated Learning for Pandemic Risk Assessment via Two-Way Knowledge Distillation," 2022 17th Wireless On-Demand Network Systems and Services Conference (WONS), Oppdal, Norway, 2022, pp. 1-8, doi: 10.23919/WONS54113.2022.9764443 

Patent Information: