This in my Demo of Chen et al. "GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks" ICML 2018
Input: Two synthetic regression tasks according to Ma et al. (KDD 2018) "Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts"
Network: One shared layer and two task specific towers
Framework: Pytorch
Update (Nov 2021): This repository is not maintained anymore. The unresolved issues are left open in case someone has a revised implementation.