Asynchronous Multi-Task Learning Based on One Stage YOLOR Algorithm

The You Only Learn One Representation (YOLOR) approach is an object detector that can encode implicit knowledge and explicit knowledge of multiple tasks simultaneously. However, the requirement of jointly feeding data is not a friendly setting for an edge device due to the high computational cost. A better strategy is to learn the concept of the new task on the device individually, one by one, without access to the old data. In other words, the model has to deal with multiple tasks asynchronously. In this work, we extend the multi-purpose network YOLOR to asynchronous multi-task learning to learn domain invariant features, which focus on capturing the relatedness between the weight of the previous task and data of the subsequent task. Further, as the number of tasks gradually increases, we accumulate significant weights by introducing task-specific masks and expert modules; the former can automatically identify important filters to prevent modification caused by new tasks, and the latter address the kernel space misalignment problem to perform multi-task feature selection. We experimentally demonstrate that the proposed training strategy significantly outperforms the traditional solution in learning multiple tasks at different times on a public dataset, which supports that the proposed approach is more competitive for resource-limited edge devices.