DropPathway

DropPathway randomly drops an audio pathway during training as a regularization technique for audiovisual recognition models. Specifically, at each training iteration, we drop the Audio pathway altogether with probability $P_{d}$. This way, we slow down the learning of the Audio pathway and make its learning dynamics more compatible with its visual counterpart. When dropping the audio pathway, we sum zero tensors with the visual pathways.

Note that DropPathway is different from simply setting different learning rates for the audio/visual pathways in that it 1) ensures the audio pathway has fewer parameter updates, 2) hinders the visual pathway to 'shortcut' training by memorizing audio information, and 3) provides extra regularization as different audio clips are dropped in each epoch.