DDP Hangs with uneven batches #80
Description
If you run training with an uneven number of batches on different GPUs, a hang will occur. Since PyTorch 1.8.1, the join context manager on DDP exists to help with this situation. I don't believe PyMarlin handles this right now