Tue Feb 18 18:22:52 2020
<9e126bf3> <https://openai.com/blog/deep-double-descent/>
— Deep Double Descent
— We show that the double descent phenomenon occurs in CNNs, ResNets, and transformers: performance first improves, then gets worse, and then improves again with increasing model size, data size, or training time. This effect is often avoided through careful regularization. While this behavior appears to be fairly universal, we don’t
