Browsing: Deep Learning

In 2022, a small team of researchers ran an experiment that should have been unremarkable. They trained a tiny neural network, far smaller than anything that gets called “AI” today, on one of the simplest tasks imaginable: modular addition, basically clock math. What is 8 plus 7 if the clock only goes up to 12?…

In a previous article, we saw why training with MSE forces a model to report only the conditional mean, and howGaussian NLL lets it learn an honest per-step uncertainty σ sigma as well. That was the one-step story. This post picks up where that left off.

In this article, I am going to review and implement the deep learning paper titled “CBAM: Convolutional Block Attention Module” by Woo et al. [1]. As the name suggests, this is essentially a block we can attach to a CNN-based model to enhance feature quality by performing an attention mechanism. Despite the name attention, it…