This academic paper explores rectified activation units (rectifiers) in neural networks, which are crucial for advanced image classification. The authors introduce a Parametric Rectified Linear Unit (PReLU), an enhanced rectifier that dynamically learns its parameters, leading to improved model accuracy with minimal added computational cost or overfitting risk. Furthermore, the paper presents a robust initialization method specifically designed for these rectifiers, enabling the effective training of extremely deep neural networks from the ground up. The research showcases that their PReLU networks (PReLU-nets) surpassed human-level performance on the challenging ImageNet 2012 classification dataset, achieving a 4.94% top-5 error rate, a significant improvement over previous state-of-the-art models. Ultimately, this work contributes to the development of more powerful and trainable deep learning models for visual recognition tasks.

Source: https://arxiv.org/pdf/1502.01852