These two papers (years 2017, 2022) introduce and then apply the Gumbel-Softmax distribution as a differentiable gradient estimator for categorical and discrete latent variables in neural networks. The first paper, "Categorical Reparameterization with Gumbel-Softmax," proposes this distribution to address the challenge of backpropagating through non-differentiable sampling operations, demonstrating its effectiveness in tasks like structured output prediction and generative modeling, where it outperforms existing gradient estimators and allows for significant speedups in semi-supervised classification. The second paper, "Gumbel-Softmax Selective Networks," leverages this same reparameterization trick to train selective neural networks with an integrated, binary option to abstain from predicting when uncertain, thereby establishing an end-to-end differentiable framework for selective regression and classification tasks. Collectively, the sources present the Gumbel-Softmax technique as a general, principled method for enabling gradient flow through discrete or binary choices in neural network training.Sources:https://arxiv.org/pdf/1611.01144https://arxiv.org/pdf/2211.10564