Functional modules are a proposed mechanism to take PyTorch's existing NN module API and transform it into a functional form, where all the parameters are explicit argument. Why would you want to do this? What does functorch have to do with it? How come PyTorch's existing APIs don't seem to need this? What are the design problems?

Further reading.

• Proposal in GitHub issues https://github.com/pytorch/pytorch/issues/49171
• Linen design in flax https://flax.readthedocs.io/en/latest/design_notes/linen_design_principles.html