PyTorch Developer Podcast

Why is PyTorch split into so many libraries? What's the point of these splits? What do Windows, mobile and CUDA have to do with the library splits?

Further reading:

• c10 folder architecture description https://github.com/.../wiki/Software-Architecture-for-c10
• Implementation of the TORCH_API visibility macros https://github.com/.../blob/master/c10/macros/Export.h
• An example of virtual call based hook to break library structure https://github.com/pytorch/pytorch/blob/master/c10/core/impl/DeviceGuardImplInterface.h

Why is the dispatcher the way it is today? How did evolve over time, and what constraints got added so that it is the kind of complicated piece it is today?

Further reading:

• How the dispatcher actually works http://blog.ezyang.com/.../lets-talk-about-the-pytorch.../
• Zachary DeVito's original version of ATen, before it got merged back into PyTorch mainline https://github.com/zdevito/ATen
• The multiple dispatch patch: https://github.com/pytorch/pytorch/pull/25653

In this episode, we will discuss how to bind a C++ object in Python? We'll try to answer the following questions: How does pybind11 do it? What's different about how we implement it for Tensor? What are some downsides of the approach?

Note from the future: I recorded and then decided I didn't like my follow up episode about how to preserve PyObjects even when they go dead in Python. Maybe some day!

Further reading:

• Python bindings for Tensor in PyTorch https://github.com/.../csrc/autograd/python_variable.cpp
• pybind11 hash map for maintaining object identity https://github.com/.../inc.../pybind11/detail/internals.h...
• Tensor subclasses don't save their properties https://github.com/pytorch/pytorch/issues/47117
  (but the situation here is more complicated than I imply in the podcast)