TorchDynamo Deep Dive

Introduction to Dynamo

1. Dynamo is a tracer
   • tracer: Records linear(=no branching) sequence of torch ops(Aten)
2. Given some inputs, it returns an FX graph with the PyTorch functions that were executed
   • torch.fx.graph is a container that stores a list of func calls
3. It can also trace integers if it detects that they changed between calls (SymInt)
4. It specializes any other value that is not a tensor or a scalar