https://www.mo4tech.com/understanding-mixed-precision-training-from-tensor-core-to-cuda-programming.html
Cuda low level.
On Cuda we can writing own neural network.

cuDNN more framework. We get functions and gpu shaders\synchronization.