GPUs were built to render game pixels in parallel — the same maths as multiplying huge matrices, which is exactly what training a neural network needs.
