the “categorical” way of defining tensor products is essentially “that thing that lets you turn multi-linear maps into linear maps”, and linear maps (of finite dimensional vector spaces) are basically matrices anyways. so i don’t see it as much of a stretch to say tensors are matrices.

(can you tell that i never took a physics class?)