Green AI

Comparison of ML frameworks in terms of energy efficiency, carbon emissions, and computational cost.

Three frameworks are compared: TensorFlow, PyTorch, and MXNet.

Environment chosen is using Nvidia RTX 3070 with 8 Gb of memory, energy data is collected sampling from nvidia smi.

• How do they differ in terms of energy efficiency?

Recording of GPU power / utilization / energy + partitioning for each HW component.

Torch is better with larger batch size and with larger models (prob. due to better parallelization) (ex. BERT).

MxNet is better with batch size of 1.

• How do execution providers differ in terms of energy efficiency or performance?

Comparison between CUDA / Tensor RT

Tensor RT always outperforms CUDA, why? It utilizes GPU more effectively (> GPU utilization).