The team improved performance by up to 12x on NVIDIA GPUs when utilizing AIT and 4x on AMD GPUs when using PyTorch’s eager mode.

On a range of popular AI models, including convolutional neural networks, transformers, and diffusers, it provides performance almost identical to that of hardware-native Tensor Core (NVIDIA GPU) and Matrix Core (AMD GPU) architectures. Meta AI has created AITemplate (AIT), a unified open-source inference solution with distinct acceleration back ends for AMD and NVIDIA GPU technology, to address these industry difficulties. Further reducing development agility, the proprietary solution may make it more difficult to debug the code swiftly. Although proprietary software toolkits like TensorRT offer customization options, they frequently fail to meet this demand. Additionally, AI production pipelines frequently need rapid development. Because of hardware dependencies in complicated runtime environments, it is challenging to maintain the code that makes up these solutions.

A machine learning system created for one company’s GPU must be entirely reimplemented to run on hardware from a different technology vendor. Currently, AI practitioners now have a minimal choice in the matter of choosing high-performance GPU inference solutions due to their platform-specific nature. GPUs are crucial in delivering the computational power required for deploying AI models for large-scale pretrained models in various machine learning domains like computer vision, natural language processing, and multimodal learning.

Meta system m100rc