New NVIDIA AI Toolkit Seeks to Ease AI Adoption

The TAO starter pack, which stands for Train, Adapt, and Optimize, streamlines the creation of AI models for vision and speech recognition.

“Actually, no AI proficiency is required. You don’t even need to be familiar with every framework. And you can use pre-trained models that are very accurate and perform well,” said Chintan Shah, chief product officer at NVIDIA, at a recent press conference.

Shah explained that the CAT system is primarily intended for inference, with some applications including language translation and object detection in robots and cars. “The presumption is that the user has annotated data. He trains it on his own data, and TAO abstracts away the complexities of TensorFlow or Pytorch. You don’t even need to be familiar with any of the deep learning frameworks,” Shah explained.

The framework includes task-specific models for machine vision, such as person detection, vehicle detection, and posture estimation. It also supports general-purpose applications, such as the YOLO v4 and EfficientNet object identification model implementations.

The CAT framework includes 20-25 speech models for applications such as speech recognition, text-to-speech, and natural language processing. CAT-based models can be used in conjunction with NVIDIA framework SDKs such as DeepStream for video analytics and Riva for conversational AI.

There is technically no limit on model size as it supports small models like MobileNet, but can scale to huge models with billions of parameters. “What happens is that the limit of the model is basically what it admits. Attempting to run a large language model on a small GPU will quickly exhaust memory and compute resources. However, we have the potential to grow it to multi-GPU and multi-node,” Shah said.

TAO, meanwhile, focuses on specific tasks, specifically speech recognition and machine vision. Larger and more complicated AI models, spread across a wide network of GPUs, require better insight into data as it is fed into the system. Depending on the results, these learning models will end up requiring manual intervention. “It’s not something that TAO addresses, but we recognize that it’s a larger concern and we’re actively working with our ecosystem partners who provide similar services,” Shah added.

TAO builds AI models and trains the data using low-code methodologies. To get started, developers can use a Jupyter notebook with a command line interface to import real and synthetic data sets for training.

The user can use augmentation techniques to enhance or generalize the data, such as transforming photos from RGB to grayscale using spatial augmentation, rotation, or color transformation. This process also includes data discovery, segmentation, and classification. “This can significantly increase the variety of your data set.” This will allow you to generalize your model much more effectively,” explained Shah.

TAO’s starter kit addresses one of the most difficult aspects of making AI accessible for effective use, primarily because not everyone can afford to hire data scientists.

The tool kit can be download for free from the NVIDIA website. Chintan Shah concluded by noting that his company is exploring the possibility of making NVIDIA GPU resources available to users through its cloud-based service, Launchpad, where customers can experience various products, including CAT.