AI Toolkit now supports NVIDIA NIM™ microservice-based foundation models for inference testing in the model playground and advanced features like bulk run, evaluation and building prompts. 

This collaboration helps AI Engineers streamline development processes with foundational AI models. 

About AI Toolkit

AI Toolkit is a VS Code extension for AI engineers to build, deploy, and manage AI solutions. It includes model and prompt-centric features that allow users to explore and test different AI models, create and evaluate prompts, and perform model finetuning, all from within VS Code. Since its preview launch in 2024, AI Toolkit has helped developers worldwide learn about generative AI models and start building AI solutions. 

NVIDIA NIM Microservices

This January, NVIDIA announced that state-of-the-art AI models spanning language, speech, animation and vision capabilities - offered as NVIDIA NIM microservices - can now run locally on NVIDIA RTX AI PCs. These microservices prepackage optimized AI models with all the necessary runtime components for deployment across NVIDIA GPUs. Developers can now develop and deploy anywhere with the same unified experience and software stack across RTX AI PCs and workstations to the cloud. Developers can jumpstart their AI development journey by downloading and running NIM containers quickly on Windows 11 PCs with GeForce RTX GPUs using Windows Subsystem for Linux (WSL2).

The Power of Collaboration

Integrating AI Toolkit with NIM provides AI engineers with a more cohesive and efficient workflow:

• Seamlessly integrate AI Toolkit with NIM to create a unified development environment without the need to switch context. Users can access any NIM supported models from AI Toolkit.
• Leverage the combined capabilities of both tools to streamline workflows and accelerate AI solution development process around foundation AI models, from within VS Code.

How to get started

Follow these steps to begin leveraging the power of NIM on AI Toolkit:

• Download and install the latest version of AI Toolkit for VS Code.
• Install NIM pre-requisites on RTX PCs using the instructions here.
• Select a NIM model from the model catalog on AI Toolkit and load it in Playground.
• Optionally, you can also add your NIM model hosted in the cloud to AI Toolkit by URL
• Explore NIM models from the playground
• Start developing prompts with new NIM models in AI Toolkit!

Screenshot from AI Toolkit for VS Code

Looking Forward

We invite you to explore the possibilities of this integration and take your development projects to new heights! Try AI Toolkit today – and please continue sharing your feedback.

Stay tuned for more updates and detailed tutorials on how to maximize the benefits of this exciting new collaboration. Together, we are shaping the future of AI development!