Nvidia Launches New AI Chatbot That Can Run On Your PC

Nvidia has introduced a new approach to running a Large Language Model (LLM) chatbot on your machine with RTX Chat. The announcement comes soon after the American chipmaker announced its latest data center GPU, the RTX 2000 Ada.

Nvidia’s new Chat with RTX app lets you build a custom chatbot tailored to your needs and it can run natively on your machine provided you have an RTX GPU. This custom GPT LLM will be connected to the content on your machine such as documents, notes, videos, and other data. Since it runs locally on a device, it also ensures fast and secure results compared to a cloud-based solution.

Like most other chatbots, users will simply have to enter a query into RTX Chat and it will get contextually relevant answers for you. The best part is that it even lets you choose between different open-source AI models such as Mistral and Meta’s Llama.

The video below shows it in action.

If you search for data within documents, it will not only give you the information you need but also cite reference files in its answers. Similar to Google’s Gemini chatbot, you can also look for information in YouTube videos.

Jensen Huang, founder and CEO of Nvidia said:

Generative AI is the single most significant platform transition in computing history and will transform every industry, including gaming. With over 100 million RTX AI PCs and workstations, NVIDIA is a massive installed base for developers and gamers to enjoy the magic of generative AI.

However, this one also comes with inherent limitations. For instance, it cannot remember previous conversations, so it starts each chat from scratch. There are also a few things that need to be ironed out and it also tends to be a resource hog. However since RTX Chat is still a work in progress, we expect to see improvements soon.

The demo version of this app is available for all Nvidia RTX 30 and 40-series GPUs.