NVIDIA's Chat with RTX Brings AI Chatbot Directly to PCs

NVIDIA released  "Chat with RTX," an AI chatbot that operates directly on personal computers, marking a significant advancement in localised AI capabilities. 

This early release demo app allows users to leverage their NVIDIA RTX 30- or 40-series GPUs to run a personal AI chatbot capable of summarising content from YouTube videos and personal documents, thus offering a new level of data analysis and information retrieval.

The Chat with RTX’s demo can be downloaded here. 

Chat with RTX operates by installing a web server and Python instance on the user's PC, utilising the powerful Tensor cores of NVIDIA's RTX GPUs to expedite queries. 

This local processing ensures quick responses and also keeps the user's data private, without the need for internet connectivity or cloud-based processing. Vin Vashishta  founder & AI advisor at V Squared said, “Running the LLM locally means fewer data privacy concerns and faster response times. RAG keeps the responses constrained and relevant to the user.”

The application also supports a variety of file formats, including .txt, .pdf, .doc/.docx, and .xml, and can analyse YouTube video transcripts for specific mentions or summaries, enhancing research and data analysis tasks significantly.

Despite its potential, Chat with RTX is still in its early stages, with users reporting some accuracy issues and limitations, such as the inability to maintain context between questions and the creation of JSON files within indexed folders. 

Chat with RTX has brought advanced AI capabilities directly to users' desktops, offering privacy, speed, and convenience. “It's a promising look at how AI could be set to overhaul Windows,” posted Tom Warren, the senior editor at Verge on X. As NVIDIA continues to refine and enhance this technology, it could impact how people interact with AI on their devices.