Brave Browser, the privacy-focused search engine, has launched Leo AI, a smart assistant powered by RTX-accelerated local large language models (LLMs) in collaboration with Ollama.
More than just a search tool, Leo AI helps users summarize articles and videos, find insights from documents, and answer questions. With this new feature, Brave is changing how we explore and understand information online!
In a recent blog post, NVIDIA highlighted the groundbreaking technology behind Brave Browser and other AI tools, emphasizing a blend of hardware libraries and user-focused ecosystems.
Powered by NVIDIA GPUs, which are considered pivotal in AI innovations, Leo AI benefits from advanced Tensor Cores designed to accelerate AI applications.
These specialized cores allow Leo AI to perform multiple calculations at the same time. Also enables it to process the large amounts of data needed for AI tasks quickly and efficiently.
The AI inference library is essential for converting requests into instructions for GPUs. Brave and Leo AI use the open-source library llama.cpp, which employs NVIDIA’s CUDA to boost performance on GeForce and NVIDIA RTX GPUs. This enhances the speed of popular models like Gemma and Llama 3.
To simplify integration, applications often utilize a local inference server that handles downloading and configuring AI models. This efficient setup guarantees users a fast and responsive experience with tools like Leo AI, making it easier than ever to access powerful AI capabilities!
Ollama has emerged as an innovative open-source project built on llama.cpp, providing users with access to advanced local AI capabilities. NVIDIA is actively optimizing Ollama for its hardware, ensuring that applications deliver faster and more responsive AI experiences on RTX.
Brave’s Leo AI can operate in both cloud and local environments through Ollama. Running AI locally offers significant benefits, such as enhanced privacy—users can seek financial or medical advice without transmitting data to external servers.
Additionally, local processing eliminates costly cloud fees and gives access to a broader range of open-source models.
Thanks to NVIDIA RTX technology, users can expect impressive performance when running AI locally. For instance, with the Llama 3 8B model, responses can reach up to 149 tokens per second, equating to around 110 words per second.
This capability translates to quicker answers for inquiries and content summaries when using Brave with Leo AI and Ollama.
Setting up Ollama is so easy. Users can download the installer from the official site and run it in the background, allowing them to access various supported models via a command prompt.
Once configured, Leo AI will efficiently utilize the locally hosted model, while users can easily switch between cloud and local options as needed!
This innovative approach not only enhances user experience but also showcases NVIDIA’s commitment to driving AI advancements across platforms.
