On September 10, Arcee AI unveiled SuperNova, a 70 billion-parameter language model composed for enterprise efficiency. With advanced features and customization options, SuperNova offers a powerful alternative to API-based services from OpenAI and Anthropic, focusing on data privacy, model stability, and tailored solutions.
Model Design and Post-Training Techniques
Arcee’s SuperNova is built using innovative in-house post-training techniques. It enhances the Llama-3.1-405B-Instruct model, refining it into the more efficient Llama-3.1-70B-Instruct. Thanks to Archee’s DistilKit, SuperNova maintains high performance while reducing model size by leveraging logits and attention masks from the 405B model.
Technical innovations
According the Lucas Atkins, lead engineer on the project, the development of the SuperNova is a multi-facted approach to post-training.
“We trained three models at once. One was distilled from Llama 405B. Another was trained with a dataset we generated with our EvolKit repository. And the third was doing a pretty exhaustive DPO on top of the current Llama 3 instruct. At the end, we use a new kind of merging technique to combine all three, preserving the strengths of each one.”
Arcee’s proprietary process has refined SuperNova’s instruction-following from a 405B parameter model into a more deployable version.
EvolKit, an open-source tool, generates complex data for fine-tuning, allowing enterprises to adapt SuperNova to their needs while maintaining high performance on modest hardware.
Enterprise Deployment and Customization
SuperNova is designed for use within an organization’s cloud environment, such as AWS VPC, with plans for Google and Azure marketplaces. It addresses data privacy and allows fine-tuning for specific needs.
Moreover, SuperNova can be fine-tuned and retrained in-house, adapting to specific domain needs or company requirements.
Open-Source Components and Transparency
While the full 70B model is not open-source, Arcee is introducing several components, including a free API for testing, SuperNova-Lite (an 8B parameter open-source version), and EvolKit.
With these open-source components, Arcee is making its field in broader AI.
It also offers customizing tools to its potential customers. Arcee SuperNova is also available in AWS Marketplace.
Performance Claims and Benchmarks
Atkins:
“SuperNova is noted for strong mathematical reasoning. This one is pretty outstanding on math benchmarks.”
This openness to independent verification enables comparisons with top AI models from companies like OpenAI and Anthropic.
Implications for Enterprise AI Strategy
SuperNova ensures data privacy, model stability, and deep customization, potentially offering long-term cost savings compared to API services. It provides a competitive edge in AI-driven industries.
AI Sovereignty Dilemma
SuperNova addresses the tension between cloud-based AI services and on-premise models, offering full control and customization but requiring in-house expertise.
Future Implications and Challenges
The success of SuperNova will depend on how it measures up against cloud models in performance, ease of deployment, customization, and cost.
SuperNova challenges the belief that cutting-edge AI must come from cloud APIs, offering a more controllable and tailored solution. Its impact could shift the balance between cloud-based and on-premise AI, potentially redefining enterprise AI strategies.
Arcee AI’s SuperNova promises to transform enterprise AI with its advanced, customizable features. Its openness to independent verification and focus on data privacy offer a strong alternative to existing API-based solutions. With these innovations, SuperNova is on the verge of redefining enterprise AI, delivering enhanced control and efficiency.
Source:
https://blog.arcee.ai/meet-arcee-supernova-our-flagship-70b-model-alternative-to-openai
