Palo Alto-based AI chip startup SambaNova has introduced a new semiconductor that can be used in running high-performance computing applications, like the company’s LLM platform SambaNova Suite, with faster training of models at a lower total cost.
ANNOUNCEMENT: Meet our newest AI chip, SN40L, which is now powering SambaNova Suite.
The SN40L is an industry-first with its revolutionary design: on the inside it offers both dense and sparse compute, and includes both large and fast memory, making it a truly “intelligent… pic.twitter.com/GvRRnUMlhx
— SambaNova Systems (@SambaNovaAI) September 19, 2023
SambaNova announced the chip, SN40L, can serve a 5 trillion parameter model with 256k+ sequence length possible on a single system node. It is being touted as an alternative to NVIDIA, which is a frontrunner in the chip race and recently unveiled what will possibly be the most powerful chip in the market – GH200 – which can support 480 GB of CPU RAM and 96 GB of GPU RAM.
A 'game changer'
The chip will power SambaNova Suite, an enterprise application that helps private and government organizations build and deploy generative AI applications. The combination of the Suite and the chip will be a ‘game changer,’ said the company.
“We’re now able to offer these two capabilities within one chip – the ability to address more memory, with the smartest compute core – enabling organizations to capitalize on the promise of pervasive AI, with their own LLMs to rival GPT4 and beyond,” said Rodrigo Liang, SambaNova co-founder and CEO.
A shortage of chips
Taiwan Semiconductor Manufacturing Company, another supplier of NVIDIA’s AI chips, will also manufacture SambaNova’s chips. There’s a shortage of semiconductors, and there is a long waiting period for businesses to get their hands on NVIDIA’s most coveted chips, which help run and train massive AI applications and models. To fill this void, companies like SambaNova, AMD, and Intel have stepped up.
“We’ve started to see a trend towards smaller models, but bigger is still better and bigger models will start to become more modular,” said Kunle Olukotun, co-founder of SambaNova Systems.
The company has raised over $1 billion in venture funding. Unlike NVIDIA, SambaNova will not be selling its chips as a separate entity. It will sell the chips as part of its custom-built tech stack, which also has the SambaNova Suite.
That will be even more challenging now that Nvidia is also moving into the full-stack, AI-as-a-service market with its DGX Cloud offering, said Dylan Patel, chief analyst at the consultancy SemiAnalysis. “The chip is a significant step forward,” he says. “I don’t believe the chip will change the landscape.”
“Customers are requesting an LLM with the power of a trillion-parameter model like GPT-4, but they also want the benefits of owning a model fine-tuned on their data. With the new SN40L, our most advanced AI chip to date, integrated into a full stack LLM platform, we’re giving customers the key to running the largest LLMs with higher performance for training and inference, without sacrificing model accuracy," added Olukotun.
Originally published on Interesting Engineering : Original article