Local and offline edge AI gets a boost as SiMa.ai launches Palette 3.0 with 3.5X faster text generation and a new model browser


Credit: VentureBeat made with OpenAI ChatGPT-Images-2.6
For companies that put AI into robots, drones or industrial cameras, making an AI model smart is only part of the challenge. The device must also handle information from its sensors, produce useful responses and keep other programs running without using up all its memory or power.
SiMa.ai is working on this problem with Palette 3.0, its software operating system for its Modalix edge AI processors. Palette 3.0 expands the kinds of AI tasks these chips can run and improves how different applications share the chip. The company says its new methods can make text generation up to 3.5 times faster.
The platform is for developers who build AI into physical products, including robots, drones and industrial systems. The Palette Neat core library, which is part of the tools for building applications, uses an Apache 2.0 license. The company has not confirmed the license terms for its compiler and firmware.
Developers can get started through SiMa.ai's Developer Center and GitHub organization. The company's DevKit 3.0 store listing gives a price of $1,499 for a development system in the U.S. and Europe. It includes a Modalix module, a carrier board, an enclosure, a power supply and 500GB of storage. The announcement does not give a separate price for Palette 3.0 software or prices for hardware bought in large production quantities.
According to a SiMa.ai community notice, the Palette SDK has been available since October 31. The company says its updated Model Browser goes live on November 17 as it explains the update's features and performance.
In an interview with VentureBeat, SiMa.ai GenAI product manager Manuel López Roldán confirmed that the changes work on existing chips.
“It’s all software,” he said. Ashok Sudarsanam, the company’s associate vice president of machine learning accelerator (MLA) software, said his team builds the compiler that turns models into instructions for the accelerator. His team also builds the software that decides when those instructions run.
“From day one, we built our hardware and software together,” Sudarsanam told VentureBeat. He said SiMa.ai updates its software to support newly released models and make applications run faster on chips customers already own.
Running several AI workloads on the same chip
The main change in Palette 3.0 is how a device handles more than one AI task at a time.
In a company presentation, SiMa.ai showed one Modalix chip running a language model alongside three vision-language models. In another demonstration, six Docker containers shared the chip's resources while doing different jobs, including finding objects and estimating 3D poses.
The company changed how applications send work to the chip. Programs now send requests directly to drivers inside the operating system kernel. This removes an intermediate service. Each hardware engine handles one request at a time, but different engines can work on separate tasks at the same time. Developers can also give individual requests different priority levels.
López Roldán gave an example involving a camera. The camera may need to keep detecting objects in its video while a language model sometimes adds information about the scene. He said developers can give object detection higher priority and let the language model use the accelerator when it is available. The demonstrations show how the chip shares its resources. However, the presentation does not say how long high-priority requests might have to wait if the chip stays busy for a long time.
SiMa.ai also demonstrated YOLOv26 object detection on 60 video streams at the same time. Each stream had a resolution of 720p. One Modalix chip handled all 60 streams at a combined speed of 600 frames per second, or an average of 10 frames per second for each stream. In a separate camera-tuning demonstration, López Roldán said developers can point a new camera at a color chart and use an AI agent to adjust its image-processing settings. SiMa.ai says developers can make these changes without recompiling application code. This lets teams adjust settings for a new camera themselves.
What the 3.5x performance claim measures
SiMa.ai says its biggest improvements in text generation speed come from a method called speculative decoding. This method drafts several possible next tokens in advance. The main model then checks those guesses together. This reduces the number of steps that must happen one after another before an answer is ready.
The company's presentation reports that this method increased the number of tokens generated per second by between 2.21x and 3.49x compared with regular generation using the same target models. The methods tested include EAGLE3, DFlash and multi-token prediction for Gemma 4. One highlighted test used DFlash N16 with Qwen3.5 9B and reached the top end of that range.
SiMa.ai measured these gains on Modalix and compared them with regular decoding on the same models. In a separate test with 900 requests to an automotive assistant, the company reported a smaller improvement in response time, ranging from 1.35x to 2.06x. SiMa.ai says all 900 requests produced the expected structured function calls.
SiMa.ai also estimates that the chip used 25.9% to 51.4% less energy for each computing request while using roughly the same average power. The chip saved energy because it finished each task sooner. This estimate covers only the chip's computing work. A robot's total energy use also depends on other parts, such as its sensors and motors.
Another important feature is support for OpenAI's GPT-OSS-20B on a 16GB Modalix module. Getting a model to fit on the device is a useful step, but customers still need to measure its response time, context limits and accuracy using the settings they plan to use.
A model browser that shows how SiMa.ai got its numbers
López Roldán showed VentureBeat how the updated Model Browser explains the trade-offs behind a performance test. Along with throughput and latency, it shows measurements such as chip power use, CPU and memory use, and how each model layer runs. For models with several configurations, developers can see how using less precise numbers affects speed and accuracy. This can help a team choose which configuration to try after fine-tuning a model with its own data.
López Roldán said the browser's download button provides a file prepared to run locally on Modalix. Developers who want to build it themselves can use scripts in a public repository. These scripts start with the original model from sources such as GitHub, Hugging Face or Roboflow. When developers are ready to prepare a fine-tuned model for use on a device, he said, “you can replicate the exact workflow.” He said SiMa.ai chose the first group of models based on requests from its customers.
“The board doesn’t even need an internet connection,” López Roldán said. He explained that customers can download update packages at a place with an internet connection and install them later on devices in the field. This matters for devices that must keep working when their internet connection is unreliable or unavailable.
How Palette compares with Nvidia and Hailo
SiMa.ai competes with several kinds of edge-computing products, rather than just one product that does exactly the same thing. Nvidia sells complete Jetson development systems and provides its JetPack software environment. Hailo sells accelerators that can add local AI processing to another computer.
The prices listed below are for development hardware. A deployment budget must also include production hardware and the work needed to connect everything.
Platform | Access and published hardware price | Software and deployment model | Relevant distinction |
|---|---|---|---|
SiMa.ai Palette 3.0 / Modalix | DevKit 3.0: $1,499; production quotes through SiMa.ai | Vendor hardware with an Apache-2.0 Neat core and additional platform components | Focus on multiple models sharing resources under a limited power budget |
Nvidia Jetson Orin Nano Super | Jetson hardware and Nvidia's software stack | Lower-priced entry into the Jetson ecosystem; not the same configuration as Modalix | |
Nvidia Jetson AGX Thor | Jetson hardware with JetPack; commercial platform rather than an entirely open-source system | Targets demanding robotics and physical-AI applications | |
Raspberry Pi AI HAT+ 2 with Hailo-10H | Add-on: $200, excluding the Raspberry Pi host | Accelerator add-on with Hailo software and model tooling | 8GB of dedicated memory for local language and vision-language workloads |
Nvidia's Jetson Thor technical overview explains how to connect a development kit to an existing robot and use JetPack 7 to build and test applications. The prices in the table come from Nvidia's FAQ.
The Hailo-based option offers a much cheaper starting point for the hardware, but buyers must also pay for the Raspberry Pi computer, storage and other parts. Raspberry Pi's documentation says the system supports language and vision-language models with up to approximately 6 billion parameters. This differs from SiMa.ai's cited example of a 20-billion-parameter mixture-of-experts model. However, the number of parameters alone does not tell us how accurate or fast a model will be.
The enterprise decision is about the complete application
Palette 3.0 is most useful to teams whose models for understanding what a device sees and reasoning about it already work on their own, but compete for computing power and memory when they run together.
For a robot, the important question might be whether its object detection system can respond within its deadline while a language model generates an answer.
SiMa.ai's decoding results give numbers showing how much faster text can be generated. But the demonstrations with several workloads do not answer whether a system will meet its response deadlines. Developers will need to test this with their own applications.
Existing customers must also plan for changes when moving to the new version. SiMa.ai's release notice says that moving from platform version 2.1.3 to 3.0.0 requires a migration. Developers must also rebuild native applications using the software components that match the new version.
