Just a few weeks after chipmaker Tenstorrent raised nearly $700 million in funding, developers can now try out Tenstorrent’s AI accelerators on Koyeb. Tenstorrent sells AI processors built around the RISC-V instruction set architecture, and has developed its own open-source neural network library, TT-NN, and open-source low-level programming model, TT-Metalium.
Tenstorrent is part of a group of companies trying to build alternatives to Nvidia GPUs and the company’s CUDA library. It competes with Axelera, Etched, Groq and others.
Koyeb was founded by former Scaleway executives, and focuses on developing a serverless cloud platform for developers looking for an abstraction layer at the cloud infrastructure level. It competes with the likes of Fly.io, Railway and Render.
Koyeb lets developers deploy applications across several virtual machines using a command line interface or a git push after integrating with the code repository. It supports Docker containers and many popular languages.
One of Koyeb’s main features is that it can automatically scale an application to hundreds of servers if needed, and when there’s less traffic, it can automatically scale down server infrastructure.
In recent months, Koyeb has been focusing specifically on AI apps. Due to the serverless nature of its platform, it can offer a low-latency experience for AI workloads.
On the hardware front, Koyeb has deployed Tenstorrent’s PCIe boards in its data centers. Developers can access Tenstorrent’s low-level TT-Metalium SDK to write host and kernel programs.
Developers will find two new types of instances in Koyeb’s documentation and admin panels:
- The TT-N300S instance has 24GB of GDDR6 memory, 192MB of SRAM, and provides up to 466 FP8 TFLOPS. It is paired with 64GB of RAM and 4 vCPUs.
- The TT-Loudbox instance has four N300S. Developers get 96GB of GDDR6, 768MB of SRAM, and up to 1,864 FP8 TFLOPS. It features 256GB of RAM and 16 vCPUs.
With this release, Koyeb is trying to position itself as a hardware-agnostic cloud platform. “This reminds us of ARM’s debut on the server market with high-performance chips,” Koyeb’s co-founder and CEO Yann Leger told TechCrunch.
“Since we introduced ARM to the market with Scaleway back in the days, offering fully customized servers in 2013-2014, we have the experience of deploying various architectures and operating diverse hardware,” he added.
As for Tenstorrent, the AI chipmaker is looking for partners to build a developer ecosystem around its open-source programming model. It will take a village to offer an alternative to Nvidia’s AI stack.
Add comment