How it works

Cerebras develops the WSE (Wafer-Scale Engine), a chip the size of an entire silicon wafer, instead of cutting the wafer into several smaller chips. This allows more cores and memory to be placed in a single processor.

The WSE-3 has 4 trillion transistors and 900 thousand AI cores. On-chip memory eliminates the need to access external RAM, reducing communication bottlenecks between chip and memory.

The company also offers Cerebras Cloud, an inference API with speed superior to traditional GPUs for large models.

Practical example

A research lab can train a 70 billion parameter model on a single Cerebras system, without needing to distribute training across dozens of GPUs. The code uses the PyTorch compatible API:

Python
import cerebras_pytorch as cstorch
import torch

model = MeuModelo()
optimizer = cstorch.optim.AdamW(model.parameters(), lr=1e-4)

# Treinamento em wafer-scale
for batch in dataloader:
    outputs = model(batch)
    loss = criterion(outputs, labels)
    loss.backward()
    optimizer.step()

What it's for

Cerebras is for training and inference of large models where the speed and simplicity of a single chip are advantageous. It eliminates the complexity of parallelizing across multiple GPUs.

For those working with AI at scale, Cerebras offers an alternative to traditional GPU clusters, with lower communication latency and greater energy efficiency per token processed.