How it works
In AI, matrices (and tensors, which are generalizations with more dimensions) represent data. A color image is a pixel matrix with three channels (RGB). The weights of a neural network are matrices. Operations like matrix multiplication are the foundation of calculations in LLMs and other models.
When an LLM processes a sentence, it is converted into tokens. Each token becomes a vector (a row of numbers). The set of vectors forms a matrix that passes through the model's layers. Each layer applies matrix multiplications to transform the data.
GPUs are essential because they do matrix multiplication in parallel. A CPU processes element by element. A GPU processes thousands of elements at the same time.
Practical example
Imagine a 28x28 grayscale image. It is a matrix of 28 rows and 28 columns, with values from 0 to 255. A vision model processes this matrix, applying filters (smaller matrices) to detect edges, shapes and patterns.
In an LLM, the sentence "o gato dorme" becomes three tokens. Each token is converted into a vector of 4096 numbers (depending on the model). The resulting matrix has 3 rows and 4096 columns. This matrix passes through the model's attention and feed-forward layers.
What it's for
Understanding matrices helps you understand how models process data. It also explains why GPUs are necessary and why larger models require more memory. For anyone working with AI, knowing how to manipulate matrices with NumPy or PyTorch is a fundamental skill.
Difference between matrix and tensor
A matrix has two dimensions: rows and columns. A tensor can have more dimensions. A scalar is a zero dimensional tensor. A vector is a one dimensional tensor. A matrix is a two dimensional tensor. Images are three dimensional tensors (height, width, color channels).
In practice, frameworks like PyTorch and TensorFlow work with tensors. The term matrix is used more generically to describe data organized in a grid.