Notice: This page requires JavaScript to function properly.
Please enable JavaScript in your browser settings or update your browser.
Aprenda Challenge: Implementing Positional Encoding | Section
Transformer Architecture

bookChallenge: Implementing Positional Encoding

Deslize para mostrar o menu

Task

Implement get_positional_encoding — a function that precomputes the full positional encoding matrix for a given sequence length and embedding dimension.

The function should:

  1. Accept seq_len and d_model as arguments;
  2. Return a tensor of shape (seq_len, d_model) where each row is the encoding for one position;
  3. Apply sin to even indices and cos to odd indices using the sinusoidal formula from the previous chapter.

Run it with seq_len=10 and d_model=16, print the resulting tensor, and inspect how the values change across positions and dimensions. Then create a random embedding tensor of shape (10, 16) and add the positional encoding to it to confirm the shapes are compatible.

Tudo estava claro?

Como podemos melhorá-lo?

Obrigado pelo seu feedback!

Seção 1. Capítulo 7

Pergunte à IA

expand

Pergunte à IA

ChatGPT

Pergunte o que quiser ou experimente uma das perguntas sugeridas para iniciar nosso bate-papo

Seção 1. Capítulo 7
some-alt