Notice: This page requires JavaScript to function properly.
Please enable JavaScript in your browser settings or update your browser.
Apprendre Challenge: Implementing Positional Encoding | Section
Transformer Architecture

bookChallenge: Implementing Positional Encoding

Glissez pour afficher le menu

Task

Implement get_positional_encoding — a function that precomputes the full positional encoding matrix for a given sequence length and embedding dimension.

The function should:

  1. Accept seq_len and d_model as arguments;
  2. Return a tensor of shape (seq_len, d_model) where each row is the encoding for one position;
  3. Apply sin to even indices and cos to odd indices using the sinusoidal formula from the previous chapter.

Run it with seq_len=10 and d_model=16, print the resulting tensor, and inspect how the values change across positions and dimensions. Then create a random embedding tensor of shape (10, 16) and add the positional encoding to it to confirm the shapes are compatible.

Tout était clair ?

Comment pouvons-nous l'améliorer ?

Merci pour vos commentaires !

Section 1. Chapitre 7

Demandez à l'IA

expand

Demandez à l'IA

ChatGPT

Posez n'importe quelle question ou essayez l'une des questions suggérées pour commencer notre discussion

Section 1. Chapitre 7
some-alt