Challenge: Implementing Positional Encoding
Stryg for at vise menuen
Task
Implement get_positional_encoding — a function that precomputes the full positional encoding matrix for a given sequence length and embedding dimension.
The function should:
- Accept
seq_lenandd_modelas arguments; - Return a tensor of shape
(seq_len, d_model)where each row is the encoding for one position; - Apply
sinto even indices andcosto odd indices using the sinusoidal formula from the previous chapter.
Run it with seq_len=10 and d_model=16, print the resulting tensor, and inspect how the values change across positions and dimensions. Then create a random embedding tensor of shape (10, 16) and add the positional encoding to it to confirm the shapes are compatible.
Tak for dine kommentarer!
Spørg AI
Spørg AI
Spørg om hvad som helst eller prøv et af de foreslåede spørgsmål for at starte vores chat
Challenge: Implementing Positional Encoding
Task
Implement get_positional_encoding — a function that precomputes the full positional encoding matrix for a given sequence length and embedding dimension.
The function should:
- Accept
seq_lenandd_modelas arguments; - Return a tensor of shape
(seq_len, d_model)where each row is the encoding for one position; - Apply
sinto even indices andcosto odd indices using the sinusoidal formula from the previous chapter.
Run it with seq_len=10 and d_model=16, print the resulting tensor, and inspect how the values change across positions and dimensions. Then create a random embedding tensor of shape (10, 16) and add the positional encoding to it to confirm the shapes are compatible.
Tak for dine kommentarer!