Challenge: Implementing Positional Encoding
Sveip for å vise menyen
Task
Implement get_positional_encoding — a function that precomputes the full positional encoding matrix for a given sequence length and embedding dimension.
The function should:
- Accept
seq_lenandd_modelas arguments; - Return a tensor of shape
(seq_len, d_model)where each row is the encoding for one position; - Apply
sinto even indices andcosto odd indices using the sinusoidal formula from the previous chapter.
Run it with seq_len=10 and d_model=16, print the resulting tensor, and inspect how the values change across positions and dimensions. Then create a random embedding tensor of shape (10, 16) and add the positional encoding to it to confirm the shapes are compatible.
Alt var klart?
Takk for tilbakemeldingene dine!
Seksjon 1. Kapittel 7
Spør AI
Spør AI
Spør om hva du vil, eller prøv ett av de foreslåtte spørsmålene for å starte chatten vår
Seksjon 1. Kapittel 7