Notice: This page requires JavaScript to function properly.
Please enable JavaScript in your browser settings or update your browser.
Impara Challenge: Implementing Positional Encoding | Section
Transformer Architecture

bookChallenge: Implementing Positional Encoding

Scorri per mostrare il menu

Task

Implement get_positional_encoding — a function that precomputes the full positional encoding matrix for a given sequence length and embedding dimension.

The function should:

  1. Accept seq_len and d_model as arguments;
  2. Return a tensor of shape (seq_len, d_model) where each row is the encoding for one position;
  3. Apply sin to even indices and cos to odd indices using the sinusoidal formula from the previous chapter.

Run it with seq_len=10 and d_model=16, print the resulting tensor, and inspect how the values change across positions and dimensions. Then create a random embedding tensor of shape (10, 16) and add the positional encoding to it to confirm the shapes are compatible.

Tutto è chiaro?

Come possiamo migliorarlo?

Grazie per i tuoi commenti!

Sezione 1. Capitolo 7

Chieda ad AI

expand

Chieda ad AI

ChatGPT

Chieda pure quello che desidera o provi una delle domande suggerite per iniziare la nostra conversazione

Sezione 1. Capitolo 7
some-alt