To understand how attention works in modern neural networks, you need to grasp the roles of queries, keys, and values. These are not just abstract terms—they are vectors that help the model decide what information to focus on. Imagine each word or element in your input is represented as a point in a high-dimensional space. The query is like a search vector sent from one position, asking: What should I pay attention to? The key is a vector attached to each possible source of information, signaling: Here's what I have to offer. The value is the actual content or information that might be passed along if its key matches the query. Geometrically, you can picture the query and key as arrows in space; their alignment (how close they point in the same direction) determines how much information flows from the value.

Consider a sentence where you want to determine which words are most relevant to the word "bank" in the phrase He sat by the bank. The model creates a query vector for bank, which is compared to the key vectors of all words in the sentence. If the key for river aligns closely with the query for bank, it means bank is likely referring to a riverbank, not a financial institution. The value vector for river is then used to inform the representation of bank. This dynamic interaction—query searching, key matching, and value passing—lets the model focus on the most contextually relevant information at each step.