Keys, queries, and values are all vectors within the LLMs. RoPE [66] will involve the rotation on the query and essential representation