I am on a very similar path as you, Laurifer. For me, Vocals are number one. They are the deciding factor in me listening to and enjoying a band or not. This is due to (what a surprise!) the fact that I'm a vocalist. Vocals are what I focus on, but I think this goes for anyone who plays an instrument. What they play is what they focus on, so yeah, mine is vocals.
After that is Lyrics, because I listen to a lot of black metal, and black metal can get terribly tedious if everything I listen to is a description of the cold, ancient pagan wastes of elder worlds and sacrificing children to bloody, stone-chiseled deities. (Not that that is what I am listening to, but just an example). I write my lyrics and poetry with a certain degree of skill, imagery and guile, and it can only be assumed that the music I listen to will have lyrics of a similar quality or proclivity. I like lyrics to be well-written, poetic, or fun. In the case of metal, lyrics are not always going to be immediately discernible, so I enjoy reading the lyrics along with the music, and if I find that the lyrics are boring or tedious, I will lose interest in both lyrics and song. That's just how I roll.
And yeah, after that falls atmosphere, performance, structure, all that good stuff. But none of it is as important to me as the aforementioned two.