On writing physical character description

I have a hard time writing about how people look. Clothes, no problem. The look in their eyes, the way they move or smile, I got it. But their physical look? I have to sit and ponder. Really think about it. Because on a fundamental level, I don’t see their looks when I look at them. I see their personality. Or my impression of that personality based on intuitive logic if we’ve just met. I call it, somewhat pretentiously, seeing with my heart. This doesn’t mean that I don’t see the physical, just that it’s not the first thing I think about when I think of someone. So, in my writing, I have to dig fairly deep into my own head to actually come up with the look of someone. I have to build them so that I know them entire before I know what they look like.