Mmm I don't know. I think it just boils down to knowing how to dress for your body. I'm 6'1, with a runner meets swimmers body and I used to wear regular fit clothes and I almost always sort of drowned in them it was not hot but then I noticed I look good in more body contouring clothes like skinny jeans, slim fit shirts, jackets etc and I get compliments on that and I'm fortunate I can get these in perfect fit off the rack.
But then my brother is 5"9 and well built, really buff. He stays away from the very tight clothes and never wears skinnies etc, he goes for his regular size & his clothes are more tailored for him, for example his chest size and height sometimes don't collerate are so he gets that fixed and it looks amazing, he also pulls it off so well, so even if he can't get it off the rack he still makes it work.