For me personally, it's generally something that is immersion-breaking. Hard to put into words, but those moments when you can tell an actor is counting in their head before saying/doing something, consciously making a facial expression that is just slightly out of place in their attempt to convey something that is out of reach for them, etc. As a general rule of thumb, if I believe the actor as the character, forget that I'm watching someone performing, and in perhaps the best cases start to subconsciously view the actor as the character (for better or worse

), they've done a good job.