There's a big difference between being confident and having an ego.
My ex-boyfriend had the worst ego ever.

Literally, he thought he was above everyone, despite him being thoroughly average looking (nothing wrong with that) and having a fairly common job.
Being confident is being passionate and driven about something, but with a level-head. An ego is thinking you're better than/above everyone else.