The answer isn't that hard... Gaga is way too controversial. Ever since Janet's nip slip, they play it super safe with the performers. They know Gaga will want to push the limit in some way, and the NFL really doesn't need that given all their recent scandals.
Also, I'd hate to say it, but Gaga is too much of a gay icon for an audience as homophobic as the NFL.