I feel like it's borderline doing the most. Since you're probably straight, what would you do if a suspect-ish guy said that to you?
I would find it funny no matter the gender of the person. I would only have a problem with someone bussing jokes on my cast and I already disliked them.
You say he's probably straight so you have reason to believe he likes boys too? All the more reason to write it.