It's not racist not to be attracted to black men, just as it's not misogynistic for gay men not to be attracted to women but when people go out of their way to discriminate with all that 'no blacks/no Asians, etc.' mess, it can definitely come across that way.
I don't understand the concept of not being attracted to a certain race. It's not like everyone of said race looks and acts the same and has the exact same interests? So shallow. I hate it.