I actually wouldn't have a problem as long as there is consent.
I've always thought people were more accepting of poligamy than homosexuality. Especially when it feeds into the patriarchal dynamic that subjugates women and marriage for same-sex couples conflicts with that dynamic. It's only been until recently that it's become a slippery slope argument that people are demonizing it.
I do agree that it could lead to abusive and conflictive relationships and legal turmoil.