Got it. If I'm following correctly I would say in that case no you are not racist for not wanting to associate with that group. I would not want to associate with a group that established it's whole state based on ethno/religious supremacy either.
You and I having an opinion of that group based and its members past behavior is not racist. That group is very well off financially and you and I having an opinion about them even if ignorant or untrue doesn't really alter that groups reality or proximity to power which are two things that are supposed to be the Hallmark of racism.