Why do many people confuse "racism" with common sense?

Just because I spent 4 years in Oakland Ca., attacked and robbed 4 times and now do not to be around a certain type of people, does that make me a racist? And please not the usual babel "No that makes you an idiot" crap.