Why do people find this very unsettling?

I'm very quiet and not a talker at all. And everyone I come into contact with seem to have a big problem with me. They always have to make comments like omg your talking way too much I can't handle it. Or ask if I'm psycho. And people that I talk to more think I'm weird and awkward. But i can relate like in pulp fiction when she says I don't understand why people have to yak about bullshit in order to feel comfortable, you can tell you found someone really special when both of you can just shut the f��k up for a minute and enjoy the silence. But I feel comfortable when people don't talk about like whatever. And when they do talk about whatever then I get really nervous. Ummmm yeah so what do u think? I don't mean to make people feel uncomfortable or frustrated with me. But I also don't give a f��k about trying to be "normal"