I can definitely relate. I mostly chalk it up to the fact that most people are idiots and feel the need to constantly fill the silence with idle chatter to avoid hearing their own thoughts in their head. Personally, I like my thoughts which is why I prefer writing. Gives me a chance to share them with people without having to engage in inane conversation about nothing. It's not really being introverted or social anxiety as I have no problem when I go out talking to people. I just don't care much about celebrities, sports, gossip or any of the other crap most people fill their time and their endless chit chat with.
The precise point I was trying to make (only much better stated- I shouldn't write after I hit the burn-out stage). People seem to expect you to carry the conversation for them and when you're not interested in celebs or that other inane crap... there's something wrong with you!