You are not alone Ipsis, though I've always been told I had Social Anxiety Disorder (which if they are considered unique disorders I suppose they are at least subsets or variant expressions of basically the same thing. In fact it took me awhile to find related threads as SAD doesn't appear to get any mention here...but I digress that's irrelevant). Forgive me, as this is a sort of combined response to your thread with an introduction to myself, I hope I'm not hijacking anything.
I'm writing this now in a somewhat dark place. I'm 34, male, single, and always have been. The loneliness can be quite heavy during these times, pressing down on me, sapping me of all my strength, motivation, and self-esteem. The older I get, the worse it seems to get. Having just observed a birthday (notice I said "observe", I tend not to celebrate my birthdays anymore), this arbitrary measurement of time always makes me take stock of everything I've missed out on, and how much harder it will be to ever have a relationship with anyone close to my age. But at the same time I'm able to live on my own, and somehow even hold a management position with my employer (I shake my head in dis-belief every day at this). I feel awkward in this position, a fraud most of the time really, and half expect to lose it at any time.
My disorder manifests primarily with the opposite sex, though I certainly have issues with my own gender, especially with: figures of authority, superiors at work, people I look up to, or anyone I perceive as intimidating or somehow threatening, which for me seems to be somewhat random; i.e. there are certain people I decide in my mind are intimidating but not for any single trait or characteristic I can put my finger on (this isn't a rational disorder of course, like any mental illness). The "avoidance" part certainly fits me. I can't count the number of times I've made up bullshit excuses to not attend a party or gathering, or even worked up the energy and courage to go, only to wait outside in my car, looking at the crowd of people through the window, and then leaving in a panic. Or, I may even get myself inside, only to leave in a hurry without saying good-by to my friends when one of those "intimidating" people or group arrives, whom I can't face for fear of making a fool of myself. And the longer the history of avoidance I've had with that particular person or click of people, of course the worse that anxiety is.
Unlike you I've struggled with substance abuse periodically (my brain just LOVES drugs), and I've pushed my poor brain (and body) to its limits more than once. As a teenager I escaped with pot, which of course most people I think would agree tends to aggravate social anxiety. So really I guess I'm not sure how much of my condition is self-imposed through abusing this drug, is genetic, learned behavior, or all of the above. Thankfully I stuck to this relatively benign drug for most of my impulsive youth, or I probably wouldn't be here at all. Later on I transitioned to the harder stuff, especially drugs that gave me temporary relief from this paralyzing, frustrating, and painful condition (ecstasy, cocaine, benzos, yum-yum). I'm sober for the most part now, and try to stay away from pot completely, though I still fuck up and sabotage myself from time to time. Anti-depressants have never worked for me however, so I'm currently experimenting with meditation, and may seek hypnotherapy in the near future.
All that being said, there are a handful of people I am comfortable with, and can even hold a conversation with, and if not for those people I don't know where I would be right now. This give me a small glimmer of hope, fleeting as it may be. Some days are better than others, and I'm optimistic about the future, if but for brief moments.
You just have to hold on to those moments I guess, and trust it will get better.