Go on your trips. Staying home won't help your anxiety, going out, distracting yourself, will. I get that you might get embarrassed, but it's only a might. If you lock yourself away, it will definitely reinforce your anxiety. Don't worry about stuff you have no control over, it achieves nothing.
I know though, easier said than done.