Here's the approach I've taken...
My pistol is normally worn at my hip in a holster on a belt. When I'm wearing sweatpants, I will wear my belt, not on my hip, but up higher on my waist, right where it's the narrowest. Now, I have to cinch my belt tighter, of course (if you can't do it tight enough with your existing belt you could always get a slightly smaller one). So, what I end up with is the pistol secured in its holster under my t-shirt or sweatshirt, and its profile is minimized. And, if necessary, I tuck the bottom portion of the holster in the sweatpants to ensure it's concealed. High and tight, that's the key.