Likewise, it was an alternation of ingrower hell and post-shave deep tissue irritation that led me to traditional shaving. I think in the six months that I've been traditional shaving, I've had a mere handful of ingrowers which have thankfully worked their way to the surface easily. These have come about from over-stretching my skin, particularly around my mouth which I've been able to get at with tweezers and a little skin breaking and douse of TCP afterwards - not a good technique, but works out for me. I'm not a pretty boy
A few things that have helped me along the way ...
First, use a sharp blade and roll in a fresh one frequently. Don't push your blade too far. If you're prone to skin issues, discard after three shaves.
Second, I believe that the condition of skin can be dramatically improved by using olive oil based pre-shave oils, shaving creams and soaps. For me, this seemed to balance my skin condition rather than alternating between dry and oily.
Third, twice a week, a good wash with an abrasive face wash. I like T&H No.10. Walnut husk and grit, I think
Worth doing while you have stubble, or on days in between shaving. This will help bring up any ingrowers.
Fourth, toning. There's another thread about this, but I think if you're prone to skin issues, using as simple a toner as you can prior to bed is a good thing. Simple witch hazel. Yes, it will tighten your skin, but if used as part of a regimen that includes skin moisturising with olive oil pre-shave and a damn good wash, then it won't leave you overly dry. Reconsider those moisturisers and balms.
Finally, if you do feel like you've just had too close a shave and know instinctively that you'll have ingrowers, keep a bump repair product handy and use it frequently. In my early days traditional shaving, I used Shaveworks ... which I believe was originally marketed for folks who shaved their "intimate areas". Chaps prone to ingrowers seemed to get on with it. It worked, but left a sticky film on my skin which I put up with but didn't like. Then, I found Clubman Club Mend. Excellent product!
I'm not saying this is the "be all and end all" to dealing with face issues, nor that it is dogmatically right for everyone; just that here's a bunch of things that for whatever reason worked out really well for me ... and now I can shave with just about anything, any time and feel only sheer joy afterwards.