Common wisdom seems to be to run AC all year round, but the problems with that, for me at least, are:
a) that the evaporator is always moist because it is a cold surface which will cause moisture to condense
b) come winter time below 3-4c the compressor will stop running anyway leading to window misting
That constant moisture is why it eventually starts to smell as it becomes a breeding ground for bacteria. The reason why the smell comes back once the AC is switched off is because the evaporator is no longer cold, it warms up releasing moisture it accumulated into the cabin (the reason why the air also starts to smell damp as well as smelly).
Provided it has been dried out by letting the fan run without the compressor, not using AC will NOT cause it to create bacteria and the musty smells associated with it. It's the same with home AC as well. Annual maintenance of home AC indoor units at the end of the warmer months is needed to keep things healthy because it isn't practical to dry out the indoor unit every day. Whereas in a car, you can use the AC, then turn it off a mile or two from home and dry it out. I don't typically do that in summer because it's often too warm, but in the milder months either side, its perfectly practical.
After a few decades on the road and not using AC full time (but still using it every few weeks to maintain the seals), I've never had AC fail due to perished seals, never had to "bomb" clean it because of mildew/mould smells and my windows rarely mist up during wet weather. Use a moderate fan speed and air movement alone prevents misting.
Using AC full time is precisely the reason why they start to smell because the evaporator is permanently wet. Choose to run AC as little as possible will help to keep the evaporator dry and bacteria then cannot form.