The bit that is slipping might be metric and they have a smaller diameter. I use a different collet for metric bits or crank the nut if I’m too lazy to change it out.
After replacinging collets a couple of times for the same issue, I began cleaning the collet and chuck area at each tool change. Knock on wood, I haven’t replaced a collet in a year or so.
My compressor is right there for short run bit changes I remove bit and blast a bit of air in the collet before putting next bit in. If it’s been running for 1hr or more, I’ll unscrew collet and clean with air before putting in next bit. it’s been working for 2yrs so far.