You are aware that you have a pinky? It's typical for fiddlers not to use the pinky and not to use the lowest quarter of the bow where the balance of the bow shifts most. If you are playing classical violin, you are expected to be able to make use of the full bow. That means that there is significant change of weight and contact when getting near the frog. Your middle and ring finger and the thumb are the principal hold and pivot for the bow, index finger and pinky are for weight control. If you let the pinky hang in the air, you don't have a counterbalance for the index finger and have to do everything with the index finger.
Have you tried controlledly and quietly closing a door with a sticky latch? The most reliable way to manage that is to pull it close with one hand and push with the other. That way when overcoming the yielding point, you don't need to counteract with the pulling hand but have the pushing hand balancing it. It's not impossible to close a door even with sticky latch well with a single hand but it is much more prone to accidents.
In a similar manner, if you have only the index finger putting on pressure without a counteracting control from the pinky at the other side of the pivot, adapting the pressure when approaching and leaving the frog just doesn't work as well. And when you have to take too much weight off because you are missing the counterweight, the stick slips out of control and your index finger is displaced when you move back to the tip.
Make no mistake: there are also some classical violinists making no use of their pinky but it's not making things easier. And the typical pinky-free fiddler will just not play near the frog and will thus have less need to adjust the pressure.