I often trash movies for poor handling of firearms, but there is a saying that I've heard in movies that is correct when it comes to drawing a handgun: Slow is smooth, and smooth is fast.
Rushing leads to misses, and you can’t miss fast enough to lose in a gunfight. Your draw should be smooth and practiced to the point that it is completely natural. You should practice your draw often, both at the range (if possible) and with an unloaded handgun in front of a mirror. Here, you can see your mistakes and learn how to smoothly draw, aim and press the trigger.
If you carry in multiple locations, try practicing a couple of draws when you strap on in the morning with an empty gun. This will help you remember where your gun is located and provide the smoothest way to bring it into play if needed.
