While it is hard to comprehend events that happen in a millisecond, for most models that is a very long time.
If you have a relatively long duration, consider reducing it and applying your loads and boundary conditions in a reduced duration range. If you cut your duration from 1 second to .1 second your job will run 10 times faster. Reduce the duration to .01 second and your job will run 100 times faster.
Always move impacting bodies close together intitally, until they're almost touching. This allows the specified duration needed to be as small as possible, while still computing the time just before, during, and immediately after impact has occurred. The solver spends as much CPU time computing the 'dead air' timesteps of the body flying through empty space, as it does calculating the material disintegrating, and then deleting those elements.