This is what I've always assumed as well. The feeling of "things that usually don't matter but may explode (if it turns out the series doesn't converge)" seems to fit the expression pretty well.