I think everyone has to find their own spread with which they are comfortable.
Right now with the market up so much and the mortgage rates so low, I can see that most people would prefer to invest the money and pay the monthly mortgage. Leverage is a powerful thing.
OTOH, if the mortgage rate on a 30-year were a more typical 5%, and the market gains were at the average of 8%, I can see why people would pay off the mortgage. I certainly would if I could. The difference for me would be a guaranteed "rate of return" vs taking a risk for a slightly higher rate.
So I think at some point the difference between the two rates becomes small enough that the possibility of increased return is offset by risk. That has to be part of the calculation when one does the math.