I'm still new here, but my family recently did such a thing after lots of prayer and many years of preparation.
We had been working for 10 years almost to clean out from debt from our previous marriages and many youthful mistakes. Our credit was battered and we had no chance at a home loan. We lived in a lovely home that is huge for us 3600sqft but a small yard, we wanted a smaller house with more land.
The market is not horrible here, but not fast either. We knew in order to buy a new home we had to get a real fixer-upper and we work out of the home so we didn't have the option to move and live in a fixer.
Our solution, take a loan on our 401k. What we paid in taxes in the penalty for us in the long term would be saved by living from that day forward with no home mortgage again. We purchased a small home on five acres and paid cash for the property. In that time we were able to fix up our little farm as we had cash. It took almost a year.
We are living just shy of completely debt free, we are waiting on our current home to be put on the market and sold. Thankfully we don't have alot in the home and that will help us to repay the loan completely.
I love Dave Ramsey, listen to him alot, and each family is different your own financial situation may vary from others. Normally, this would not be my first choice, but for us it was a leap towards a better future.