If you don't think you're going to be there for the long haul, then no, I would not refinance unless you can get really low fees, points, etc. If it will put you ahead in the next 3-5 yrs then do it, otherwise it sounds like you might be moving anyway and then it won't have been worth it.
I don't know how far out you are but do you think it might get easier living rural as the kids get older and can drive themselves? Is it you that feels isolated or the kids? Just some things to think about... :)
Oh, and I would never stash money at home! You can invest in CDs if risk is a concern but at least you will make something on them.
And assuming your home loan is about 6-7%? You have to remember that will be lowest interest rate you'll ever get on any loan so it's worth keeping that loan vs paying it off and having to get another loan down the line for something else. Make sense? My DH explains this way better than me ;)