I agree with Kendra, to listen to your DH and his leading. Continue to pray, continue to seek His will and listen. Being quiverfull doesn't always = 20 kids either. You could leave it up to God and have no more children. You could leave it up to God and have 10 more children. Truely being quiverfull is leaving the #, the timing, thehealth, the financial and all of it up to God. Trust Him to provide all your needs as He sees fit.
We haven't had much money and things seem to be a struggle with us for money, but He has always met our needs and sometimes our wants, too. I don't think God ever promised life would be rosey, but He does promise to take care of our needs if we trust Him. Matt 6:31-34