It seems like everyone here, including myself, are baffled as to why you would want to hold back a child that is not only doing fine academically but doing well.
I sympathize with your husband, but he skipped a grade. That is not really relevant here because your son did not skip a grade.
If you go to the middle school or high school in your town and walk into a random class and have everyone announce their birthdays there will be kids older and younger. Some almost a year older than others. That happens everywhere. Your son isn't going to be the only kid with a summer birthday. There will be plenty.
I also have a summer birthday, which means I turned 18 after I graduated. I was also one of the smallest kids in my grade at 4'11" and 90lbs. I have literally been this same height and weight since I hit puberty in 7th grade. (Well except for when I was pregnant of course) It never mattered, no one ever treated me like a baby because we were all the same grade. No one but my closer friends knew when my birthday was anyway.