By way of introduction, we are a bilingual Japanese/American family living in Japan.
A children's picture dictionary is ejiten (絵辞典)or kodomo zukan (子供図鑑). The Junko Yoshida kodomo zukan that PP linked to are really popular but you are right, they are board books meant for younger children (0-4 years). Each one covers specific topics and I think they have a fair amount of vocabulary for those topics. You would just need to get more than one for sure to be sure to cover the topics you want to look at. For example, we have one that covers "around town" (shops, hospital, train station, festivals, etc.) and a general one that covers animals, fruit/veg, vehicles, insects, flowers, body parts, numbers. I'm not sure I would pay 18 USD for them though (the retail price in Japan is 1000 yen but you can easily get them used for around 500) but otherwise they might actually serve you well, if you got most/all of the series (I think there are 4 or 5 of them) .
I think this one might be more along the lines of what you are looking for though
https://www.kumonshuppan.com/gakusan/gakusan-syousai-jiten/?code=61023
If you click below the picture of the cover where it says 立ち読みする in blue you can see some sample pages (needs flash). It's also on Amazon but with no sample pages. https://www.amazon.com/Kumon-no-kotoba-ejiten-Tooru/dp/4774313386/ref=sr_1_1?s=books&ie=UTF8&qid=1544533744&sr=1-1&keywords=kumon+ejiten
Here's another that might be what you are looking for https://www.amazon.co.jp/学研-ことば-えじてん-無藤隆/dp/4053032946/ref=sr_1_1?s=books&ie=UTF8&qid=1544534199&sr=1-1&keywords=学研絵辞典
The second one has twice as many words (around 3000) as the first one (around 1500). That's just based of the publisher's specs online. I haven't seen either IRL.
These are both geared more towards younger kids (3-6) but might work for your boys?
This one is for younger kids too though (2-6 is the age range the publisher gives).