My kids love Cal Academy of the Sciences, Exploratorium, Alcatraz, and though it is touristy--the cable cars are such fun. You might want to consider getting a city pass because the main attractions really add up fast.
Off the beaten path, if your boys like building things, Lawrence Hall of the Sciences in Berkeley, is a great math and science museum where they can design a roller coaster track, play math games, and build with thousands of kapla blocks. Your littlest might enjoy the steam trains down the street in Tilden park.
Christine W