My 5 year old is having a great time with The Usborne Complete Book of Art Ideas and the Usborne Art Treasury. I stocked a cabinet with the supplies needed for most of the activites- papers, glue, paints, crayons, and pastels. DD loves opening the book and picking a project she wants to do.
She loves it so much that she doesn't let me forget it- all I do is read her the instructions and help her get the supplies out.
It's not a full art curriculum, so it would probably not be enough for your older child; but its a great way to learn a few techniques and experiment with different art media- I love that it's not about doing crafts.