Our experience:
My son took lessons first at age 6 in a very low-key environment; his coach recognized his ability and suggested competitive play which he did through the Jr. PGA primarily. He has competed since and in junior high joined the golf team at our local school. He now in his first year of college golf.
Starting with lessons is probably a good idea. Our local course has a great price for a junior membership which makes it very affordable. He also has been a member of Youth on Course which also provides discounts. Indoor facilities are a option, not necessary, and type of simulator at them matters. I'm assuming the cost is comparable to a lot of other sports' start up costs with golf equipment coming in a variety of prices; apparel is important as well and try eBay for both.
The culture is what you make it in my opinion. I have always been a consistent spectator (never a player though).
Please feel free to ask specific questions if I didn't answer something and good luck!