from my perspective, the israelites did lack faith at times throughout the 40 years, and they whined sometimes. when i read about their time in the wilderness, i don't think it's crazy though, but rather, i've always found it comforting. they were human, just like us, and often reacted in the same manner i probably would have. i've got to go to church, but i'm glad to share passage that i think the pastor could have been referencing, and there are several. or you could just email him. i think it's unfair to say how misinformed he was.
ETA - also in hebrews 3:7-9 in talks about the israelites lack of faith in the wilderness....so you can look at that too.