I've been in two book clubs and felt that same way. The only reason why I kept either one going is because I wanted to see the ladies in the book club.. It was the only time I got to see them. Now, I see none from my first book club and only 1 person from the 2nd one I joined.
I just don't like *having* to read other people's book choices. Kinda like being in school again. Our parish started a book club about a year ago, everyone keeps asking me to come, but nope - I just have no interest.
Plus, I've found that group discussion of a book is usually inane. Sorry, I guess that makes me a book snob.