You're not alone. We've tried for five full years now and it's exhausting. I've got an only child so it's extra hard. My DD just celebrated another birthday with no one to invite. I don't know why it is so hard, but it really makes your heart hurt after awhile.
There are virtually no other homeschoolers here in our area and the public schoolers are incredibly mean. There are no coops, no homeschool support groups, no parent support groups, etc. We are secular homeschoolers so that makes it tough as well. We must have bad luck as we have encountered so many rotten kids/parents. After so many negative encounters, we have given up on trying to find any friendly public school kids and there are no homeschoolers so.....................It's been very tough. DD has also been burned many times, so she's not even the same child she was five years ago. She craves friends so we continue to hope.
I've tried to start a group six times over the last five years with no luck of any kind. I wish I had the answer. Feel free to vent.
(I did want to clarify our situation. We've put ourselves out there with no luck. We have since left the area and are now driving to another town (25 miles away) for activities. It is what it is.)