Keep in mind that you don't need to justify your declination of the invitation. An invitation is not a summons and no one owns the control about where you spend your holiday time but you. Go with a form of "that doesn't work for me" or "sorry, but I already have plans here at home".
There is absolutely no need to go there and be a human meat shield, taking abuse.