Oh that doesn't surprise me, and no, I don't hate him JUST because of Twilight. I LOVE the books and the movies overall haven't been half bad (namely Eclipse and Breaking Dawn are the best).
But I just don't find him attractive and I think he isn't that good of an actor. Same goes for Kristen Stewart. I saw Remember Me and Welcome to the Reiley's solely for the fact to see if Pattinson and Stewart COULD act.. they can't. It seems to me that Pattinson plays the same character in everything and Stewart has the same range of facial expression as a plank of wood. To me, Stewart RUINED Snow White and the Huntsman. Her yelling at the end to rally her troops is just one example. All I could do was role my eyes and not laugh.
I'm not trying to hate on you, I'm just sharing my opinion. I LOVE your work, it's all so beautiful.