My freinds take advantage of the fact I like anime. When they ask me why I like it I just say "why do you like music?" or "Why do you like movies?" or something like that. There are some idiots out there who think all anime is hentai (Manga porn) so when I draw in class they always blurt out and say I'm drawing hentai.
All of them are idiots you don't need to keep it to yourself just come out with it. It doesn't matter what others think, all that matters is that your happy and that you enjoy it.