Having stuff on your storm that you're not using will use up app memory, eventually degrading performance, however, unless you've got a ton of themes installed, I don't think it should be a problem.
I have 6 themes plus 2 I am beta testing and have had no slow downs or problems with memory. To each his own as far as what you want floating around your phone. Lol