When you go back to App World the third menu item on the top is My Apps or My World something like that, it will list your current apps and display if there are updates, it's a simple click and download and install. So far i don't believe any of the apps i have updated required device restarts but that may change once we apps that dig deeper into the os and try to deliver on RIM's superapp pledge.
If you check there may already be some, i know the games had updates but i deleted the games off the PB.