Actually, you can do it from DM. Go to Back up and Restore > Advanced, and in the right window pane you'll see Device Databases. Select the Address Book, and I assume the Address Book All, and click Clear. All gone! I assume you would choose both, I've never cleared mine but I've cleared the SMS's, Cache, etc. from here.