I have searched unsucessfully for an answer to this;
I have tried backing up my 8330 which is running a hybrid OS and with DM 5.01.
All goes well until it tries to back up BBGroups at which point it "hangs". I am normally a pretty patient guy but after 2 attempts, both times waiting almost an hour I give up & kill DM & reboot my BB
I've been having the exact same problem you describe. Unfortunately, I haven't found any answers to the problem. One workaround I've come up with has been when you're in DM, click on the Backup and Restore then go to Advanced. From here you can select which device databases you want to backup (in other words, skip over BBGroups) and you can at least get the rest of your stuff backed up.