Ok, I gave up the multi calendar support, and gave up google mobile sync, and removed my gmail account again, and when I readded the gmail account, I choose to use the native calendar sync function in the blackberry email settings, and it worked.
I guess if I spend more time on this I may be able to find a way to get multi calendar support back through google mobile sync, but I don't think giving BB user the perfect calendar experience is very high on Google's priority list, so I won't bother anymore....