I was thinking about that or a blanket or something. A friend of mine suggested sticking it in one of those portable coolers but I don't have one and can't see buying one just for one day's use.
The only thing that I can think of that might cause problems (probably not) is if it was really cold and you brought it into a really warm enviroment. The condensation that the temperature differences would cause might get too much moisture inside the phone. But I wouldn't worry about it at all. If you just leave it on your seat or in the glovebox, and then when you get in the car the process of heating up the cold car would fix any issues it might have being cold.
Now leaving your BB all alone in your cold car is another issue, it doesn't sound very nice. I'm sure it will get over it though.