I understand what you're saying, but on the unit, it seems like there is a small bezel, or gasket around it... What I was thinking was too remove that, and go bigger...
I personally been using the trackball without the holder, and have been doing so for quite some time. There are no issues with doing so, however, you can't install a bigger trackball than the oem because the housing for the trackball wouldn't allow it. It is meant to fit snug onto the mainboard. Hope that helps