Just curious. I would guess not based on the way the inner wheel is designed to to hold the bearings in place, but I thought I would ask. Bearings are expensive and I would prefer not to muck them up.
Spacers will not hurt your bearings. Going without them probably won't either but could cause you to be slower if the space between the bearings is not 100% correct. Rubber wheels are not always precise to spec because they sometimes change size as they cool after being molded. The spacer helps hold the bearings straight in case the wheel's holes are slightly off.
Comments