Often it's the rubber mount donut that shrinks, and the bearing/shaft is left to hop around.
If you buy the new bearing assembly, you may be able to swap the rubber alone easier that pressing a bearing off/on.
That said, if it's an old tired bearing, change the whole thing.