Nope. I pushed in with the socket. You do have to push those clips in, or it won't turn.
It's just when you turn it over, upside down, the inner part looks close to falling out.
I would have thought though, that just pushing in with the socket, and turning the socket with only my fingertips...