If anyone wants to use a controller for a mouse on Raspberry Pi you can install this:
xserver-xorg-input-joystick
It saved me time rather than trying to configure joy2key.
@Kimiko_0 Thinking about it. I think it's because the gamepad has a different default mapping for the buttons. I know on a few of my Nintendo USB controllers the buttons were all different and some didn't even have a mapping.