I successfully used the 'Button remapper' app (https://play.google.com/store/apps/details?id=com.irishin.buttonsremapper&hl=en_CA&gl=US) on my Nvidia Shield to remap the Red, Green, Yellow and Blue buttons on the X1 to open specific apps!
I was able to get my X1 to work with my PS5. Similar to your experience I added it as a Bluetooth keyboard. I then had to go into the settings on the phone app and remap the buttons by trial and error for it to function properly. I may have to go back and make some adjustments but I can scroll through PS5 menus with the remote without issue.