I'm one of many people who can get a bit frustrated by the trackpad not always working out which axis I swiped correctly.
I've been wondering if this is simply down to how we use it. I am a lefty, so I hold it in my left hand and swipe with my thumb. The point being, my thumb movements aren't perfectly axis-aligned to the remote - the base of my thumb acts as a pivot, so instead of swiping perfectly right for example, I actually move down a bit as I go across (it is technically an arc, but who wants to code curved axes?).
So one thought as a test is to have a calibration feature in my app, where you repeatedly swipe in each direction so I can determine *your* up, down, left and right axes and work from them to determine what you actually meant.
I imagine the end result will be slightly rotated axes, for lefties clockwise, for righties, anti-clockwise.