It's very hard to press buttons and select gems on the Android version. if my finger only slightly rolls or moves when I tap, it is considered a swipe and the button doesn't get pressed because the screen starts to scroll. Or when I'm using a swap power, or wven just playing the main game, sometimes it swaps in a random direction unless I very deliberately attack the screen from the right direction
It is the design of the game that when you tap on tile and then even if by mistaken as well you slightly rolls to any direction then it will consider it a swap. Usually, there are 2 methods for making the match. Either you can swap tiles by sliding to any direction or you can simply tap on one tile then remove the finger and then tap the other nearby tile which you want to select to make the match.