Sort:  

Click on upvote again, and you will see an option to remove your vote