To delete a payment, go to the "Finance>Payments"

Locate the payment you want to delete and click the red "x" for that payment.