There is.... Just need a little research. This is all connected to one big Live account. I just went to my Xbox portion of my live account option and guess what. There is an option to remove my credit card. Now thats on a website connected to an email address that every time points are added gets emailed. The address is even given as part of the credit card charge. Now I don't have my X-box in front of me and it may not give an option to not remember (or delete), but then again if you don't want to use a credit card they sell Gold subs and points at best buy so you don't have to enter a credit card.I'm going to repost this, since a large majority of the people here seem to be missing this part.
You do not get the option to not save your credit card information. There is no option. If you pay for something on your xbox using a credit card, all your info is saved so you don't have to put it in again, you have no choice in the matter (other than to not use a credit card). There is no checkbox, there is no "remove card" link, there is no "delete", there is no notification asking you to agree to this, it's just how xbox live works. There are only two ways to remove a credit card from your xbox live account. You can add a new card and remove the old one, or you can call up microsoft and have them delete the xbox live account (this has to be done immediately, so you lose the remainder of your subscription).
I'm still not defending this pair of idiots, just trying to show how much of a scam this xbox live service is.
Microsoft does give you billions of ways including turning purchasing off altogether to not rack up 2k in charges throughout a 6 month period.