On another forum a guy used Naptha. I tried it and it worked perfectly. Just used a rag to put it on then rub the adhesive off. I bought a gallon at WalMart and threw away all of the Goo Gone and similar products I have used. I refilled bottles with the Naptha. It does not affect the paint at all.

 

Naptha does work well, but it's hard to beat the 3M adhesive/decal remover aerosol.

I use it a lot when removing stickers and decals.

Heat gun helps remove the bigger chunks of stubborn felt/adhesive, then the 3M aerosol to finish it up.

 

Here's the trick, no eraser or other junk required:
After using the fishing line to saw off the badge, carefully start prying off a corner of the remaining spooge with your fingernail. Now here's the trick, after you have enough started pull it away and down kind of opposite of the direction you are going, like the command strips. Avoid the temptation to pull up or it will just break off that little piece and you'll have to start over. Helps to have area warmed up with heat gun too.