I've pretty much come to the conclusion this is the reason for my ABS and parking brake light staying on.

What do I have to do to get this thing out? I've already unbolted it but it doesn't want to easily pop out. Any more force and I can see the thing breaking.

 

After taking off the nut that holds on the brake lines, did you take off the other nut? If so, you are almost there. There isn't much room to work under there, but take a flat-edge screwdriver and gently tap around the base with a hammer. It won't break, butwill come out, it's just a little snug as you will find out when you put in the new one. I had the sameissue with both lights on. Pop in the new sensor, go for a short drive and no more light issues.