Pull the belt, run a string around where the belt would go. Then, eyeball the string to see if it is completely straight.

Straight edge might be easier.

 

Okay, so heres an update.
I looked and looked and it was certainly close to being aligned. I ended up checking it with a edge and it showed it was out a little bit. I pushed the pulley on about 1/8 from flush, and it does look better. Ive got about 2 hours drive time on it and it looks great, where as before it would show wear in about 30 minutes. I think its fixed, but i would have never of guessed a pulley no more than an 1/8 out of alignment throwing belts every couple of hours...

Anyways, thanks for the help guys. Always appreciated.
Erik.