Yeah I'd check to make sure it's not packed full of crap like said.

Mine sprung back as far as I can remember when I did it on my truck. It's been a couple years but I think it sprung back.

 

if it doesn't spring back then it could possibly get pulled toward the driver side and the belt could pop off. that wouldn't be good.

mine did the same thing and i got one from a junk yard for 35$ that had a good spring to it. its only 3 bolts that hold it on, easy to change.