I've both worn them and not worn them on XC trails...for me I guess it depends on what I'm out to do. If I'm out to just ride clean, fast and hard, then I don't. I'll occasionally put them on when I'm out to try to push what I can do on some of the trails, like trying to hit whoops like you do dirt jumps and get as high as possible...sometimes I don't. I didn't last Sat., and I wish I had right now....
what are you gonna do though!!
Going up to Santos for any light freeriding (I don't have the balls or the bike for any of the bigger stuff) I always put 'em on - I get enough cuts, bruises, and scars even with them.
So, I guess, if it helps you have the confidence to push yourself, then my thought is who the hell cares if they are "uncool."