I've rode alot of stuff out there on a rigid, and regularly ride it all on hardtail (SS FTW!). After awhile you learn to use a little finess instead of bulling over everything. I'm faster on a FS bike for sure, but it is good to relearn so of the finess and good line picking that you need on a SS for sure.
I learned more one day than ever. My freehub quit on me way in the back of the TOE, so I had to walk out. Well I started riding down stuff as much as I could, because walking sucks. I learned more that morning about being smooth and picking good momentum lines than I had learned in 4 months of riding. It was kinda crazy, but having to walk out like that made me a much better rider.