You are doing great! He pedaled for about 2 hours straight with very little stopping and more importantly, Very little coasting.....every time he would coast, I would almost run into him. It won't be long before you have to change your name to something more appropriate.
The cardio that you are improving will really help you when you are out riding the technical stuff. There is a lot to be said for being able to breath while riding moonscape,or gatorback.
Saddle time, spinning and yes miles.....Keep at it!