Are your handlebars too narrow?
I ride bar ends and dont have a problem, but have a Sram/Avid set up and my bars are quite wide.
I ride bar ends only because I always have. Learning to ride in CA they were great for long climbs. Now it just feels like my hands want to slip off the bars without them so I keep them on. They do offer alternate places to put your hands if you get tired of the same positions on longer rides.
(I have had one crash...barend + vine + speed = DOWN