I'd vote for the M5 all day, but that's just my personal preference.
A lot of people out there really like the M340. For me, the 3 series is just too small of a cabin for a daily driver.
Consider going out and driving a couple different models and that should help you weigh in the different factors that you're considering.
