Well well well... Setting up a simple blog is more work than I thought.
With work stealing all my time, finding some time to actually dig into setting up a blog meant skipping a night's sleep.
Here's my choice: a plain Drupal 7 with the following modules: Markdown, Tagadelic, CAPTCHA, Ajax comments, Pathauto.
I won't go into too much details about setting up Drupal, since it's pretty easy for such a simple site. I got most of my inspiration for the setup from this site.