They're such different watches. I've had a look at both, plus the 5396.
The 5146 is traditional classic Patek IMO. I agree it does seem to go with casual wear, but it's certainly a dressy watch as there is a lot going on on the dial.
The 5205 couldn't really be described at sporty, but its' clean lines and interesting dial make it more of an allrounder, but it is quite thick for a dressy watch.
The 5396 sits somewhere between the two - it's cleaner than the 5146 and thinner than the 5205.
I think Patek is actually being very clever providing these three choices in the (relatively affordable) annual calendar range. By the time you throw in the Nautilus AC, there really is an AC for everyone.
On balance, my favourite is the 5205, with the darker dial but I think they are all fantastic and it really is personal preference.