I, personally, DESPISE smart/unicode quotes.
Bear in mind when composing questions and answers that not every system is the same as yours - in particular, not every system has full unicode support (I occasionally read Stack Exchange sites in a terminal based web browser, with a terminal font that does not support smart quotes, so they look like this: �).
Rather than risk pasting gibberish into someone's environment I would much prefer to see standard 7-bit-ASCII straight quotes, apostrophes, and backticks (what you get when you type ", ' or ` in the text box with no special treatment) used as those work everywhere, and functionality trumps subjective aesthetics and typographical pedantry.
While I wouldn't go out of my way to remove smart quotes from a post, and would discourage others from doing so just for that purpose (smart quotes alone are not really a substantial issue) if I were editing a post for some other reason I would probably "fix" the quotes at the same time, depending on whether or not they were immediately bothering me.
Along the same lines, I will ask you to please refrain from "edit warring" over which direction quotation marks face. In the grand scheme of a world where SSL is broken and an airliner full of people vanished it's really not that important!