Well...
He added California AND Texas.
So, in fairness, that's one Union state as well as one to-be Confederate state...
And I'd also say that Polk didn't do any more to bring about the Civil War than any other leader before him...really, it was inevitable the moment the Founders decided to sort of skirt the issue of slavery in the Constitution and make that a states-choice deal and leave it for another age:
Which was a fine move, a logical one...the colonies could barely agree on the Constitution as it was, if the Founders had tried to tackle slavery then as well, there's a good chance there'd be a CSA in existance today, and it very well might have sported the likes of Washington and Jefferson among THEIR first Presidents...
So the Civil War was, really, inevitable, and it seems unfair and rather spurious to blame Polk for it due to unrest in the nation that had, essentially, always been there since the beginning--hence the Missouri Compromise, Compromise of 1950, compromise after compromise to keep everyone appeased, states joining in twos, one Free for every Slave state to keep the Senate balanced--and for a War that was destined to happen since the beginning.
And...
He DIDN'T fight the Mexican-American War to expand slavery, just the US, and, again, he expanded both the Free and Slave sides immensely with California and Texas, so, again, that seems to offset...
And as far as his destroying the Jackson-Whig system...
So?
Political parties die...I fail to see why this case was such a horror...
Because it left a power vacuum politically?
Well, politics can do that, and it WAS filled...by the Republican party.
Granted, that's a horrifying thought...but hey--we at least got Lincoln and TR out of the deal. ;)
So really, your argument against Polk comes down to he sort of maybe contributed to a political party that was already ailing dying, which is natural, that there were bad political moods during his term, which there always were and always will be as long as the US allows freedom of speech and different ideas, and twenty years on from him we had the Civil War, which was inevitable from Day 1 after the Constitution.
So I'm not convinced by your case against him, really...it seems to rely far too much on unfair hindisight, and even that's somewhat blind, as matters such as the Civil War and Free/Slave states were out of his control, it seems unfair to blame him for them...