“Big picture, it seems this would [lead to] a what-goes-around-comes-around. All the governors would be removed” after a presidential election, said Justice Brett Kavanaugh, and “we would really [have] at-will removal.”