Rumors of a Baker exit flew after wife Susan Glasser was removed from her post.