Because the not-fascists are broadly satisfied to say “I told you so” from the moral high ground, and thus aren’t super invested in winning. Those that are are so idealistic that they cant actually politic effectively, think never voting for a moderate will convince the party to try harder to appeal to them instead of write them off as unreliable.