I'm basically an optimist because I do think there's this historical modernisation process, and by and large it's been very beneficial to people. But there are blips. History doesn't proceed in a linear way.
I've always had a Marxist understanding of history: democracy is a result of a broad modernization process that happens in every country. Neocons think the use of political power can force the pace of change, but ultimately it depends on societies doing it themselves.
The rationale for tenure is still valid. But the system has turned the academy into one of the most conservative and costly institutions in the country. Yes, conservative: Economists joke that their discipline advances one funeral at a time, but many fields must wait for wholesale generational turnover before new approaches take hold.
I've figured out in the course of my life that the one thing I'm good at doing is writing books, and it would be crazy to trade that in for something else.