I believe that if something is common, it serves a purpose.
I've read Scott's The Rise and Fall of Online Culture Wars and it occurred to me: I suspect that people need to hate someone.
Or maybe not "need", but.. it's fun. Addictive. Hating your coworkers, hating your family, hating the government, hating Jews, blacks, gays, asexuals. Hating men, hating women. Hating racists, hating SJWs.
And it's not merely fun. There are also benefits: more bonding, and more meaning.
With this in mind, I suspect that what's been going on for the last few thousand years is a search for better ways to hate people. Better as in — not messing with the society, or serving an urgent need, etc.
If you accept this paradigm, it immediately creates a number of super interesting questions.
For example: Torah prescribes stoning for a number of crimes, which seems like a pretty fun outlet for hatred. However, with time the stoning laws were made more and more unenforceable. Why? What was the problem with stoning? Someone had actually put effort into stopping stoning from happening — what was driving them?
At this point, somebody will say "duh, stoning is nasty and people want less nastiness, it seems reasonable". I don't buy it. Cruel and unusual methods of torture (much more horrible than stoning) were invented pretty much everywhere. Romans loved watching people get eaten by lions. You can't say people don't like nastiness. My whole shtick is that stoning must have been adaptive at some point and became maladaptive later. What changed? How did it become maladaptive?
Another question: hating gays was a very widespread thing. Now there is a giant movement dedicated to hating people who hate gays. Say it with me: hating gays must have been adaptive at some point and became maladaptive later. What changed? Why was hating gays important? Why did it stop being important?
In the past 70 years, young lefty people have been switching from one anger target to another. Examples include: the anti-war movement, Occupy Wall Street, Black Lives Matter, New Atheism, the whole environmentalism thing.
The question is: are there any reasons for this switching, apart from the obvious "whatever happened most recently will be the issue of the day"? Sure, an anti-war movement makes more sense when there's a war going on, etc. But apart from that — why the switch from atheism and feminism to anti-racism? Is it more adaptive? If so, how?
Going to talk about American slavery specifically here. Why did America have slavery? Why did it stop having slavery?
The first question seems to have been answered by The Last Psychiatrist already: anything that lets you have cheap labor is good. Nothing personal. Just the thirst for cheap labor.
As for the second question, I don't think "because people realized slavery was bad" is the right answer. From Lewis S. Gould's The Republicans: a History of the Grand Old Party:
Slavery had receded from [the North] by 1853, but northerners did not have a coherent view of the institution’s future. Radical abolitionists, a definite minority, opposed slavery on moral grounds. Others disliked slavery because its spread might bring blacks into the North and West as competitive cheap labor. In 1848, northern opponents of slavery established a Free Soil Party that sought to block the spread of slavery in the West. Still others, driven by racist impulses, wanted African Americans to stay in the South or be returned to Africa. Whatever their attitudes toward slavery, residents of the North often resented the South’s political power and regarded the land below the Mason-Dixon line as backward, out of step with progressive currents of the nineteenth century
But why did slavery give the South political power? Because of the three-fifths rule: 3/5th of the slave population of a state counted towards its House of Representatives seats. The more slaves you have, the more seats you get. (The rule was originally instituted because the slave states did not want slaves to be counted as property for taxation purposes; if the slaves were to be taxed, they wanted to get something in return.)
The North needed slaves much less than the South did — no cotton plantations, no tobacco plantations. It also seems like the North liked industrialization more than the South did, and relying on free labor was messing with that. Not sure about this last point, though. Anyway, then the Civil War happened, the South was defeated and the slavery was gone. But there is still prison labor, there are still illegal immigrants, and a ton of stuff can be outsourced to India, so America will still have cheap labor for a while. Phew.
This is probably the most interesting question of the lot. Why was gender discrimination so absolutely ubiquitous throughout the time? Why was it important that women couldn't get higher education and couldn't vote? What exactly was this an adaptation for?
It's not like everybody was just dumb and evil. From Susan Goodier's "No Votes for Women", a book on the anti-suffrage movement in the US:
Women were morally responsible for protecting the private sphere and their families from the consequences of industrialization. [...] A well-functioning society depended on women to perform their function in the separate sphere of the home.
[...] [In the view of anti-suffrage women] the only way society could advance was for women to maintain their own sphere of influence and remain distinctly different from men.
They had actual reasoning! They had arguments for why the society wouldn't function well if women could vote. In fact, they even had arguments for why women's suffrage would make them less powerful rather than more:
[Anti-suffragists] feared the loss of women's power to influence the nation-state by using their uniquely feminine techniques. For suffragists, obtaining the vote meant having authentic power to influence the state. Most anti-suffragist women argued that obtaining the vote would reduce women to simply holding political power on the same level as men. To anti-suffragist leaders, such equality invalidated the power to reform the nation-state that women derived from their special sphere.
[...] Anti-suffrage women represented a significant portion of the population that feared that the tremendous changes of the Progressive Era would reduce or eliminate women's political and social power.
A giant problem with researching this stuff is that people often use the easiest available argument. If you feel women's suffrage is going to be bad for complicated reasons, and you can either think about those reasons or simply quote the Bible — you will quote the Bible. Then, in 2021, people will just say "everyone was dumb and believed the Bible, and that is why we must destroy the Bible". I don't buy this either.
If you have links/books that explore gender discrimination from the POV I want to explore it from, ping me — I'd love to hear about them.