Here’s a provocative question: Are democracies really always better than dictatorships or other forms of power organization? Does living in a democracy make people happy? If you have a ruler who makes responsible decisions for the good of the people, is it even necessary to involve the people in these decisions?
You admit then, that at least initially, you would need a democratic system to train the AIs alignment on. And if human values change gradually over time, as they have so far, this would need to be a regular process of retraining and realigning the AI.
Yes, I agree with you. I would also like to see the process of AI alignment be a democratic process that is regularly adjusted to reflect people’s values.