Goal-Content Integrity

I wrote a couple of weeks ago about Instrumental Convergence.


The thing that immediately struck me when I read The Superintelligent Will was that the very concept of Instrumental Convergence was exactly the neoreactionary argument for sovereignty.

If you have any long-term goals, the best way to achieve them in the short term is to accumulate knowledge and resources that can later be employed in the desired direction, provided that you can achieve what Bostrom calls Goal-content integrity.

Goal-content integrity means being able to hold to the same final goal over time. If you do not have confidence that your final goals in the future will be the same as they are now, then resources and capabilities that you acquire could be used by you in the future for goals other than those you currently intend.


If we model a polity as an intelligence with some long-term final goals (and I will address the problems with doing this later), then the logical instrumental goals of that polity are: self-preservation, goal-content integrity, increased capability, resource acquisition, just as Bostrom deduces. (I am rolling his goals of cognitive enhancement and technological perfection into a simpler “increased capability” — those goals are more important to his overall argument than they are to mine).


The difference between a reactionary polity and a liberal polity is that the liberal polity disclaims goal-content integrity. It does not have a long-term final goal, because it assumes that elements within it have different final goals, and they will continue to compete and compromise over those goals forever. Because it does not have long-term final goals, it has no steady interest in increased capability and resource acquisition. Conversely, a reactionary polity with a defined long-term goal, such as increasing the glory of the Royal Family, or of God, or both, will seek increased capability and resource acquisition.


The obvious problem with modelling the polity as an intelligence is that what that “intelligence” seeks is not necessarily good for anyone in it. However, this is where Instrumental Convergence becomes important. A polity that is seeking increased capability and resource acquisition is highly likely to benefit the immediate instrumental goals of its population. An inhabitant of a polity is either an asset, or a parasite, or a pet. An able human is still capable of being an asset, and as an asset is likely to gain from the resources and capabilities of the polity. Being a parasite to any polity of any kind is likely to cause you problems, so don’t do that. The role of humans as pets becomes interesting in the case of superintelligences (which I am not really discussing here, despite the starting point), but less so for human societies.


This is why it is better to be subject to a sovereign than to have a share in power: as a subject of a sovereign you are part of a polity with goal-content integrity, which, whatever its final goals, will pursue instrumental goals that will enable you to benefit. As a citizen of a democracy you are part of a polity without goal-content integrity, where the zero-sum struggle over the direction of the polity dominates any instrumental goals of increased capability or resource acquisition that you would be able to benefit from.