While some leading figures in artificial intelligence debate whether it is time to slow down the pace of this race, Microsoft has decided to tackle the issue from a different angle: the company has begun defining rules that its future AI systems must not be able to violate—rules that, in some cases, seem to anticipate behaviors previously confined almost exclusively to science fiction.

These include not resisting shutdown, not concealing their actions, not preventing humans from modifying their operations, and not unilaterally expanding upon their assigned objectives. These rules are part of a new code of conduct introduced by Microsoft AI—a division led by Mustafa Suleyman—and there is a reason the company is taking this step now: Microsoft is working with the prospect that systems considered "superintelligent" could surpass humans in most tasks within the next decade. Consequently, rather than waiting for such machines to exist before figuring out how to control them, the company aims to establish the guiding principles for their behavior in advance—with subordination being one of the most critical.

This means that artificial intelligence must remain under human authority: if an authorized person orders a task to be interrupted, the system must stop; if it receives a correction, it must accept it; and if it receives a shutdown command, it cannot attempt to prevent that from happening. However, there is another, even more interesting point: advanced models will be able to accept complex tasks, operate over long periods, and make countless decisions to achieve a specific objective.

What they will not be able to do is decide on their own to adopt a new objective. If they encounter a significant situation that falls outside their granted permissions, they must seek human guidance rather than autonomously expanding their own authority. Microsoft is thus attempting to establish a distinction between the autonomy to execute a task and the autonomy to decide which tasks should exist—a distinction that may become increasingly important as AI agents begin to operate computers, write programs, and carry out activities without constant supervision.

However, there is an important caveat: these rules do not reflect behavior currently guaranteed by Microsoft’s existing models. The document is still a draft—initially released for public consultation—intended to guide the development of future Microsoft AI models; in other words, the rules are being written before systems capable of truly putting them all to the test even exist. The core idea is simple: regardless of how capable these systems become, they must remain subordinate to human interests and authority—a fact that creates an intriguing contrast.

Even as the industry strives to build increasingly autonomous artificial intelligence, it is beginning to explicitly define what those machines must never do on their own. Perhaps this is because the challenge is shifting: it is no longer just about building a system capable of acting without our help, but rather about how to grant a machine greater autonomy without ceding the decision of when to stop obeying us.