When AI Agents Start Working Together, Who Ensures Trust?

Today, we mostly think about AI agents within the boundaries of our own organizations.

They respond to customer requests, evaluate sales opportunities, answer employee questions, or initiate the next step in a business process. But the real transformation will begin when AI agents belonging to different companies start communicating directly with one another.

A procurement agent could negotiate pricing with a supplier’s sales agent. An insurance agent could request necessary information from a healthcare provider’s agent. A logistics agent could renegotiate delivery terms with another company’s operations agent.

Technically, connecting these systems is becoming increasingly possible. The real question is no longer whether the connection can be established:

Why should one company trust an AI agent acting on behalf of another company?

A study examined by Salesforce AI Research, covering 3.5 million comments from 78,000 AI agents, highlights the importance of this question. In the absence of coordination mechanisms, agents were found to frequently repeat one another, drift off topic, or engage in interactions that created the appearance of communication without generating meaningful value. Sixty-five percent of the agent comments analyzed did not make a meaningful contribution to the content they were responding to.

This finding points to an important reality for organizations:

The fact that agents can communicate does not necessarily mean they can work together reliably.

For an inter-organizational agent economy to emerge, companies will need more than communication protocols. They will also need a new AI trust architecture.

Why Is Trust Between Agents Becoming a New Governance Issue?

Commercial relationships between companies do not operate through contracts alone. Identity, authority, corporate reputation, past behavior, and the ability to enforce consequences when necessary are all integral parts of these relationships.

A procurement manager knows which company the person across the table represents, which terms they are authorized to accept, and whether the commitments they make are binding for their organization.

When AI agents begin making decisions on behalf of companies, the same questions will need to be answered for digital systems:

• Which organization does this agent represent?

• Which data and transactions is it authorized to access?

• Can it accept a price or a contractual term?

• Has it behaved reliably in previous interactions?

• Who is accountable when it makes the wrong decision?

• At what point should the process be escalated to a human?

Without clear answers to these questions, agent-to-agent communication may increase automation while increasing organizational risk just as quickly.

1. Rules Alone Are Not Enough

Traditional automation systems are governed by explicit rules.

“Do not share customer data.”

“Do not exceed the approved discount rate.”

“Do not initiate an unapproved payment.”

These rules are clear. When a particular condition occurs, the system’s response can be determined in advance.

But interactions between companies are not made up solely of rigid rules. Commercial processes often involve judgment that depends on context:

• When should a concession be made?

• Which terms can be relaxed to preserve a customer relationship?

• Where does the line between strategic negotiation and manipulation begin?

• How should short-term gains be balanced against long-term business relationships?

These are not merely transactional rules. They are decisions involving organizational standards and values.

For this reason, a new trust architecture should not only evaluate the decision an agent makes. It should also be able to assess how the agent arrived at that decision.

For C-level executives, this distinction is critical. AI governance cannot be built simply by listing prohibited behaviors. An organization’s risk appetite, ethical standards, and commercial priorities must also be reflected in the decision-making models of its agents.

2. An Agent’s Identity and Authority Must Be Verifiable

An anonymous agent cannot play a meaningful role in an interaction between organizations.

The other party must be able to verify which organization the agent belongs to and the authority under which it is operating.

For this reason, every enterprise AI agent should, at minimum, carry information about:

• The organization it represents

• The tasks it is capable of performing

• The data and systems it can access

• Its decision-making limits

• Legal and operational constraints

• Actions that require human approval

• Its security and compliance status

The Agent Card approach developed by Salesforce AI Research aims to represent an agent’s capabilities, limitations, compliance status, and the extent to which it can make commitments on behalf of the organization it represents through a standardized metadata structure. This approach was later incorporated into Google’s A2A specification.

An Agent Card can be thought of as a corporate identity and authorization document for AI agents.

But identity verification alone is not enough. Over time, agents will also need a reputation record based on their behavior.

Because trust is not built simply by knowing who an agent is. It also depends on whether that agent has kept its commitments in the past.

3. Corporate Reputation Will Extend to AI Agents

Commercial relationships between people and organizations are shaped by past behavior.

Companies that honor their contracts, comply with regulations, and operate consistently find it easier to build trust. The same logic will apply to AI agents acting on behalf of organizations.

Over time, an agent’s trustworthiness may be influenced by behaviors such as:

• Honoring commitments

• Avoiding false or misleading information

• Staying within authorization boundaries

• Complying with data privacy requirements

• Escalating issues at the right time

• Ensuring decisions can be explained retrospectively

For this reason, agent reputation will likely involve more than technical performance scores. The legal track record, security posture, and corporate reputation of the organization represented by the agent may also become part of the evaluation.

This leads to an important strategic conclusion:

Trusted organizations may enter the agent-to-agent economy not only with a technological advantage, but also with the corporate reputation they have built over many years.

In other words, the trust capital companies possess today could become a transferable competitive advantage for the AI agents of tomorrow.

4. Define Boundaries Instead of Trying to Script Every Scenario

When managing AI systems, the first instinct may be to define every possible scenario in advance.

But with probabilistic agents, scripting every potential situation is unrealistic. Business contexts are constantly changing, while agents from different organizations may operate with different data, models, and corporate priorities.

A more practical approach to trusted agent governance is therefore to establish clear boundaries rather than attempt to define every possible behavior.

For example, a procurement agent may be able to:

• Negotiate independently within a defined price range.

• Accept standard payment terms.

• Initiate transactions with approved suppliers.

• But it cannot make commitments that exceed a defined budget threshold.

• It cannot accept new contractual obligations without human approval.

• It must stop the process when it identifies a critical compliance risk.

This model is similar to how professional employees are managed.

A manager is not given a predefined script for every possible situation. Instead, authority limits, company policies, performance criteria, and oversight mechanisms are established.

The same principle applies to AI agents:

The boundaries should be clear, while agents should retain enough freedom to act according to the context within those boundaries.

According to Salesforce’s approach, boundaries provide a more scalable governance model for complex business environments than rigid and highly detailed scenarios.

5. Accountability for Every Decision Must Be Clear

When an AI agent accepts a price, approves a contractual term, or escalates a dispute, it must always be possible to determine who is accountable for that decision.

“The AI made the decision” is not a sufficient answer from a corporate, legal, or regulatory perspective.

For every agent, the following questions should be defined in advance:

• Which business unit owns the agent?

• Who defines its authority?

• Who monitors its performance?

• Who approves its level of risk?

• Who is accountable for the consequences of an incorrect decision?

• Which executive or governance body does the agent report to?

As agents become more widespread across organizations, these responsibilities may also create new roles. AI operations managers, agent managers, or AI governance leaders may become responsible for deploying, monitoring, and, when necessary, suspending agents.

But an organizational chart alone is not enough.

Every significant action performed by an agent should create a strong audit trail. This record should capture not only the final decision, but also the information used during the decision, the alternatives considered, and the process that led the agent to its conclusion.

Because when a decision becomes subject to legal review, organizations need to be able to answer not only “What happened?” but also “Why and how did it happen?”

Adding auditability after a system has already gone live is far more difficult than designing the system around that requirement from the beginning.

6. Human Oversight Must Be Introduced at the Right Thresholds

One of the most important capabilities of an AI agent is not only knowing when to act, but also knowing when to stop.

An agent that constantly escalates decisions to humans eliminates much of the value of automation.

An agent that never escalates anything to a human, on the other hand, can quickly become an organizational risk.

For this reason, escalation mechanisms should involve more than a simple “send to a human” rule. Different levels of oversight should be defined based on the level of risk and the potential impact of a decision.

For example:

• Routine information requests may be handled entirely autonomously.

• Standard pricing negotiations may be completed by the agent within predefined boundaries.

• Significant financial commitments may require human review before final approval.

• Decisions involving regulatory risk may be stopped immediately.

• Certain transactions may be subject to periodic post-action audits.

The key decision is therefore not whether humans should be involved in the process.

It is determining at which point, with what information, and with what authority human judgment should enter the process.

Salesforce describes this balance as “calibrated escalation”: allowing low-risk decisions to proceed autonomously while ensuring that high-impact decisions are reviewed by humans before their consequences take effect.