Peer to Peer

Blockchain is a peer-to-peer append-only log that provides a shared and transparent picture of all transactions on a particular network. While a blockchain’s utility has become clear in the financial sector, there are many other applications of blockchain technology across a swath of other industries. One such application is for use with multi-agent systems.

For the scope of this paper, a multi-agent system should be defined as a group of autonomous agents that collaborate with each other to achieve objectives within their environment. This definition contains four categories that must be explored: the agent; objectives, the environment, and the need to collaborate.

The Agent.

Agents are autonomous entities with knowledge about themselves and their objectives and have the ability to make observations about their environment. The knowledge they have about themselves, the objectives that they are trying to achieve, and the real-time observations they make about their environment are what help an agent to make a decision and ultimately take an action based on that decision.

Objectives.

All agents must have objectives they are seeking to achieve. This gives the agent purpose and direction. It also enables the agent to interact in their world in a way that the human operator prefers. Objectives can be tied to rewards if the agent is successful or punishment if the agent fails. In active inference, a Partially Observable Markov Decision Process (POMDP), the agent’s objectives are twofold. First, the agent wants to make decisions that minimize its expected free energy despite uncertainty about the real world, and integral to this is the agent’s desire to balance immediate rewards and future gains. Humans do this every day; the goal is to make autonomous agents do this also.

The Environment.

The real-world environment is rich and dynamic. Humans and animals have senses that enable us to interact with and make decisions about ourselves within our environment. Autonomous Agents require sensors that enable them to do the same. Sensors on agents could include cameras, LIDAR, GPS, and many other things that help a drone to understand itself and its environment. Similarly, in a financial application, a sensor may be data about the markets, news feeds, or user input. The agent’s task and purpose are what define what it needs to know about its environment.

Combining the Categories.

Combined, an agent’s decision-making under an Active Inference framework looks something like this:

Prior belief + present sensing => future decision making (probabilistic belief about the future).

Here’s an example. Suppose it rained last night but you slept through it. You have no rain gauge and you didn’t check the weather. The truth is that it rained, but you don’t know the truth, you must infer it. What you observe, however, is that the grass is wet and there is a puddle on the driveway.

You also have prior knowledge. Wet grass and a puddle could mean rain, or a sprinkler system came on, or a mop bucket was dumped out.

Your prior knowledge about the causes of wet grass + your ability to sense that the grass is in fact wet will lead you to a conclusion about the truth: it rained last night. While this is a simple example, it’s important to see how humans do this every day and that the goal of active inference models in artificial agents is to get agents to perform these tasks also.

The Final Component: A Need to Collaborate.

In multi-agent systems, there is a need for the agents to collaborate with one another. This can be for a variety of reasons. For example, two drones may be tasked to find a lost hiker in a forest. The two drones launch together and begin their search. The ability to collaborate means that the drones should seek to maximize their coverage over the search area while minimizing redundant actions. It would not be beneficial for two drones with the same sensors to search the same spot. But it would also not be beneficial for two drones to search independently. Therefore, collaboration is necessary to maximize resources and minimize waste.

There must also be a mechanism in place to allow for drone failures. Weather, fuel, sensor degradation, mechanical failure, and even nefarious actors can cause a system to fail. When consensus mechanisms rely on leader-follower models or centralize operations and objectives, there is a greater chance that the entire system will fail and not just the drone experiencing the failure. Finally, consensus mechanisms need to allow additional agents to join, be rapidly caught up to speed, and begin contributing to the objectives. For similar reasons that drones may fail, new drones will be launched to replace old ones. A mechanism must be able to account for this.

There are many consensus mechanisms to enable agent collaboration, but this article will focus on blockchain. The strength of Blockchain is that it enables leaderless decision-making among peers and allows for peers to enter and drop from the network without causing faults. Further, blockchain consensus has many elements that support fault tolerance, even failures that occur when an agent fails but continues to report or when the agent is intentionally reporting faulty information.

Using blockchain as a consensus mechanism allows the system to maintain a decentralized log of every decision that was made so that new agents can immediately gain situational understanding. Finally, blockchain maximizes transparency. It allows human decision-makers to observe progress, evaluate decisions, and even intercede in those decisions when required. This transparency and ability to intercede are excellent for defense applications where decision-making should be highly scrutinized.

Conclusion

Multi-agent systems need a mechanism to coordinate their activities. This ensures agents can collaborate to meet their objectives, reduces redundant action, maximizes resources, and ensures transparency of the system. Blockchain for multi-agent systems is a capability that must be further explored.

Written by Thane Keller

Thane is a native of Northern Virginia that has been traveling the world with his wife and four children. Thane researches and writes about technology, innovation, leadership, decision-making, and organizational change.