BigFix Platform PeerNest feature

Introduction

Starting from 9.5 Patch 11, the BigFix client includes a new feature named PeerNest, that allows to share binary files among clients located in the same subnet.

A practical use case is a branch office connected to the data center through a slow link: with earlier BigFix versions, the suggested configuration required a relay to be installed in the branch office in order to download and cache large payloads; with PeerNest, the BigFix clients can share downloaded binaries and therefore reduce the number of communications going outside of the branch office even if a relay is not installed locally.

PeerNest documentation is available here: https://help.hcltechsw.com/bigfix/10.0/platform/Platform/Config/c_P2P.html.

This article will show how the communication and synchronization among the peers work.

PeerNest networking

When a client with peer enabled starts, it will join the multicast group 224.0.0.75 (IPv4) or ff02::75 (IPv6) and (if not set as passive) will start listening on the TCP client port (default 52311).

The multicast group is used for the synchronization among the peers; all their communications flow through the UDP protocol. The listening TCP is an HTTPs server used for file sharing.

PeerNest priority

Each peer has a priority assigned (default 100), used both in the Master Election process and in choosing the peer to download from.

Master Election

When multiple Clients are executing actions requiring the prefetch of a binary file, they check with their peers if the file is already cached in the subnet. If the binary was not cached, then the Clients can elect one of them as responsible for downloading from the Relay: the peer with the highest priority, among the peers requiring the file, will manage the download; if all peers have the same priority, the computer with the lowest ID will download the file from the Relay.

Starting from Version 10 Patch 2, a new feature allows time optimization for actions that require prefetching of large files. This feature ensures that the peers elected for downloads are those with real ability to store files rather than simply those with highest priority.

Download from peers

The Client priority comes also into play when there are 2 or more peers available to share the same file. The Client that wants to download the file creates a memory list of peers serving the file; it will pick up the peer randomly with weighted probability, based on the priority: for instance, if the memory list is made up of two peers, say C1 with priority W, and C2 with priority 2W, picking C2 will be twice as likely as picking C1.

How PeerNest works

When multiple Clients are executing actions requiring the prefetch of a binary file, they check with their peers if the file is already cached in the subnet.

The peer requesting the file sends an UDP message to the multicast group with message type 1 (requesting a file) and the hash of the file.

If any of the peers is caching the file, it will replay with a unicast UDP message to the sender in order to confirm the availability; the message is of type 4 (file available for the download) and contains the ComputerID of the sender and its priority. Peers in passive mode never reply to type 1 requests.

If the file requested is not available, the Master Election process takes place, and the elected master starts downloading the file from the Relay. The master peer periodically (every 2 minutes) notifies all the other peers that a download is in progress; the message is an UDP multicast with type 3 (download in progress) and hash of the file. Peers in passive mode do not participate, they just wait for a file availability message and then download the file from the sender.

Here some use case scenarios with some examples from the PeerNest log.

Scenario 1: one Peer requests a file

Peer 1 (computerID 1083549643) starts an action and requires a file (having hash cb19f836c2830ff88ff45694565da65be73b7a69):

All other peers receive the request:

No answer comes from the other peers within the allowed timeframe (_BESClient_PeerNest_ResponseTimeoutSeconds setting) and the peer requesting the file starts downloading it immediately from the Relay:

Peer 2 (computerID 16321087) requests the same file, and the Peer 1 notifies it that a download is already in progress:

When the peer finishes the download, it moves the file into the PeerNest cache and notifies the other peers about its availability with an UDP message of type 4 containing the file hash and the URL; the other peers, in turn, will download the file (if they need it) and share it

Scenario 2: more Peers requesting a file at the same time

Peer 1 (computerID 16333978, priority 300), Peer 2 (computerID 1086694000, priority 100) and Peer 3 (computerID 1614746242, priority 100) require a file (having hash cb19f836c2830ff88ff45694565da65be73b7a69) at the same time:

The Master Election process takes place and the Peer with the greater priority or the lower id (if all the Peers have the same priority) starts the download:

If no periodic updates are received, the master is considered down, and a new election process is started over:

When the elected master finishes the download, it moves the file into the PeerNest cache and notifies the other peers about its availability with an UDP message of type 4 containing the file hash:

Authors of this Blog: 

Daniele Coli

Annalisa Pizza

Debora Perrotti