How to Use ProGet to Support a Multisite Development in a Federated Architecture

A "federated architecture" often refers to software that's developed in multiple locations with multiple teams. ProGet supports multisite development by defining how each location interacts with packages through replication and mirroring. To help illustrate how ProGet can support federated development, we will use the fictitious company "Kramerica".

In this article, we will describe three different federated development scenarios and provide step-by-step instructions on configuring feed replication and mirroring. And because this can be configured on a feed-by-feed basis, organizations can use all three scenarios for different packages.

Scenario: Centrally-Approved Packages

In this scenario Kramerica's package approval workflow takes place at its headquarters in New York. London and Tokyo need to have the packages on-site so that they can be used by the development team, but Kramerica only wants them to use the approved packages published by New York.

Since we want the London and Tokyo locations to initiate communication with New York for their packages, the New York feed will be setup as the "incoming" replication feed and the branch locations as the "outgoing".

Scenario: Coequal Development

In this scenario, Kramerica's teams are working on similar projects and developing their own content (packages, container images, feeds) they want to share with each other.

In this example, developers in London and Tokyo are both working on Kramerica libraries but wish to use their own ProGet instance for bandwidth and security reasons. To accomplish this, the feeds are configured so that both locations can publish packages, and it will automatically replicate to/from the other location.

Even though we want Tokyo and London to mirror each other, one will need to be configured as the "incoming" replication feed, and the other as "outgoing". Either feed can be setup as the incoming feed in this scenario, but we will use Tokyo.

Scenario: Global Package Distribution

In this scenario, one of Kramerica's branch offices (Los Angeles) develops packages that all other locations need access to. Instead of having all locations connect to Los Angeles, the New York location can distribute packages from Los Angeles to other sites in Tokyo, London, and Berlin. This keeps all locations pointed at a central source.

This scenario is unique in that the New York Location will need to be setup as both an incoming and outgoing feed. It will be outgoing to Los Angeles so that it can replicate packages from it and incoming to the branch locations so they can replicate from New York. This means that you will need to configure two separate replications in the New York feed. One replication will be setup as incoming (pushing to branch locations) and another as outgoing (pulling from Los Angeles feed).

How to Configure Feed Replication and Mirroring

All three scenarios must configure incoming and outgoing replication feeds to function. For each scenario repeat the following steps while changing the relevant configuration options in ProGet.

Replication and mirroring take place on the feed level. Different feeds can use different scenarios, or an entire location can function in a certain way by configuring each feed in that location the same way. For example, in the central publishing scenario, each feed in New York will need to be setup for incoming replication and each feed in London & Tokyo setup for outgoing replication.

Step 1: Setup Incoming Replication Feed

First, your incoming replication feed will need to be setup in ProGet. In most cases the feed that you will be publishing packages from will be your replication feed.

Navigate to Replication> Configure New Replication.

Choose the feeds you wish to setup for incoming replication.

Communication type:
Select Incoming to allow other ProGet instances to connect to this instance.

Authentication mode:
Require a specific sync token or allow any permitted API key

• Sync Token
  Click the generate button to generate a sync token and make sure to copy it as you will need to paste it in the outgoing feeds you wish to connect.
• API key
  Navigate to Settings>Integrations & Extensibility> API Keys> and create a feed level API key. Make sure to copy it as you will need to paste it in the outgoing feeds you wish to connect.

Replication Mode Options:
Mirror content: Push and pull packages so that the feed is in sync. The other instance should be configured to Mirror Content.
Push Content from Other Instance: Upload missing content to the other instance and delete content on that instance that was deleted on this instance.
Pull Content from Other Instance: Download missing content from the other instance and delete content that was deleted from that instance.
Custom / Advanced: Mix and match the qualities of pushing and pulling content.

Choose your desired replication mode and continue to Summary.

Step 2: Setup Outgoing Replication Feed

Once you have an incoming feed setup, you will need to configure the outgoing feed that will connect to your incoming feed. The outgoing feeds are normally those locations that do not publish their own packages but rather replicate from an approved package promoting feed.

Navigate to Replication> Configure New Replication. Under Feeds, select the feeds you wish to setup for outgoing replication feed and continue to the Communication mode tab.

Communication type:
Select Outgoing to allow ProGet to routinely connect to another instance.

Other instance base URL:
Copy and paste the instance base URL from the incoming feed that you wish to connect to.

Authorization
Paste the Sync token or API key you generated in Step 1 here.

Other feed names: Keep the box checked if your feed names are the same on both instances. Otherwise uncheck and enter the name of the incoming feed you are connecting to next to Name for "feedname".

Choose your desired replication mode and continue to Summary (see step 1 for a detailed summary of the options).

Step 3: Replicate Packages

After your incoming and outgoing feeds are connected, packages will automatically be periodically replicated. However, you can also manually replicate packages instantly.

Navigate to Replication> Replication Overview in your outgoing feed and select run.

After running, the packages from your incoming feed will be replicated to the outgoing feeds based upon the configuration you setup.

Replication can also be leveraged to set up your Disaster Recovery plan or Edge Computing.

ProGet v6 And Earlier

Setting up replication in ProGet v6 and earlier is the same in theory, but with slightly different steps due to a change in UI.

To configure feed replication, navigate to Manage Feed>Connectors & Replication>Feed Replication> Configure.

Incoming Replication Configuration:

• Outgoing Option:
  Leave this as disabled since you are configuring an incoming feed.
• Incoming Options:
  Allow external feeds to replicate from this feed (read-only)
  Allow local feeds to be changed by external feeds (read/write)

Outgoing Replication Configuration:

• Outgoing Options:
  Only apply external changes to local feed(pull-only)
  Only apply local changes to external feed (push-only).
  Mirror external feed

• Incoming Options:
  Leave this as Disabled since you are configuring an outgoing feed.