Internet Explorer is no longer supported. Many things will still work, but your experience will be degraded and some things won't function. Please use a modern browser such as Edge, Chrome, or Firefox.

HOWTO: Multisite Development / Federated Architecture

view on GitHub

A "federated architecture" often refers to software developed at multiple sites with multiple teams. ProGet supports multi-site development by defining how each site interacts with packages through replication and mirroring. To illustrate how ProGet can support federated development, we will use the fictional company "Kramerica".

In this article, we will describe three different scenarios for federated development and provide step-by-step instructions for configuring feed replication and mirroring. Since this can be configured on a per-feed basis, enterprises can use all three scenarios for different packages.

Scenario: Centrally-Approved Packages

In this scenario Kramerica's package approval workflow takes place at its headquarters in New York. London and Tokyo need to have the packages on-site so that they can be used by the development team, but Kramerica only wants them to use the approved packages published by New York.

Central Publishing Diagram

Since the London and Tokyo sites are to initiate communication with New York for their packages, the New York feed is set up as the "incoming" replication feed and the branch sites as the "outgoing" feed.

Location Communication Mode Replication mode
New York Incoming Push Content to Other Instance
Tokyo Pull Content from Other Instance Outgoing
London Pull Content from Other Instance Outgoing

Scenario: Coequal Development

In this scenario, Kramerica teams work on similar projects and develop their own content (packages, container images, feeds) to share with each other.

In this example, developers in London and Tokyo are both working on Kramerica libraries, but want to use their own ProGet instance for bandwidth and security reasons. To achieve this, the feeds are configured to allow both sites to publish packages, which are then automatically replicated to/from the other site.

Coequal Development Diagram

Also, if we want Tokyo and London to mirror each other, one must be configured as the "incoming" replication feed and the other as the "outgoing". In this scenario, either feed can be configured as an incoming feed, but we will use Tokyo.

Location Communication mode Replication mode
Tokyo Incoming Mirror Content
London Outgoing Mirror Content

Scenario: Global Package Distribution

In this scenario, one of Kramerica's sites (Los Angeles) develops packages that all other sites must access. Instead of connecting all sites to Los Angeles, the New York branch can distribute packages from Los Angeles to other sites in Tokyo, London, and Berlin. In this way, all sites remain aligned to one central source.

Global Package Distribution Diagram

This scenario is unique in that the New York location must be set up as both an incoming and outgoing feed. It goes to Los Angeles to replicate packets from there, and goes to the branch offices so they can replicate from New York. This means that you need to configure two separate replications in the New York feed. One replication is set up as incoming ( pushing to the branch offices) and the other is set up as outgoing ( pulling from the Los Angeles feed).

Location Communication mode Replication mode
Los Angeles Incoming Push Content to Other Instance
New York Incoming & Outgoing Push & Pull
Tokyo Outgoing Pull Content from Other Instance
London Outgoing Pull Content from Other Instance
Berlin Outgoing Pull Content from Other Instance

How to Configure Feed Replication and Mirroring

In all three scenarios, incoming and outgoing replication feeds must be configured to work. For each scenario, repeat the following steps, changing the appropriate configuration options in ProGet.

Replication and mirroring take place at the feed level. Different feeds can use different scenarios, or an entire site can work a certain way by configuring every feed at that site the same way. For example, in the centralized publishing scenario, each feed in New York must be set up for incoming replication, and each feed in London and Tokyo must be set up for outgoing replication.

Step 1: Setup Incoming Replication Feed

First, your incoming replication feed must be set up in ProGet. In most cases, the feed you use to publish packages will be your replication feed.

Navigate to Replication> Configure New Replication.

Configuration Replication

Choose the feeds you wish to setup for incoming replication.

Feed Communication Mode

Communication type:
Select Incoming to allow other ProGet instances to connect to this instance.

Authentication mode:
Requires a specific sync token or allows any valid API key

  • Sync token
    Click the "Generate" button to generate a sync token and copy it, as you will need to paste it into the outgoing feeds you want to connect.
  • API key
    Navigate to Settings > Integration & Extensibility > API Key > and create an API key for the feed layer. Copy it as you will need to paste it into the outgoing feeds you want to connect.

Replication Mode Options

Replication Mode Options:
Mirror content: Push and pull packets so that the feed is synchronized. The other instance should be configured to mirror content.
Push content from other instance: Upload missing content to the other instance and delete content that has been deleted on this instance.
Retrieve content from other instance: Download missing content from the other instance and delete content that has been deleted on this instance.
Custom / Advanced: Mix and match push and pull content properties.

Select the desired replication mode and proceed to the summary.

Incoming Feed Summary

Step 2: Setup Outgoing Replication Feed

Once you have set up an incoming feed, you need to configure the outgoing feed that will be associated with your incoming feed. The outgoing feeds are usually the sites that do not publish their own packages but replicate from an approved package promoting feed.

Navigate to Replication > Configure New Replication. Under Feeds, select the feeds you want to set up for outgoing replication feed and proceed to the Communication Mode tab.

Outgoing Feed Communication Mode

Communication type:
Select Outgoing to allow ProGet to routinely connect to another instance.

Other Instance URL:
Copy the base URL of the instance from the incoming feed you want to connect to.

Authorization
Paste here the sync token or API key you generated in step 1.

Other Feed Names: Leave the box checked if your feed names are the same on both instances. Otherwise, uncheck the box and enter the name of the incoming feed you are connecting to next to Name for "Feed Name".

Replication Mode Options

Select the desired replication mode and proceed to the summary (see Step 1 for a detailed summary of options).

Outgoing Feed Summary

Step 3: Replicate Packages

After your incoming and outgoing feeds are connected, packets are automatically replicated at regular intervals. However, you can also manually replicate packages immediately.

Navigate to Replication > Replication Overview in your outgoing feed and select Run.

Run Replication

After execution, the packages from the incoming feed are replicated to the outgoing feeds based on the configuration you set up.

Replication can also be leveraged to set up your Disaster Recovery plan or Edge Computing.

ProGet v6 And Earlier

Setting up replication in ProGet v6 and earlier is the same in theory, but with slightly different steps due to a change in UI.

To configure feed replication, navigate to Manage Feed > Connectors & Replication > Feed Replication > Configure.

Incoming Replication Configuration:

  • Outgoing Option:
    Leave this option disabled as you configure an incoming feed.
  • Incoming Options:
    Allow external feeds to replicate from this feed (read only)
    Allow local feeds to be modified by external feeds (read/write)

Outgoing Replication Configuration:

  • Outgoing Options:
    Apply only external changes to the local feed (pull-only)
    Apply local changes to external feed only (push-only).
    Mirror external feed

  • Incoming Options:
    Leave this set to Disabled, as you are configuring an outgoing feed.