HOWTO: Multisite Development / Federated Architecture
  • 27 Jun 2022
  • 6 Minutes to read
  • Dark
    Light
  • PDF

HOWTO: Multisite Development / Federated Architecture

  • Dark
    Light
  • PDF

How to Use ProGet to Support a Multisite Development in a Federated Architecture

A "federated architecture" often refers to software that's developed in multiple locations with multiple teams. ProGet supports multisite development by defining how each location interacts with packages through replication and mirroring. To help illustrate how ProGet can support federated development, we will use the fictitious company "Kramerica".

In this article, we will describe three different federated development scenarios and provide step-by-step instructions on configuring feed replication and mirroring. And because this can be configured on a feed-by-feed basis, organizations can use all three scenarios for different packages.

Scenario: Centrally-Approved Packages

In this scenario Kramerica's package approval workflow takes place at its headquarters in New York. London and Tokyo need to have the packages on-site so that they can be used by the development team, but Kramerica only wants them to use the approved packages published by New York.

Central Publishing Diagram

Since we want the London and Tokyo locations to initiate communication with New York for their packages, the New York feed will be setup as the "incoming" replication feed and the branch locations as the "outgoing".

Location Communication Mode Replication mode
New York Incoming Push Content to Other Instance
Tokyo Pull Content from Other Instance Outgoing
London Pull Content from Other Instance Outgoing

Scenario: Coequal Development

In this scenario, Kramerica's teams are working on similar projects and developing their own content (packages, container images, feeds) they want to share with each other.

In this example, developers in London and Tokyo are both working on Kramerica libraries but wish to use their own ProGet instance for bandwidth and security reasons. To accomplish this, the feeds are configured so that both locations can publish packages, and it will automatically replicate to/from the other location.

Coequal Development Diagram

Even though we want Tokyo and London to mirror each other, one will need to be configured as the "incoming" replication feed, and the other as "outgoing". Either feed can be setup as the incoming feed in this scenario, but we will use Tokyo.

Location Communication mode Replication mode
Tokyo Incoming Mirror Content
London Outgoing Mirror Content

Scenario: Global Package Distribution

In this scenario, one of Kramerica's branch offices (Los Angeles) develops packages that all other locations need access to. Instead of having all locations connect to Los Angeles, the New York location can distribute packages from Los Angeles to other sites in Tokyo, London, and Berlin. This keeps all locations pointed at a central source.

Global Package Distribution Diagram

This scenario is unique in that the New York Location will need to be setup as both an incoming and outgoing feed. It will be outgoing to Los Angeles so that it can replicate packages from it and incoming to the branch locations so they can replicate from New York. This means that you will need to configure two separate replications in the New York feed. One replication will be setup as incoming (pushing to branch locations) and another as outgoing (pulling from Los Angeles feed).

Location Communication mode Replication mode
Los Angeles Incoming Push Content to Other Instance
New York Incoming & Outgoing Push & Pull
Tokyo Outgoing Pull Content from Other Instance
London Outgoing Pull Content from Other Instance
Berlin Outgoing Pull Content from Other Instance
Sync Tokens and Source URLs

Remember to copy the sync token and instance base URL from the incoming feed that you wish to connect and to paste them into the relevant outgoing feeds.

In the example above New York will use the sync token and instance base URL generated by the Los Angeles feed as well as generate its own sync token since it is also an incoming feed. The Tokyo, London, and Berlin feeds will use the sync token and instance base URL generated by the New York feed since they will be replicating from there instead of Los Angeles.

How to Configure Feed Replication and Mirroring

These steps/screenshots in this guide were based on ProGet v2022's replication UI, which are available as a preview feature in ProGet v6.0.11+. The concepts are mostly the same in earlier versions; see ProGet v6 and earlier to learn about the differences.

All three scenarios must configure incoming and outgoing replication feeds to function. For each scenario repeat the following steps while changing the relevant configuration options in ProGet.

Replication and mirroring take place on the feed level. Different feeds can use different scenarios, or an entire location can function in a certain way by configuring each feed in that location the same way. For example, in the central publishing scenario, each feed in New York will need to be setup for incoming replication and each feed in London & Tokyo setup for outgoing replication.

Step 1: Setup Incoming Replication Feed

First, your incoming replication feed will need to be setup in ProGet. In most cases the feed that you will be publishing packages from will be your replication feed.

Navigate to Replication> Configure New Replication.

Configuration Replication

Choose the feeds you wish to setup for incoming replication.

Feed Communication Mode

Communication type:
Select Incoming to allow other ProGet instances to connect to this instance.

Authentication mode:
Require a specific sync token or allow any permitted API key

  • Sync Token
    Click the generate button to generate a sync token and make sure to copy it as you will need to paste it in the outgoing feeds you wish to connect.
  • API key
    Navigate to Settings>Integrations & Extensibility> API Keys> and create a feed level API key. Make sure to copy it as you will need to paste it in the outgoing feeds you wish to connect.

Replication Mode Options

Replication Mode Options:
Mirror content: Push and pull packages so that the feed is in sync. The other instance should be configured to Mirror Content.
Push Content from Other Instance: Upload missing content to the other instance and delete content on that instance that was deleted on this instance.
Pull Content from Other Instance: Download missing content from the other instance and delete content that was deleted from that instance.
Custom / Advanced: Mix and match the qualities of pushing and pulling content.

Choose your desired replication mode and continue to Summary.

Incoming Feed Summary

Step 2: Setup Outgoing Replication Feed

Once you have an incoming feed setup, you will need to configure the outgoing feed that will connect to your incoming feed. The outgoing feeds are normally those locations that do not publish their own packages but rather replicate from an approved package promoting feed.

Navigate to Replication> Configure New Replication. Under Feeds, select the feeds you wish to setup for outgoing replication feed and continue to the Communication mode tab.

Outgoing Feed Communication Mode

Communication type:
Select Outgoing to allow ProGet to routinely connect to another instance.

Other instance base URL:
Copy and paste the instance base URL from the incoming feed that you wish to connect to.

Authorization
Paste the Sync token or API key you generated in Step 1 here.

Other feed names: Keep the box checked if your feed names are the same on both instances. Otherwise uncheck and enter the name of the incoming feed you are connecting to next to Name for "feedname".

Replication Mode Options

Choose your desired replication mode and continue to Summary (see step 1 for a detailed summary of the options).

Outgoing Feed Summary

Step 3: Replicate Packages

After your incoming and outgoing feeds are connected, packages will automatically be periodically replicated. However, you can also manually replicate packages instantly.

Navigate to Replication> Replication Overview in your outgoing feed and select run.

Run Replication

After running, the packages from your incoming feed will be replicated to the outgoing feeds based upon the configuration you setup.

Replication can also be leveraged to set up your Disaster Recovery plan or Edge Computing.

ProGet v6 And Earlier

Setting up replication in ProGet v6 and earlier is the same in theory, but with slightly different steps due to a change in UI.

To configure feed replication, navigate to Manage Feed>Connectors & Replication>Feed Replication> Configure.

Incoming Replication Configuration:

  • Outgoing Option:
    Leave this as disabled since you are configuring an incoming feed.
  • Incoming Options:
    Allow external feeds to replicate from this feed (read-only)
    Allow local feeds to be changed by external feeds (read/write)

Outgoing Replication Configuration:

  • Outgoing Options:
    Only apply external changes to local feed(pull-only)
    Only apply local changes to external feed (push-only).
    Mirror external feed

  • Incoming Options:
    Leave this as Disabled since you are configuring an outgoing feed.


Was this article helpful?