CI/CD for NuGet Packages

NuGet is the packaging platform used with .NET and provides developers easy access to NuGet.org, a public package gallery with thousands of free and open source packages. By discovering and incorporating these code packages into their own applications, developers can save time by not solving the same problem over and over. In addition, NuGet packages are more manageable and less error-prone than copy/pasting code found on the Internet. They have dependency management built-in; developers can easily find out when new version of packages are published (bug fixes, security bugs) and rebuild their application using the new version of the packages.

Although NuGet was originally designed for free and open-source packages hosted at NuGet.org, with a private package repository like ProGet, you can host your own packages without making them public. This gives you the same benefits of code reuse and code sharing within your organization. By using BuildMaster and ProGet's advanced features, you can apply Continuous Integration & Continuous Delivery (CI/CD) principles to rapidly test and deliver these packages alongside your application.

NuGet Package CI/CD Pipeline

CI/CD pipelines for NuGet packages are similar to CI/CD pipelines for applications: a Build Trigger monitors source control and automatically creates a build that enters a deployment pipeline. However, instead of creating build artifacts at the first stage and deploying those artifacts to each environment, you will create and publish a pre-release package, publish it to a package source, then repackage at each stage.

• Build: create a pre-release Continuous Integration (CI) version of a package (e.g. ProfitCalc v3.2.4-ci.11) and publish to your package source (i.e., a ProGet feed)
• Test: instruct ProGet to repackage that version into a more stable, Release Candidate (RC) version (e.g. ProfitCalc-3.2.4-rc.11)
• Release: instruct ProGet to repackage the RC version to a stable version (e.g. ProfitCalc-3.2.4)

If you have a lot of CI packages due to many commits, you should consider publishing those packages to a different feed (a CI feed, for example), and instruct ProGet to promote the repackaged packages.

NuGet Package CI/CD Development Workflow

Teams creating and using their own NuGet packages for the first time are often frustrated by the extra steps required to separate code across multiple projects. However, this package CI/CD workflow can greatly simplify package development and usage for developers.

1. Develop and test package changes locally (i.e., on the workstation)
2. Commit a change to a package's repository, then wait for the CI package to be built
3. Update the application to use the CI package (locally), then make the appropriate changes and test (locally)
4. Promote package from CI build to test phase in BuildMaster
5. Upgrade application to use the RC package, then commit changes
6. Proceed as with any other application change

The RC package can be promoted to a stable package at any time, and other developers can download and use RC packages as needed. And since all this is managed as a pipeline in BuildMaster, everyone can see what stages each package is in.

Still, you should never release an application that uses unstable (pre-release) packages. This means:

• release packages in production before your application is promoted
• be able to use an automatic system to ensure that applications don't use pre-release packages

Creating a Package Version Number

NuGet packages use the semantic version format, which is a three-part number (MAJOR.MINOR.PATCH) that determines how backward- and forward-compatible changes will be. Before building a NuGet package, you will need this version number to create the package.

This is where BuildMaster comes in: it's the source of truth for build and release numbers, which means you simply use it to generate your package version number. There is no need to pass this version number to source control, and if you do, it becomes even harder to keep these numbers in sync.

The easiest way to generate a package version number is to compare the release and build numbers in an OtterScript expression:

set $packageVersion = $ReleaseNumber-CI.$BuildNumber;

You can then use this variable in subsequent operations in your OtterScript.

When developing a package for the first time, you should use - worth adding version 0.y.z should be used in development

Building NuGet Packages

A NuGet package is a .zip file that contains a package manifest file and compiled code (DLLs). Microsoft supports three different methods to create NuGet packages, all of which you can do with operations in BuildMaster:

• dotnet CLI uses the DotNet::build operation
• nuget.exe CLI uses the NuGet::Create-Package operation
• MSBuild uses the MSBuild::Build-Project operation

All of these methods and operations will ultimately create the same package, but before doing that make sure to create and set a version number.

For the NuGet CI/CD template application, we use the DotNet::Set-ProjectVersion to edit the project file, and the DotNet::Build operation to build the code and create the package.

DotNet::Set-ProjectVersion
(
    FromDirectory: ~\Src\$PackageName,
    Version: $ReleaseNumber,
    AssemblyVersion: $ReleaseNumber.0,
    FileVersion: $ReleaseNumber.$BuildNumber,
    PackageVersion: $ReleaseNumber-CI.$BuildNumber
);

DotNet::Build ~\Src\$PackageName
(
    Configuration: Release
);

Other methods utilize a nuspec file to be created (nuget.exe spec), which you similarly edit using the `Replace-Text operation.

Publishing and Attaching

Instead of capturing the build output as an artifact, you can use the NuGet::PublishPackage operation to publish it to a package source that you've configured:

NuGet::Publish-Package ~\Src\$PackageName\bin\Release\$PackageName.$packageVersion.nupkg
(
    Source: CIPackages,
    AttachToBuild: true
);

A package source is essentially a ProGet feed URL with a resource credential (user name/password).

The second parameter (AttachToBuild) indicates that a package reference will be associated with the build. Think of this as a reference to the source that was published; you can easily navigate to it from the UI, use it later in repackaging operations.

Publishing to NuGet.org

To publish packages to NuGet.org, you first need to create an account. Once you have a NuGet account, you can publish packages to NuGet.org from BuildMaster by simply creating resource credentials pointing to NuGet.org.

Repackaging NuGet Packages

Repackaging is used to create a new package from an existing package using exactly the same verified content while maintaining the integrity of the original package and providing an audit trail to show when and why the repackaging occurred.

You can use the ProGet::Repack-Package operation to instruct your ProGet server to perform to repackage:

ProGet::Repack-Package
(
    Name: ProfitCalc,
    NewVersion: $PackageVersion(ProfitCalc, stable)-RC.$BuildNumber
);

In this case, the ProfitCalc package is already attached to the build from an earlier Publish-Package, which means BuildMaster can simply use the referenced package application to publish.

HOWTO: Step-by-step for BuildMaster and ProGet

This requires a ProGet instance, a BuildMaster instance, and .NET SDK to be installed .

Step 1: Navigate to a NuGet Feed in ProGet

Whether you create a new feed or use an existing one, navigate to the desired NuGet feed in ProGet.

On the feed page, copy the API URL from the top right corner.

This API will be used in Step 6.

Step 2: Navigate to API Settings

Navigate to ProGet’s settings via the gear icon and select API Keys under the Integrations & Extensibility section.

Step 3: Create an API Key

In the API Settings, click Create API Key and a pop-up will appear.

In the pop-up window, choose the Key Type (for this tutorial, choose Feed), select the feed you want to connect (we named a NuGet feed 'tutorial' for this guide), and select the package permissions.

Package permissions will vary depending on your organization's preferences.

Leave the other settings at their default and click Save API Key.

Step 4: Build a New Application in BuildMaster

In a BuildMaster instance, create a new NuGet application.

The page will automatically go to the application overview.

Step 5: Create a Secure Resource

After the template is applied, you’ll need to create a secure resource to clear the first warning shield.

From the overview page, hover over Settings and click Resources.

On the Resource page, click Create Secure Resource on the bottom right corner.

In the pop-up, select NuGet Package Feed from the list of available options.

Name the resource as appropriate.

On the next screen, select the create... buttom at the bottom of the pop-up.

On the Select Secure Credentials Type page of the pop-up, select API Key/Token.

In the Select Secure Credentials Type select API Key/Token.

Step 6: Create an API Key within Secure Credentials

In the Create Secure Credential API window, name the API and leave the remaining items on default.

Save the credential, and the page will return to the Create Secure Resource page.

Paste the NuGet feed API Key from Step 1.

From the Credentials drop down, select the Credential just created.

Click Save Resource.

Step 7: Edit the Package Name in BuildMaster

Navigate back to the top of the Application by click Overview.

On the overview page, click Package Name from the list of settings. This will help clear the warning shields.

In the Edit Variable pop-up, input the name of the secure resource just created and add a desired package's name.

Step 8: Confirm the CI/CD Connection between ProGet and BuildMaster

Your NuGet CI/CD pipeline is now ready to use, so create a new build in BuildMaster. You’ll be prompted to create a release and a new build.

If you’ve configured everything correctly, when you go back to the ProGet feed, you’ll see a new package with -ci ending in the feed.

ProGet clearly indicates that it’s a pre-release version, and you can view the metadata, files, and more by clicking the package's name.

As you deploy your package through this NuGet CI/CD pipeline, BuildMaster will continue to push pre-release and finally the stable version of your package to ProGet.

Prevent the Release of Applications with Unstable Package References

Unstable (pre-release) packages are important to use when testing a package's code prior to its release, but by definition they're unsuitable for release. As such, you should not deploy applications that reference the pre-release packages you create.

BuildMaster can help with this by performing an automated check against a variable that you set your application's build process using the DotNet::Get-Dependencies operation.

Inedo's Use Case

To prevent us from accidentally releasing a product (such as Otter) with a pre-release version of one of our libraries (like InedoLib), we use the following OtterScript in our build plan to set the $UnstableDependencies variable, then we test that variable prior to deploying to the Release stage.

set %prereleasePackages = %();

foreach $projPath in @FileMask(**.csproj)
{
    DotNet::Get-Dependencies
    (
        ProjectPath: $projPath,
        Dependencies => %nugets
    );

    foreach $k in @MapKeys(%nugets)
    {
        set $v = %nugets[$k];

        if $MatchesRegex($v, -)
        {
            set %prereleasePackages[$k] = $v;
        }
    }
}

set $buildVar = none;

set @preList = @();

foreach $k in @MapKeys(%prereleasePackages)
{
    set @preList = $ListInsert(@preList, $k $(%prereleasePackages[$k]));
}

if $ListCount(@preList) != 0
{
    set $buildVar = $Join(', ', @preList);
}

Set-BuildVariable UnstableDependencies
(
    Value: $buildVar
);