Importing an Artifact from a Drop Folder

BuildMaster can be configured to capture build output or build artifacts from a drop folder and store them until they are ready for deployment. Using drop folders is a common practice when build and release teams are separated organizationally.

It's also a great way to get started with incremental automation.

Modeling the Import Process: Manual Import

In many cases, build engineers or CI systems will notify release teams once an build is copied into a drop folder. The BuildMaster model for this process is simple and requires minimal setup:

• a known location to act as the drop folder - a path on disk or network accessible to the BuildMaster service or agent
• basic implementation in BuildMaster (outlined below)

In the following example below, BuildMaster assumes that the build output will be located in a subfolder of the root drop folder, and the name of the subfolder matches the release number within BuildMaster, for example:

\\filesv.corp.local\HdarsDropFolder\Releases\2.4.1

You should capture a build artifact from this folder, since it will ensure you deploy the same set of files each time you deploy this release.

To capture the build output from this folder, you can use a Create-Artifact operation:

Create-Artifact HdarsVendorFiles
(
    From: \\filesv.corp.local\HdarsDropFolder\Releases\$ReleaseNumber
);

Once captured, you can deploy to servers or the cloud in subsequent pipeline stages:

Deploy-Artifact HdarsVendorFiles
(
    To: D:\Websites\HdarsSite
);

BuildMaster Application Template

The "Drop Folder Import" application template will automatically create the following relevant items:

• Import plan - a script that captures the build output from the drop folder
• Release pipeline - a pipeline that uses the import plan in the first stage
• $DropFolder configuration variable - referenced in the import plan as the root folder for the build output

Once the application is generated in BuildMaster, creating a new build will capture all files in $DropFolder\$ReleaseNumber into a build artifact that can be deployed to future pipeline stages.

Modeling the Import Process: Drop Folder Monitoring

Importing from a drop folder can be automated a step further by configuring a build trigger. A build trigger runs periodically and, based on the existence of a special marker file, imports the build output as a build artifact into BuildMaster. Once imported, the files are deleted from the drop folder.

When compared to the manual drop folder import, this pattern tends to be more useful for automated processes that copy files into the drop folder such as an existing CI system, or a vendor providing frequent builds.

BuildMaster Application Template

The "Drop Folder Monitor" application template will automatically create the following relevant items:

• Monitor plan - a script that runs periodically, and if ready.txt exists in the $DropFolder, the import plan is run
• Import plan - a script that captures the build output from the drop folder
• Release pipeline - a pipeline that uses the import plan in the first stage
• $DropFolder  configuration variable - referenced in the import plan as the root folder for the build output

Once the application is generated, BuildMaster will continuously monitor $DropFolder for a file named ready.txt, and when found, will create a new build to capture all other files in $DropFolder into a build artifact that can be deployed to future pipeline stages. After the files are captured, they are deleted from the drop folder to prepare for the next time files are dropped into it.

The Marker File: ready.txt

Because BuildMaster periodically scans a drop folder with arbitrary files in it, there is no way to determine whether that folder is populated with all the files intended to be part of the artifact or not. Using the presence of a marker file is a simple way to signal to BuildMaster that the folder's contents are accurate and complete. Of course, this file should be added only as the final build task, or by human intervention.

Customizing the Process

The patterns and templates above are designed to get you started with drop folder automation and may be customized to suit the needs of your organization. Some customization ideas include:

• using the @FileMask function to iterate directories and extract a release number
• omit searching for ready.txt and just assume the drop path files are correct and will not experience a race condition (e.g. all files are copied to \drop-folder-tmp and the directory is renamed to \drop-folder when all files are present)
• using $FileContents to read the contents of a text file in order to make automation decisions
• using the Create-Build and Create-Release operations in the monitor plan to create builds or releases for other applications in BuildMaster