Asset Directory API

The Asset Directory API provides simple RESTful access to all asset storage capabilities. Like feeds, asset directories expose their APIs through the URL /endpoints/«AssetDirectoryName»/...

Asset Directories are represented internally in ProGet as Feeds, so Feed API keys can also be used to authenticate for Asset Directories.

Content API

The Asset Content API is accessible under /endpoints/«AssetDirectoryName»/content/..., and provides the primary access point for working with assets. This endpoint is designed to function like a standard web server, so hosted assets can be accessed by simple GET requests with support for browser caching.

Get Asset Endpoint

GET .../content/«path_to_asset»

Gets the asset at the specified path, returning it as content. This endpoint returns a status of 200 (success), 304 (success, not modified), 404 (asset not found), 401 (auth required), 403 (access denied).

On success, the asset is returned as content.

Test for Asset Endpoint

HEAD .../content/«path_to_asset»

Returns the headers only for the GET endpoint. This can be used to determine if an asset exists. This endpoint returns a status of 200 (success), 304 (success, not modified), 404 (asset not found), 401 (auth required), 403 (access denied).

Note that as this is a HEAD request, there is no content in the response.

Create or Replace Asset Endpoint

POST .../content/«path_to_asset»

Creates a new asset (or overwrites an existing asset) at the specified path using the request content as the asset. This endpoint returns a status of 201 (success), 401 (auth required), 403 (access denied). If specified, the Content-Type header will be stored for the asset.

On success, the asset is saved to the asset directory.

Create Asset Endpoint

PUT .../content/«path_to_asset»

Creates a new asset (but will not overwrite an existing asset) at the specified path using the request content as the asset. This endpoint returns a status of 201 (success), 400 (asset already exists), 401 (auth required), 403 (access denied). If specified, the Content-Type header will be stored for the asset.

On success, the asset is saved to the asset directory.

Replace Asset Endpoint

PATCH .../content/«path_to_asset»

Overwrites an existing asset at the specified path using the request content as the asset. This endpoint returns a status of 201 (success), 404 (asset not found), 401 (auth required), 403 (access denied). If specified, the Content-Type header will be stored for the asset.

On success, the existing asset is replaced.

Delete Asset Endpoint

DELETE .../content/«path_to_asset»

Deletes the asset at the specified path. The path must refer to an invidual asset and not a directory. It is not considered an error to delete a file that does not exist. This endpoint returns a status of 200 (success), 400 (path refers to a directory), 401 (auth required), 403 (access denied).

On success, the asset is deleted from the asset directory.

Directory API

The Asset Directory API is accessible under /endpoints/«AssetDirectoryName»/dir/..., and extends the Content API by providing explicit endpoints for working with directories.

List Directory Endpoint

GET .../dir/«path»?recursive=«true/false»

Returns all assets and directories in the specified path as an array of JSON items. When recursive is false or not specified, only items contained in the specified path are returned. When recursive is true, all items in subdirectories are also returned. This endpoint returns a status of 200 (success), 404 (directory not found), 401 (auth required), 403 (access denied).

Data Specification

Asset Item

On success, a JSON array of the above objects is returned.

Create Directory Endpoint

POST .../dir/«path»

Creates a directory at the specified path. If any of the parent directories do not exist, they will be created as well. It is not an error if the directory already exists. This endpoint returns a status of 201 (success), 401 (auth required), 403 (access denied).

Delete API

The Asset Delete API is accessible under /endpoints/«AssetDirectoryName»/delete/..., and further extends the Content API by providing explicit endpoints for deleting items without using an HTTP DELETE request.

Delete Item Endpoint

POST .../delete/«path»?recursive=«true/false»

Deletes the asset or directory at the specified path. When recursive is false or not specified and the path refers to a directory, the directory will only be deleted if it is empty. When recursive is true, the item and all of its contents (if it is a directory) will be deleted. It is not an error if the directory does not exist. This endpoint returns a status of 200 (success), 400 (directory not empty and recursive=false), 401 (auth required), 403 (access denied).

Metadata API

The Asset Metdata API is accessible under /endpoints/«AssetDirectoryName»/metadata/..., and provides endpoints for reading or updating metadata on an asset or subdirectory. This API was added in ProGet v6.0.0.

Get Asset Metadata Endpoint

GET .../metadata/«path_to_asset»

Returns metadata for the specified asset file/directory as a JSON object with the following properties:

Update Asset Metadata Endpoint

POST .../metadata/«path_to_asset»

Updates the Content-Type or the user-defined metadata of an asset. The request body must have a Content-Type of application/json, and must contain a JSON object with the following properties:

Cache Header settings

See client-side caching for more information.

Export API

The Asset Export API is accessible under /endpoints/«AssetDirectoryName»/export/..., and provides endpoints for downloading batches of assets at once.

Export Directory Endpoint

GET .../export/«path»?format=«zip/tgz»&recursive=«true/false»

Returns the contents of the specified directory as either a ZIP or a TGZ archive. The format argument may be either zip (for a zip file) or tgz for a GZipped tar file. When recursive is false or not specified, only items contained directly in the specified path are included. When recursive is true, the archive will contain all subdirectories as well. This endpoint returns a status of 200 (success), 404 (directory not found), 400 (invalid format), 401 (auth required), 403 (access denied).

Import Directory Endpoint

POST .../import/«path»?format=«zip/tgz»&overwrite=«true/false»

Adds the contents of the uploaded archive to the specified path. The format argument may be either zip (for a zip file) or tgz for a GZipped tar file. When overwrite is false or not specified, items already in the asset directory will never be overwritten. When overwrite is true, items in the asset directory will be overwritten. If the specified directory does not exist, it will be created. This endpoint returns a status of 200 (success), 400 (invalid format), 401 (auth required), 403 (access denied).

Multipart Asset Upload

For very large assets, an upload can be performed using multiple requests using the multipart upload API. This endpoint functions in the same way as the standard Create Asset Endpoint, except requires some additional query string arguments. A multipart upload consists of one or more chunk uploads followed by a final POST to indicate that the upload is complete.

Note: Due to technical limitations, multipart uploads are currently not supported for cloud-based package stores. ProGet may extend support for multipart uploads to these in a later version, but currently any attempt to initiate a multipart upload with an S3/Azure backed package store will result in an HTTP 400 error.

Multipart Upload: Upload Chunk

POST .../content/«path_to_asset»?multipart=upload&id=«uuid»&index=«partIndex»&offset=«byteOffset»&totalSize=«byteSize»&partSize=«partSize»&totalParts=«partCount»

Initiates a new multipart upload or uploads the next part in a previously started multipart upload. All arguments are required:

• id - Unique identifier for the upload. It is the client's responsibility to generate this. It does not need to be a GUID, though a GUID is perfectly valid.
• index - Zero-based index of the part being uploaded.
• offset - Zero-based byte offset of the part being uploaded.
• totalSize - Size of the entire asset being uploaded in bytes. This must be exactly the sum of all individual uploaded part sizes.
• partSize - Size of the part being uploaded.
• totalParts - Total number of parts that will be uploaded for the entire asset.

This endpoint returns a status of 200 (success), 401 (auth required), 403 (access denied), 400 (invalid arguments such as index/offset out of range, overlapping parts, invalid size).

Multipart Upload: Complete Upload

POST .../content/«path_to_asset»?multipart=complete&id=«uuid»

Completes a multipart upload after all individual parts have been uploaded. The asset will not be added to the asset directory until the upload has been marked as complete with this endpoint.

This endpoint returns a status of 200 (success), 401 (auth required), 403 (access denied), 400 (invalid arguments such as index/offset out of range, overlapping parts, invalid size).

On success, the asset is saved to the asset directory.

Note: Orphaned parts will be deleted during the Feed Cleanup scheduled task. Orphaned parts can occur when a client uploads one or more parts but never completes the upload.

Multipart Upload: PowerShell Script

This PowerShell script can be used to automatically perform a multipart upload if necessary. Feel free to use it as-is, or as a starting point for your own solution:

<#
 .Synopsis
  Transfers a file to a ProGet asset directory.
 .Description
  Transfers a file to a ProGet asset directory. This function performs automatic chunking
  if the file is larger than a specified threshold.
 .Parameter FileName
  Name of the file to upload from the local file system.
 .Parameter EndpointUrl
  Full URL of the ProGet asset directory's API endpoint. This is typically something like http://proget/endpoints/<directoryname>
 .Parameter AssetName
  Full path of the asset to create in ProGet's asset directory.
 .Parameter ChunkSize
  Uploads larger than this value will be uploaded using multiple requests. The default is 5 MB.
 .Example
  Upload-ProGetAsset -FileName C:\Files\Image.jpg -AssetName images/image.jpg -EndpointUrl http://proget/endpoints/MyAssetDir
#>
function Upload-ProGetAsset {
param(
    [Parameter(Mandatory = $true)]
    [string] $fileName,
    [Parameter(Mandatory = $true)]
    [string] $endpointUrl,
    [Parameter(Mandatory = $true)]
    [string] $assetName,
    [int] $chunkSize = 5 * 1024 * 1024
)

function CopyMaxBytes {
param(
    [System.IO.Stream] $source,
    [System.IO.Stream] $target,
    [int] $maxBytes,
    [long] $startOffset,
    [long] $totalSize
)
    $buffer = [Array]::CreateInstance([System.Byte], 32767)
    $totalBytesRead = 0
    while ($true) {
        $bytesRead = $source.Read($buffer, 0, [Math]::Min($maxBytes - $totalBytesRead, $buffer.Length))

        if($bytesRead -eq 0) {
            break
        }
        $target.Write($buffer, 0, $bytesRead)
        $totalBytesRead += $bytesRead
        if($totalBytesRead -ge $maxBytes) {
            break
        }

        $overallProgress = $startOffset + $totalBytesRead
        Write-Progress -Activity "Uploading $fileName..." -Status "$overallProgress/$totalSize" -PercentComplete ($overallProgress / $totalSize * 100)
    }
}

if(-not $endpointUrl.EndsWith('/')) {
    $endpointUrl += '/';
}

$targetUrl = $endpointUrl + 'content/' + [Uri]::EscapeUriString($assetName.Replace('\', '/'))

$fileInfo = (Get-ChildItem -Path $fileName)
if($fileInfo.Length -le $chunkSize) {
    Invoke-WebRequest -Method Post -Uri $targetUrl -InFile $fileName
} else {
    $sourceStream = New-Object -TypeName System.IO.FileStream -ArgumentList ($fileName, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read, [System.IO.FileShare]::Read, 4096, [System.IO.FileOptions]::SequentialScan)
    try {
        $fileLength = $sourceStream.Length
        $remainder = [long]0
        $totalParts = [Math]::DivRem([long]$fileLength, [long]$chunkSize, [ref]$remainder)
        if($remainder -ne 0) {
            $totalParts++
        }

        $uuid = (New-Guid).ToString("N")

        for($index = 0; $index -lt $totalParts; $index++) {
            $offset = $index * $chunkSize
            $currentChunkSize = $chunkSize
            if($index -eq ($totalParts - 1)) {
                $currentChunkSize = $fileLength - $offset
            }

            $req = [System.Net.WebRequest]::CreateHttp("${targetUrl}?multipart=upload&id=$uuid&index=$index&offset=$offset&totalSize=$fileLength&partSize=$currentChunkSize&totalParts=$totalParts")
            $req.Method = 'POST'
            $req.ContentLength = $currentChunkSize
            $req.AllowWriteStreamBuffering = $false
            $reqStream = $req.GetRequestStream()
            try {
                CopyMaxBytes -source $sourceStream -target $reqStream -maxBytes $currentChunkSize -startOffset $offset -totalSize $fileLength
            }
            finally {
                if($reqStream -ne $null) {
                    $reqStream.Dispose()
                }
            }

            $response = $req.GetResponse()
            try {
            }
            finally {
                if($response -ne $null) {
                    $response.Dispose()
                }
            }
        }

        Write-Progress -Activity "Uploading $fileName..." -Status "Completing upload..." -PercentComplete -1

        $req = [System.Net.WebRequest]::CreateHttp("${targetUrl}?multipart=complete&id=$uuid")
        $req.Method = 'POST'
        $req.ContentLength = 0
        $response = $req.GetResponse()
        try {
        }
        finally {
            if($response -ne $null) {
                $response.Dispose()
            }
        }
    }
    finally {
        if($sourceStream -ne $null) {
            $sourceStream.Dispose()
        }
    }
}
}