HOWTO: Proxy Packages from PyPI.org in ProGet

ProGet let's you create "Feeds" to proxy packages from The Python Package Index (PyPI) and install them just as you would when installing them from PyPI directly.

Using ProGet as a proxy will let you assess vulnerabilities in PyPI packages, tell you which packages are being downloaded and used frequently, and cache packages, allowing teams to access them even if PyPI is down.

This article will look at how to set up a feed in ProGet to proxy PyPI packages, as well as how to create a private repository for your internal Python packages. The PyPI commands on this page use pip, however you can also integrate other tools such as PipEnv and Poetry with your PyPI feeds.

Step 1: Create a New Feed

To begin, we will create a PyPI feed that will proxy Python packages from PyPI. Select "Feeds" and "Create New Feed". Next, select "Python Packages" under "Developer Libraries".

Now select "Connect to PyPI.org" which will allow us to proxy packages from PyPI.

Then select "No, Create One Feed", as we will be creating a single feed to proxy PyPI packages. From here, name the feed (we will call it public-pypi for this example). Then click "Create Feed".

The next several options will let your feed use ProGet's Vulnerability Scanning and Blocking amd Licensing Detection and Blocking features, letting you assess vulnerabilities and create policies for licenses. Select "Set Feed Features", which will create the feed, and redirect you to the newly created public-pypi feed, which will list packages proxied from PyPI.

Step 2: Using the Feed in Python Clients

To let your teams use the public-pypi feed when installing packages you can either include it when running the pip install command, or set it globally with the pip config command. You can also add it if using PipEnv or Poetry.

Using pip config

To set your public-pypi feed globally as a default source for all installations, you can store it in the pip config file. Use the pip config command with a --global parameter containing your public-pypi endpoint URL.

$ pip config --global set global.index-url https://«proget-server»/pypi/public-pypi/simple

$ pip config --global set global.index-url https://proget.corp.local/pypi/public-pypi/simple 

This command will generate a pip config file that looks like:

[global]
index-url = https://proget.corp.local/pypi/public-pypi/simple 

Using pip install

You can also use pip install for one-off package installations. However, as it is not persistent, you'll need to enter your public-pypi feed URL for every installation. For a more long-term setup, it's better to use a pip config file.

To install Python packages with the pip install command, you will need to add a --extra-index-url parameter containing the endpoint URL of your public-pypi feed:

$ pip install «package-name»==«package-version» --extra-index-url https://«proget-server»/pypi/public-pypi/simple

$ pip install flask==2.3.3 --extra-index-url proget.corp.local -i https://proget.corp.local/pypi/public-pypi/simple

(Optional) Authenticating to Your PyPI Feed

By default your public-pypi feed does not need to be authenticated to can be viewed anonymously. However if you've configured your feed to require authentication, you can authenticate to it when using pip install or with pip config. Alternatively you can authenticate with PipEnv or Poetry.

(Optional) Creating a Package Approval Flow

In this guide we looked at proxying packages from PyPI. However, without oversight approval, developers will be able to install any OSS packages from PyPI without oversight. We recommend implementing some form package vetting of your PyPI packages, which can be done by creating a "Package Approval Flow".

To set up a package approval flow, refer to HOWTO: Approve and Promote Open-source Packages. The guide uses NuGet feeds as an example, but the steps are identical when creating PyPI package feeds.

After creating your "Unapproved" and "Approved" feeds, follow the steps in "Using the Feed in Python Clients" to add the "Approved" feed (e.g. pypi-approved) as a source.