Global configuration

Python library consumption in a large organization is typically managed by a repository manager. Commonly used repository manager applications are Cloudsmith, JFrog Artifactory, and Sonatype Nexus Repository. The repository manager acts as a single point of access for developers and development tools to retrieve the required libraries.

At a high level, adopting the use of Chainguard Libraries consists of the following steps:

• Add Chainguard Libraries as a remote repository for library retrieval.
• Add the public PyPI repository as a remote repository.
• Create a group, virtual, or polyglot repository combining these repository sources with any desired internal repositories. Configure the Chainguard Libraries repository as the first choice for any library access after any desired internal repositories.

You should also:

• Remove all prior cached artifacts in the virtual server or proxy public repository. This step reduces confusion about the origin of libraries and assists technical evaluation and adoption of Chainguard Libraries.
• Remove any repositories that are no longer desired or necessary. Depending on your library requirements, this step can result in removal of some proxy repositories or even removal of all proxy repositories.

If your organization does not use a repository manager, you can still use Chainguard Libraries. However, this approach requires configuration of multiple build and development platforms and utilities to use Chainguard Libraries. For this reason, adopting the use of a repository manager is the recommended approach. Refer to the direct access documentation for build tools for more information.

Considerations for fallback approach

Before configuring your repo manager, consider how you want to handle packages that aren’t yet available in the Chainguard Libraries repository. If you configure a fallback to PyPI, packages sourced from that registry are not covered by Chainguard’s malware-resistance guarantees. See the fallback approaches described in the Chainguard Libraries quick start for guidance on choosing the right approach for your environment.

Cloudsmith

Cloudsmith supports Python repositories for proxying and hosting and polyglot repositories that combine multiple repositories sources with compatible formats. Refer to the Cloudsmith Python Repository documentation and the Cloudsmith documentation for creating a repository for more information.

Initial configuration

Use the following steps to add a repository with both Chainguard Libraries for Python and PyPI as upstream sources.

First, create a repository:

1. Log in to your Cloudsmith instance as user with administrator privileges.
2. Select the Repositories tab near the top of the screen.
3. Navigate to the Repositories Overview, then select + New repository.
4. At the new repository form, enter the name python-all for your new repository. The name should include python to identify the repository format. This convention helps avoid confusion, since repositories in Cloudsmith are multi-format.
5. Select a storage region that is appropriate for your organization and infrastructure.
6. Select + Create Repository.

Next, configure the upstream proxies:

1. Select the name of the new python-all repository on the repositories page to configure it.
2. Access the Upstreams tab and click + Add Upstream Proxy.
3. Configure an upstream proxy with the format python and the following details:
   • Name: python-chainguard
   • Priority: 1
   • Upstream URL: https://libraries.cgr.dev/python/
   • Mode: Cache and Proxy
   • Add the Username and Password value from Chainguard Libraries access in Authentication Settings
4. Select Create Upstream Proxy.
5. If you want to use the separate repository with remediated Python libraries, repeat the preceding two steps with the name python-chainguard-remediated, the priority 2, the same authentication details, and the URL https://libraries.cgr.dev/python-remediated/.
6. Configure another upstream proxy with the following details
   • Name: python-public
   • Priority: 3
   • Upstream URL: https://pypi.org/
   • Mode: Cache and Proxy
7. Select Create Upstream Proxy.

Build tool access

See the page on build tool configuration for Chainguard Libraries for Python for information on accessing credentials and setting up build tools.

Google Artifact Registry

Google Artifact Registry supports the Python format for hosting artifacts in Standard repositories and proxying artifacts from public repositories in Remote repositories. Use Virtual repositories to combine them for consumption with pip and other build tools. Use the Python package documentation for Google Artifact Registry as the starting point for more details.

Initial configuration

Use the following steps to add the Pypi Package Index and the Chainguard Libraries for Python repository as remote repositories and combine them as a virtual repository:

1. Log in to the Google Cloud console as a user with administrator privileges.
2. Navigate to your project and find the Artifact Registry with the search.
3. Activate Artifact Registry if necessary.
4. Navigate to your project and find the Secret Manager with the search.
5. Activate Secret Manager if necessary.

Before configuring the repositories, you must create a secret with the password value as retrieved with chainctl:

1. Navigate to the Secret Manager
2. Press Create secret.
3. Set the Name to chainguard-libraries-python.
4. Use the Password from chainctl output to set the Secret value.
5. Press Create secret.

Navigate to Artifact Registry and select Repositories in the left hand navigation under the Artifact Registry label to configure a remote repository for the Pypi Package Index:

1. Press Create a Repository or the + button.
2. Set the Name to python-public.
3. Set the Format to Python.
4. Select Remote for the Mode.
5. Select PyPi for the Remote repository source.
6. Choose a suitable Region for your development in Location type.
7. Press Create.

Configure a remote repository for the Chainguard Libraries for Python repository:

1. Press the + button to add another repository.
2. Set the Name to python-chainguard.
3. Set the Format to Python.
4. Select Remote for the Mode.
5. Select Custom for the Remote repository source.
6. Set the URL for the Custom repository to https://libraries.cgr.dev/python/.
7. Select Authenticated in Remote repository authentication mode.
8. Set Username for the upstream repository to the value as retrieved with chainctl.
9. Select the chainguard-libraries-python secret in the list for the Secret input.
10. Choose the same suitable Region for your development in Location type as configured for the python-public repository.
11. Press Create.

Combine the two repositories in a new virtual repository:

1. Press the + button to add another repository.
2. Set the Name to python-all.
3. Set the Format to Python.
4. Select Virtual for the Mode.
5. Press Add upstream repository in Virtual upstream repositories.
6. Use the Browse button to locate and select the python-chainguard repository as Repository 1 and set the Policy name 1 to python-chainguard.
7. Use the Browse button to locate and select the python-public repository as Repository 2 and set the Policy name 2 to python-public.
8. Press Add upstream repository in Virtual upstream repositories.
9. Set the Priority value for the python-chainguard policy name to a higher value than the python-public priority value.
10. Choose the same suitable Region for your development in Location type as configured for the python-public repository.
11. Press Create.

JFrog Artifactory

JFrog Artifactory supports PyPI repositories for proxying and virtual repositories to combine multiple sources into a single repository. The following instructions are based on the PyPI Repository documentation for Artifactory.

Initial configuration

Use the following steps to add the Chainguard Libraries for Python index and the PyPI public index as remote repositories and combine them as a virtual repository:

1. Log in as a user with administrator privileges.
2. Press Administration in the top navigation bar.
3. Select Repositories in the left hand navigation.

Configure a remote repository for the Chainguard Libraries for Python index:

1. Select Create a Repository and choose the Remote option.
2. Select PyPI as the Package type.
3. Set the Repository Key to python-chainguard.
4. Set the URL to https://libraries.cgr.dev/python/.
5. Set User Name and Password / Access Token to the values as retrieved with chainctl.
6. Set the PyPI Settings - Registry URL to https://libraries.cgr.dev/python/.
7. Optionally click the Test button to verify connection and authentication.
8. Access the Advanced configuration tab and deactivate the Block Mismatching Mime Types setting in the Others section.
9. Press Create Remote Repository.

If you want to use the separate repository with remediated Python libraries repeat the preceding steps with the name python-chainguard-remediated, the same authentication details, and the URL https://libraries.cgr.dev/python-remediated/.

Configure a remote repository for the PyPI public index:

1. Select Create a Repository and choose the Remote option.
2. Select PyPI as the Package type.
3. Set the Repository Key to python-public.
4. Set the URL to https://files.pythonhosted.org.
5. Set the PyPI Settings - Registry URL to https://pypi.org/.
6. Select Create Remote Repository.

Combine the two repositories in a new virtual repository:

1. Press Create a Repository and choose the Virtual option.
2. Select PyPI as the Package type.
3. Set the Repository Key to python-all.
4. In the Repositories section, find the python-chainguard and python-public repositories. Ensure the python-chainguard repository is the first in the displayed list. Use the icon on the right of the repository name to drag and drop repositories into the desired position.
5. Select Create Virtual Repository.

At this point, you have a virtual repository set up in Artifactory that allows you or others in your organization to access Chainguard Libraries for Python, optionally including remediated versions, with your chosen tools. This setup falls back to the public PyPI index in cases where a package is not available in Chainguard’s index.

Validate the remote repository

After creating the python-chainguard remote repository, validate that Artifactory is successfully proxying through to Chainguard before proceeding. Because Artifactory falls back to the public PyPI index when a connection to a remote repository fails, a misconfigured repository may silently resolve packages from PyPI rather than Chainguard — and the build will succeed without any visible error.

Common sources of misconfiguration include invalid or expired credentials, or an incorrect or incomplete repository URL. The Artifactory Test button on the repository configuration screen is not a reliable indicator; it may fail for a correctly configured repository, and may pass for an incorrectly configured one. Instead, use the following steps to verify that fetching an artifact through Artifactory produces the same checksum as fetching it directly from libraries.cgr.dev.

1. Find the direct URL for a specific package wheel from the Chainguard index. This example uses urllib3. You can substitute any artifact you know to be available.

curl -sSf \
  -u "${CHAINGUARD_PYTHON_IDENTITY_ID}:${CHAINGUARD_PYTHON_TOKEN}" \
  https://libraries.cgr.dev/python/simple/urllib3/ \
  | grep -o 'https://[^"]*\.whl' | head -1

2. Fetch a package file directly from libraries.cgr.dev and compute its checksum:

curl -sSf -L \
  -u "${CHAINGUARD_PYTHON_IDENTITY_ID}:${CHAINGUARD_PYTHON_TOKEN}" \
  <url-from-step-1> \
  | sha256sum

3. Fetch the same file through the Artifactory remote repository and compute its checksum:

curl -sSfL \
  -u "${ARTIFACTORY_USERNAME}:${ARTIFACTORY_TOKEN}" \
  "https://<artifactory-host>/artifactory/<python-remote-repository>/${path-to-wheel}" \
  | sha256sum

Replace artifactory-host with your Artifactory instance hostname and replace python-remote-repository with your remote repository name. Replace path-to-wheel with the path component of the URL from step 1 (for example: /files/15f7d141c3b76b85/37e321caa85a8f41/urllib3/urllib3-1.26.9-py2.py3-none-any.whl)

The checksums returned by the commands must match.

If the checksum from the Artifactory remote repository differs from the direct fetch, or if the Artifactory fetch fails entirely, review the following before proceeding:

• URL: The remote repository URL must be set to https://libraries.cgr.dev/python/.
• Credentials: You may need to regenerate your pull token with chainctl auth pull-token --repository=python and update the Artifactory repository credentials. Expired tokens fail silently.
• Advanced Configuration: Ensure all recommended Advanced settings from the initial configuration steps have been applied.

Do not proceed to virtual repository setup or build configuration until the checksums match.

Build tool access

See the page on build tool configuration for Chainguard Libraries for Python for information on accessing credentials and setting up build tools.

Sonatype Nexus Repository

Sonatype Nexus Repository allows for merging multiple remote repositories as a repository group. The below instructions are based on the Nexus documentation for PyPI

Initial configuration

The following steps create remote repositories for Chainguard Libraries for Python, a remote repository for the public PyPI index, and a repository group combining these sources.

First, log in to Sonatype Nexus as a user with administrator privileges and access the Server administration and configuration section within the gear icon in the top navigation bar.

Next, configure a remote repository for the public PyPI index:

1. Select Repository - Repositories in the left hand navigation.
2. Select Create repository.
3. Select the PyPI (proxy) recipe.
4. Provide a new name, such as python-public.
5. In the Proxy - Remote storage field, add the following URL: https://pypi.org/.
6. Select Create repository.

Configure a remote repository for the Chainguard Libraries for Python repository:

1. Select Repository - Repositories in the left hand navigation.
2. Select Create repository.
3. Select the PyPI (proxy) recipe.
4. Provide a new name, such as python-chainguard.
5. In the Proxy - Remote storagefield, add the following URL: https://libraries.cgr.dev/python/.
6. In HTTP - Authentication, set the Authentication type to username and enter the the username and password values as retrieved with chainctl.
7. Select Create repository.

If you want to use the separate repository with remediated Python libraries repeat the preceding steps with the name python-chainguard-remediated, the same authentication details, and the URL https://libraries.cgr.dev/python-remediated/.

Finally, create a new repository group and add the two repositories:

1. Select Repository - Repositories in the left hand navigation.
2. Select Create repository.
3. Select the PyPI (group) recipe.
4. Provide a new name, such as python-all.
5. In the section Group - Member repositories, move the new repositories python-public and python-chainguard to the right and move the python-chainguard repository to the top of the list with the arrow control. If you configured the python-chainguard-remediated repository, also move it to the right and the top of the list.

Build tool access

See the page on build tool configuration for Chainguard Libraries for Python for information on accessing credentials and setting up build tools.