Troubleshooting Tips
On this page
This page contains tips for troubleshooting problems that one may encounter when working with Chainguard Enforce.
Debug using chainctl
You can run chainctl
in debug mode using the verbose flag -v
or --v
and passing an integer value of 2
or larger, as in -v=2
.
Using this flag, you can log all requests and responses or errors.
For example, to grab all logs related to the clusters listed for a group, you can run:
chainctl cluster ls --group $DEMO_GROUP -v=2
The output will return log data to support you in debugging.
2023/03/28 22:23:58.418086 "level"=1 "msg"="Commandline flags" "active-within"="168h0m0s" "api"="api" "audience"="audience" "config"="" "console"="api" "group"="$DEMO_GROUP" "help"="false" "issuer"="oidc" "name"="" "output"="" "registry"="registry" "timestamp-authority"="timestamp" "v"="2"
2023/03/28 22:23:59.000042 gRPC: "level"=2 "msg"="iam.Groups.List" "request"="name:'$DEMO_GROUP' "
2023/03/28 22:23:59.089459 gRPC: "level"=2 "msg"="iam.Groups.List" "response"="items:<id:'' name:'$DEMO_GROUP' description:'Root of $DEMO_GROUP ' > "
2023/03/28 22:23:59.090213 gRPC: "level"=2 "msg"="tenant.Clusters.List" "request"="active_since:<seconds:1677651909 nanos:89511000 > uidp:<descendants_of:'' > "
2023/03/28 22:23:59.177413 gRPC: "level"=2 "msg"="tenant.Clusters.List" "response"=""
NAME | GROUP | REMOTEID | REGISTERED | K8S VERSION | AGENT VERSION | LAST SEEN | ACTIVITY
-------+-------+----------+------------+-------------+---------------+-----------+-----------
By default, your next command passed without the -v
flag will revert to the standard chainctl
experience. Alternatively, you can pass -v=0
to your chainctl
command for the default experience, without logging.
How to disable admission control
There may be urgent situations where having admission control enabled is not desirable. For example, it could be preventing the cluster from functioning correctly or may be getting in the way of the operator.
In cases like this, you can completely disable the Chainguard Enforce admission webhook with the following commands.
kubectl delete validatingwebhookconfiguration enforcer
kubectl delete mutatingwebhookconfiguration enforcer
After your urgent situation is over, reinstall Enforce in your cluster to restore the webhook to its normal operation.
Chainguard Enforce is unable to parse a given SBOM
One issue that may come up when working with Chainguard Enforce is that it won’t ingest an SBOM as expected. This section outlines several potential causes for this issue and how you can address them.
Check your permissions
Chainguard Enforce needs access to your images in order to parse the associated SBOMs. If your image is in a private repository, check out our guide on setting up cloud account associations to grant Enforce read access to the image.
SBOMs using older versions of CycloneDX
For CycloneDX, Chainguard Enforce currently only supports version 1.4.
SBOM included as an in-toto attestation
We are in the process of updating Chainguard Enforce so it can readily parse SBOM attestations out of the box. In the meantime, you can parse SBOMs through implementing specific policies that cover this use case.
You can create a policy covering your image through the Chainguard Enforce console or using the chainctl
command line tool. Additionally, you can check out our policy catalog for our collection of policies that work directly with Chainguard Enforce.
Policy does not cover a given image as expected
When working with the ClusterImagePolicy
, note that the glob wildcard *
does not cover the /
character. For example, a glob pattern like gcr.io/apple/*
will cover paths like gcr.io/apple/yak
or gcr.io/apple/zebra
, but not gcr.io/apple/banana/cashew
.
To match everything — including the /
character — use the **
wildcard instead.
Enforce does not block Pod creation as expected
The first thing to check is whether you labeled your namespace with policy.sigstore.dev/include=true
. You can double check whether the default
namespace is correctly labeled with the following command.
kubectl get ns -l policy.sigstore.dev/include=true
If it is indeed labeled like this, you’ll receive output like the following.
NAME STATUS AGE
default Active 24s
If you need to label your namespace, you can do so with the following command. Note that this example labels the default
namespace.
kubectl label ns default policy.sigstore.dev/include=true --overwrite
Be sure to use the exact string, variations like included
won’t work.
Sometimes Chainguard Enforce is installed using the observer
profile. Essentially, this means that Enforce just has read-only permission for workload discovery, and it cannot actually enforce policies. You can run chainctl cluster ls
to find what profile the cluster is using.