Cosign: The Manual Way

Eddie Zaneski

When I first used Cosign, the software artifact signing CLI from the Sigstore project, I was amazed at how painless signing and verifying could be.

For example, in the three commands below we create a public/private key pair, sign the text file, upload it to the Rekor transparency log, and verify the signature of the message.

# create public/private keys
$ cosign generate-key-pair
Enter password for private key:
Enter password for private key again:
Private key written to cosign.key
Public key written to cosign.pub

# sign an artifact and output the signature
$ cosign sign-blob --key cosign.key --output-signature sig message.txt
Using payload from: message.txt
Enter password for private key:
tlog entry created with index: 2014997
Signature wrote in the file sig

# verify the signature
$ cosign verify-blob --key cosign.pub --signature sig message.txt
tlog entry verified with uuid: 7e53fd5d089af142b7909598d214e13ca76001cc575fddaad3210adbee86363e index: 7155742
Verified OK

These quick commands show how easily Sigstore can be integrated into software security practices for any developer. I am the kind of person who always wants to know what’s going on under the hood. In this tutorial, I will walk you through doing Cosign the manual way so you too can understand the ins and outs of Cosign.

Tools

If you want to follow along you’ll need the following installed from your package manager of choice. I’ve noted the version used in this post but different minor versions should be fine. However, keep in mind that OpenSSL can drastically vary per system.

openssl (3.0.8) Note: macOS uses libressl but it should still work
jq (jq-1.6)
curl (7.88.1)
xxd (2022-01-14) Note: macOS requires coreutils installed via Homebrew
go (go1.20.2)
cosign (v2.0.0)
rekor-cli (v1.0.1)

The Blob

A blob is an arbitrary collection of raw data like a picture or the executable binary that your source code produces. Cosign is capable of signing and verifying blobs.

In our case, we’ll be signing a spooky message. Let’s write that message to a file.

$ echo 'Beware The Blob!' > message.txt

We’ll be using Cosign to sign this .txt file blob.

Keys

Before we can sign our message we need to generate a key pair. A key pair is a set of two keys, one private and one public, that can both sign/verify and encrypt/decrypt data. The private key is generated by some algorithm and the public key is derived from the private.

There are a handful of different algorithms in use today, but the most common are RSA, ECDSA, and Ed25519. I fell deep down the rabbit hole learning about the differences between these algorithms, potential NSA backdoors and which versions of OpenSSL support a given algorithm. To keep things simple, we’ll use RSA but I encourage you to chase the rabbit on your own.

The RSA algorithm in use today is defined by the Public Key Cryptography Standards #1 (PKCS1) specification. Let’s generate a 4096 bit RSA private key with OpenSSL.

$ openssl genrsa -out key.pem 4096
Generating RSA private key, 4096 bit long modulus (2 primes)
......................................................................................................................
.............++++
.................++++
e is 65537 (0x010001)

And from that private key we can output the public key.

$ openssl rsa -in key.pem -pubout -out pub.pem
writing RSA key

$ cat key.pem
-----BEGIN RSA PRIVATE KEY-----
MIIJKgIBAAKCAgEA1BgrTaqV3zS+TOx6A/n+59ECOlXl7Uk7W82wNe7kUgfVAIGj
Bci+Tc7O/nf/7GCMlzli/4n5WE0Ny2i/Kj4Ycsu6TUEcW6XaJSz4R4TBTHAcQiNq
8EkBQ2S5SuIIEekvCdVffkob3NtipOd/FaiLS1NVUAFcqOGHl2DYEkhP2puBS+Ad
…

$ cat pub.pem
cat pub.pem
-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA1BgrTaqV3zS+TOx6A/n+
59ECOlXl7Uk7W82wNe7kUgfVAIGjBci+Tc7O/nf/7GCMlzli/4n5WE0Ny2i/Kj4Y
csu6TUEcW6XaJSz4R4TBTHAcQiNq8EkBQ2S5SuIIEekvCdVffkob3NtipOd/FaiL
…

We now have our keys but this is still a bit of magic. What is a PEM and what is this syntax? To answer that, let’s look at doing the same thing we just did in Go (omitting errors for brevity).

// main.go
package main

import (
	"crypto/rand"
	"crypto/rsa"
	"crypto/x509"
	"encoding/pem"
	"os"
)

func main() {
	key, _ := rsa.GenerateKey(rand.Reader, 4096)
	b := x509.MarshalPKCS1PrivateKey(key)
	priv := pem.EncodeToMemory(&pem.Block{
		Type:  "RSA PRIVATE KEY",
		Bytes: b,
	})
	os.WriteFile("key.pem", priv, 0600)

	b, _ = x509.MarshalPKIXPublicKey(key.Public())
	pub := pem.EncodeToMemory(&pem.Block{
		Type:  "PUBLIC KEY",
		Bytes: b,
	})
	os.WriteFile("pub.pem", pub, 0600)
}

Go’s crypto library is a joy to work with and the implementation for rsa.GenerateKey is worth peeking at. After calling it to generate our private key, we have a bunch of mathy bits that we need to represent somehow — enter Abstract Syntax Notation number One (ASN.1) and Distinguished Encoding Rules (DER).

ASN.1 is the syntax we express our key in (think a .proto file) and DER is how we encode it (think Protobuf wire format). For a more detailed explanation, Let’s Encrypt has a great article. After calling x509.MarshalPKCS1PrivateKey our private key is an ASN.1 DER encoded PKCS1 byte array 😅.

We still have one last step, however. Byte arrays aren’t human readable and are difficult to copypasta. As we often do when transmitting binary data we need to Base64 encode our bytes. Slapping on a header (-----BEGIN RSA PRIVATE KEY------) and footer (-----END RSA PRIVATE KEY-----) to help parsers identify our content finally leaves us with our Privacy-Enhanced Mail (PEM) file. PEM is a standard container for certificates and keys and was created to send binary data over email without messing with their contents.

Since PKCS1 is specific to RSA our public key is serialized to Public Key Infrastructure X.509 (PKIX) which is a generic public key representation that includes information like the key algorithm. X.509 is the standard that defines the format for public key certificates like those used in your web browser. You can see above that the header is the generic -----BEGIN PUBLIC KEY-----.

Time to Sign

With our keys created, we are ready to use them to sign our message. We could sign the file in its entirety, but that would require the full file when verifying the signature. This is fine for small text files but when signing and uploading 3 gigabyte containers, it’s much more efficient to hash the payload and sign the digest. That way all that’s needed to verify a signature is the public key and the hash value. SHA-256 is most commonly used today for computing a message digest.

$ openssl dgst -sha256 -sign key.pem -out message.txt.sig message.txt

We can confirm our signing was successful by verifying the signature with the public key.

openssl dgst -sha256 -verify pub.pem -signature message.txt.sig message.txt
Verified OK

Get Transparent

Now that we have our signature, we can upload everything to the Rekor transparency log so others can find and verify it. Rekor supports a handful of different distinct types including Java JARs and RPM packages. The basic type is known as a Rekord but since we signed the hash of our file we’ll use a Hashed Rekord.

Most of the fields in Rekor’s types require Base64 encoding, so let’s store these values in environment variables to make writing the payload easier.

$ SIGSTORE_SIG_CONTENT=$(cat message.txt.sig | base64 | tr -d '\n')
$ SIGSTORE_PUBLIC_KEY=$(cat pub.pem | base64 | tr -d '\n')
$ SIGSTORE_HASH_CONTENT=$(shasum -a 256 message.txt | cut -d " " -f 1)

Next, we’ll write out the payload to a file to make it easier to inspect and use in a request.

$ cat <<EOF > hashedrekord.json
{
  "apiVersion": "0.0.1",
  "kind": "hashedrekord",
  "spec": {
    "data": {
      "hash": {
        "algorithm": "sha256",
        "value": "$SIGSTORE_HASH_CONTENT"
      }
    },
    "signature": {
      "content": "$SIGSTORE_SIG_CONTENT",
      "publicKey": {
        "content": "$SIGSTORE_PUBLIC_KEY"
      }
    }
  }
}
EOF

Do a quick sanity check that our payload was created successfully.

$ cat hashedrekord.json
{
  "apiVersion": "0.0.1",
  "kind": "hashedrekord",
  "spec": {
    "data": {
      "hash": {
        "algorithm": "sha256",
        "value": "d8d321
…

We’re now ready to send our data off to Rekor. Let’s save the response to a file so we can poke around.

$ curl -X POST -H "Content-Type: application/json" --data-binary @hashedrekord.json https://rekor.sigstore.dev/api/v1/log/entries > response.json

The top-level key of the response will be the database shard ID (16 characters) + entry UUID (64 characters) in the transparency log.

# top level key
$ jq -r 'keys[0]' response.json
24296fb24b8ad77a7e53fd5d089af142b7909598d214e13ca76001cc575fddaad3210adbee86363e

# shard id
$ jq -r 'keys[0]' response.json | cut -c -16
24296fb24b8ad77a

# entry uuid
$ jq -r 'keys[0]' response.json | cut -c 17-
7e53fd5d089af142b7909598d214e13ca76001cc575fddaad3210adbee86363e

The entry UUID is actually the hash of the Merkle Tree leaf node for our entry. I won’t attempt to explain Merkle Trees in depth, but the basic idea is that inclusion of a node can be cryptographically verified all the way up to the root hash of the tree. RFC 6962 explains how this works and we’ll step through this verification in a bit. For now, let’s see where this hash comes from.

The RFC states that the hash of a leaf node is SHA-256(0x00 || d(n)). That is, the SHA 256 sum of the hex byte 0x00 concatenated to the contents of the entry, which in our case is the Hashed Rekord. We can do this in bash with process substitution.

$ shasum -a 256 <(cat <(printf '\x00') <(jq -rcj '.' hashedrekord.json )) | cut -d ' ' -f 1
7e53fd5d089af142b7909598d214e13ca76001cc575fddaad3210adbee86363e

Now let’s retrieve the entry from Rekor using this ID and save it to a file. If you inspect the contents, you’ll notice a handful of new fields.

$ curl https://rekor.sigstore.dev/api/v1/log/entries/24296fb24b8ad77a7e53fd5d089af142b7909598d214e13ca76001cc575fddaad3210adbee86363e > entry.json

First, let’s check that our Hashed Rekord is in the body.

$ diff <(jq -rc '.[].body' entry.json | base64 -d) <(jq -rcj '.' hashedrekord.json)

The body matches, which confirms that our entry was created, but how would the recipient of our message go about verifying and proving that for themselves? That’s where our next step comes in.

Trust but Verify

Well, there are a few important things they must check:

That the message digest matches the signature for the public key inside the Hashed Rekord. (We’ll leave this as an exercise for you, dear reader: you can reverse the steps from earlier.)
That we were in possession of the key when the message was signed. (You could check AWS/GCP KMS access logs)
That the entry is indeed included in the transparency log. (RFC 6962)

The next thing that we can verify in the entry is the Signed Entry Timestamp. I’ll leave this explanation to our friend Hayden Blauzvern.

As a transparency log, Rekor provides cryptographic proofs of inclusion in a log. Fetching an inclusion proof requires querying the log. The log returns a checkpoint (signed tree head) as a commitment to the current state of the log and the inclusion proof.

Requiring an online lookup for every entry that you’re verifying could cause a lot of increased latency in a verifier, and requires that the log have very high availability. Ideally, Rekor could provide an inclusion proof that could be verified offline – Rekor does this with a “signed entry timestamp” (SET).

An SET is a structure signed with the same private key that signs Rekor’s checkpoints. It is a “promise” of inclusion. It does not contain cryptographic proof, but since it is signed by the log, the log is committing to including the entry. A verifier that trusts Rekor can verify the SET without needing to do an online lookup. Asynchronously, for additional assurances, a log monitor can verify that an entry is truly present in the log for each SET a verifier views.

We can start by fetching Rekor’s current public key.

$ curl https://rekor.sigstore.dev/api/v1/log/publicKey > rekor.pub

Next we can pull the SET out of the entry into its own file.

jq -r '.[].verification.signedEntryTimestamp' entry.json | base64 -d > set.sig

The attestation and verification fields in the entry are not included in what is signed by the timestamping authority, so let’s remove them.

jq -cj '.[] | del(.attestation, .verification)' entry.json > set.json

Finally, we can verify the SET.

$ openssl dgst -sha256 -verify rekor.pub -signature set.sig set.json
Verified OK

The last thing we need to verify is that our entry was actually included in the Merkle tree. As mentioned earlier, this is defined in RFC 6962. RFC 9162 will eventually replace 6962 and the pseudocode is easier to follow (read through it first).

What follows is a bash implementation of this algorithm that I am equally proud of and upset by. It builds on everything we’ve covered so far with the addition of the xxd tool. xxd is used to convert between binary and hex and we use it to build up the binary representation of our tree nodes from the hashes in our entry. These hashes should eventually compute to the rootHash.

#!/usr/bin/env bash

set -euo pipefail

# https://datatracker.ietf.org/doc/rfc9162/#:~:text=2.1.3.2.%20%20Verifying%20an%20Inclusion%20Proof

entry=$(cat entry.json)
mapfile -t hashes < <(jq -rc '.[].verification.inclusionProof.hashes | .[]' <<< "$entry")
rootHash=$(jq -r '.[].verification.inclusionProof.rootHash' <<< "$entry")
startHash=$(shasum -a 256 <(cat <(printf '\x00') <(jq -r '.[].body' <<< "$entry" | base64 -d)) | cut -d ' ' -f 1)
logIndex=$(jq -r '.[].verification.inclusionProof.logIndex' <<< "$entry")
treeSize=$(jq -r '.[].verification.inclusionProof.treeSize' <<< "$entry")

if [[ $logIndex -ge $treeSize ]]; then
  echo "verification failed! log index larger than tree size"
  exit 1
fi

echo -e "0x00 || leaf\nstart: ${startHash}\n\n"
echo -e "want: ${rootHash}\n\n"

r="${startHash}"
fn="${logIndex}"
sn=$(( treeSize - 1 ))

for i in "${!hashes[@]}"
do
  if [[ $sn -eq 0 ]]; then
    echo "verification failed! tree is incomplete"
    exit 1
  fi

  lsb=$(( fn & 1 ))

  if [[ ($lsb -eq 1) || ($fn -eq $sn) ]]; then
    echo "0x01 || ${hashes[i]} || ${r}"
    r=$(shasum -a 256 <(cat <(printf '\x01') <(xxd -r -p <<< "${hashes[i]}") <(xxd -r -p <<< "${r}")) | cut -d ' ' -f 1)
    while [[ ($lsb -eq 0) || $fn -eq 0 ]]; do
      fn=$(( fn >> 1 ))
      sn=$(( sn >> 1))
      lsb=$(( fn & 1 ))
    done
  else
    echo "0x01 || ${r} || ${hashes[i]}"
    r=$(shasum -a 256 <(cat <(printf '\x01') <(xxd -r -p <<< "${r}") <(xxd -r -p <<< "${hashes[i]}")) | cut -d ' ' -f 1)
  fi
  fn=$(( fn >> 1 ))
  sn=$(( sn >> 1))
  echo -e "${r}\n\n"
done

if [[ "${r}" == "${rootHash}" ]]; then
  echo "verification successful! got: ${r} want: ${rootHash}"
  exit 0
else
  echo "verification failed! got: ${r} want: ${rootHash}"
  exit 1
fi

Running this monstrosity, we can see that verification was successful!

$ ./verify.sh
./verify.sh
0x00 || leaf
start: 7e53fd5d089af142b7909598d214e13ca76001cc575fddaad3210adbee86363e


want: b054e63c475d8751b14b51a54255918e4ae0aa26e0ee60c9c7ee3e333396c4ad


0x01 || 2fb6c414fe3710a0ca820e20a352d827e920ef94de1a62cf032f0b508583d343 || 7e53fd5d089af142b7909598d214e13ca76001cc575fddaad3210adbee86363e
16e4d477cd081846cd410daf07299d17056dc5b31f77c40a716e8520a0457b1c


0x01 || af8df402b3a2f95a2e11e693d0401237c23ab1ef29844ff5b122ef7b868eb802 || 16e4d477cd081846cd410daf07299d17056dc5b31f77c40a716e8520a0457b1c
6e76df3263704eccc1c047bbc1704bbf8071a817a0c5626a2308da7d9bf3cae9


0x01 || 54b4fb9c6af94d6c8c9361176de175cb26334f26cf0c2238c8dda23ca256f7b1 || 6e76df3263704eccc1c047bbc1704bbf8071a817a0c5626a2308da7d9bf3cae9
dff1cbc7728565c69328b8618b14fe7033baa5e8936098ec8933009c3988104e


0x01 || dff1cbc7728565c69328b8618b14fe7033baa5e8936098ec8933009c3988104e || 7e12ab8d4338713ca3a427db4eb41ecdbe3e6ae15fd490716409d8c89c63ae1d
6ad77f4dcac7a5d47d3087283129429d50cd2d9a4410f5d4900afd78e33ad40a


0x01 || 94cc41349e90789f7f80dce5df0339373749400539d41c8a3ecca485e3c92603 || 6ad77f4dcac7a5d47d3087283129429d50cd2d9a4410f5d4900afd78e33ad40a
4fdd278ae02e4cbae8dbc0650e4d3282e208ffec9603c68f73bcafe328488ac7


0x01 || a50a2b2e4a506dd075f59dab6db168164eb52e437756f53162f04be6ce0b5c3b || 4fdd278ae02e4cbae8dbc0650e4d3282e208ffec9603c68f73bcafe328488ac7
65cfed18ce41b642653912e86402b45d5df0109de29edbf0cb20c62f3444fdc0


0x01 || 65cfed18ce41b642653912e86402b45d5df0109de29edbf0cb20c62f3444fdc0 || 52f10e8d704883a11ed3f157d079afffd814f563d5c013d5752ea81744aac4b5
749a1795a60841356d1928b78af94814b127b7fea9aede093eb39b410e166f27


0x01 || a0441f106e6bb9ec4acc5f3126beea2ae60b721648c3c7ba741368458cef89c6 || 749a1795a60841356d1928b78af94814b127b7fea9aede093eb39b410e166f27
5d0ebe394bc76ddf46868b03466d3042b4efcb70760f997731ee6758c272208f


0x01 || 5d0ebe394bc76ddf46868b03466d3042b4efcb70760f997731ee6758c272208f || df2f19f10ed0def555c68cda80b4fdd8f535ba929c1cfc36d9c789732d20e305
9138f73ed31973bab8e005465ee04bec1cfb2a56e0f6eb9f316d964127c9877e


0x01 || 9138f73ed31973bab8e005465ee04bec1cfb2a56e0f6eb9f316d964127c9877e || 93663301e472c817961f75499d26844b246665d275538e5a24e16b065cda7afe
874589aee5388025c078a59e1f6e66ac65d185cdc766620860ba0680cfbd26bc


0x01 || c2ed30483aafe58267a6740c2fb76408eb6366ceb48c9f0509c0f2c04b101f05 || 874589aee5388025c078a59e1f6e66ac65d185cdc766620860ba0680cfbd26bc
0926480781b8d7bc2790d7e63972846bf98b79b47a1968a47f9c0992f319da44


0x01 || b593f43b0ca6048ca4e1564817b0381f25f66d6c6b205673d6e6bc1c582a02bd || 0926480781b8d7bc2790d7e63972846bf98b79b47a1968a47f9c0992f319da44
1af9b80f354a6c824fe9893f7278ce192dea39f4cb800707681947408b9f8c60


0x01 || b8f5ef19b43c82a6deee74a4b36c41a4b7f030d46f9862913ae5ab74177d7a34 || 1af9b80f354a6c824fe9893f7278ce192dea39f4cb800707681947408b9f8c60
b76bf4a5ee7dbf09372243c01c86d620fc3581e8c4b5659ce549b397ec9608bc


0x01 || 0b4e9c3dce66ce4f393b3b3ae5f141a0d20902538d5fc99bd030348672b69b81 || b76bf4a5ee7dbf09372243c01c86d620fc3581e8c4b5659ce549b397ec9608bc
31fcf995aaadf0cc7ce249b3ae06fb73653648c6279dd8401a75e59538a85718


0x01 || 9a991dc567e6dd4fa21d6235df691252ca81eab5e1301c849c8aa2151363e6a7 || 31fcf995aaadf0cc7ce249b3ae06fb73653648c6279dd8401a75e59538a85718
36a05fd84e2e4f497a41a3e6e633b59a49873d731e1d4e2e5c91ecb5ece28091


0x01 || 732af72ebcc0c8a16dbdba657c7b755d2097691d08882f308dbc5b133372c170 || 36a05fd84e2e4f497a41a3e6e633b59a49873d731e1d4e2e5c91ecb5ece28091
5219998b60322aa7e1aa3c02cf15bedc67d89876dfc46f1b89937c8e496aa947


0x01 || 2747468d0ed5e5b1138bba7b7968367a9842437d9004b3166391f115cb867d1e || 5219998b60322aa7e1aa3c02cf15bedc67d89876dfc46f1b89937c8e496aa947
b054e63c475d8751b14b51a54255918e4ae0aa26e0ee60c9c7ee3e333396c4ad


verification successful! got: b054e63c475d8751b14b51a54255918e4ae0aa26e0ee60c9c7ee3e333396c4ad want: b054e63c475d8751b14b51a54255918e4ae0aa26e0ee60c9c7ee3e333396c4ad

This is the same flow that the Rekor CLI will step through.

$ rekor-cli verify --uuid 24296fb24b8ad77a7e53fd5d089af142b7909598d214e13ca76001cc575fddaad3210adbee86363e

Current Root Hash: f0e0de7e6b03385bc086c46703b7a2abbbd16ae10fc28b3f480125b1898536fb
Entry Hash: 24296fb24b8ad77a7e53fd5d089af142b7909598d214e13ca76001cc575fddaad3210adbee86363e
Entry Index: 7155742
Current Tree Size: 2995354
Checkpoint:
rekor.sigstore.dev - 2605736670972794746
2995354
8ODefmsDOFvAhsRnA7eiq7vRauEPwos/SAElsYmFNvs=
Timestamp: 1668548431487913062

— rekor.sigstore.dev wNI9ajBEAiB/Lcxmn82//9QIwqVPbVSgzEAfACmAnZNLD9RuIH9QiAIgLToW3Bd8Y26Wwz3JuuZBsC1/IhUExSbu1NET/nzoajc=


Inclusion Proof:
SHA256(0x01 | 24296fb24b8ad77a7e53fd5d089af142b7909598d214e13ca76001cc575fddaad3210adbee86363e | 2fb6c414fe3710a0ca820e20a352d827e920ef94de1a62cf032f0b508583d343) =
        768fdb04aac9d523a34adbfeecd3268f920ff250ffc809867bae28bd10eb5f15
…

If you have any question you can reach me @eddiezane!

Stay spooky…

Special thanks to Appu Goundan and Hayden Blauzvern.

Last updated: 2023-03-29 08:49