How to hash and sign Ledger requests

Each ledger operation needs to be signed by a private key that has permissions to perform that operation. We need to have a deterministic way of generating hashes and signatures in order to be able to verify them on other devices.

Hashing

The first part of this process is making sure to generate stable string representations of our content in order to always get the same hash for the same input. We are using JSON as our wire transport, so it makes sense to use JSON for hashing as well.

JSON is not a deterministic format, so JSON serializers in various programming languages don’t necessarily provide the same output for the same input data. To resolve this, ledger uses a JSON serialization standard described in RFC 8785. The RFC document describes the rules for serialization, there are reference implementations for several popular programming languages linked in the specification as well.

Many existing serializers should be compatible with the rules described in the RFC, the most important rule that usually needs to be implemented manually is the ordering of JSON object properties. Each JSON object in the output needs to have all properties alphabetically ordered.

Example payload serialization in nodeJS:

import stringify from 'safe-stable-stringify'
 
export function serializeData(data: any): string {
  return stringify(data)
}

After we have a canonically serialized data, now we can hash it. For hashing we will use SHA-256 hashing algorithm, and we will calculate our hash by performing the following steps:

1. Serialize the input data using RFC 8785 compatible serializer
2. Hash the serialized data using SHA-256 as a hex encoded string

Example implementation in nodeJs:

import crypto from 'crypto'
 
const HASHING_ALGORITHM = 'sha256'
 
export function createHash(data: any): string {
  const serializedData = serializeData(data)
  return crypto
    .createHash(HASHING_ALGORITHM)
    .update(serializedData)
    .digest('hex')
}

Signature digests

Ledger also supports attaching additional data when signing objects. This can be done by including data in the custom property of the signature object.

Signing only the primary payload hash like the one we generated in the previous chapter wouldn’t allow us to ensure the integrity of this additional data included in signatures. This is why an additional signature digest is calculated in these situations.

To include additional data in the hash we are using double hashing algorithm. Double hashing has some other benefits besides the ability to extend the hash with additional data. Most important among those is that it prevents certain cryptographic attacks. That is why signature digest is used for all signatures, regardless if they include custom data.

Steps to calculate a signature digest are as follows:

1. Create a primary payload hash following the steps from previous chapter
2. Serialize the additional data (custom) that is added in the signature using the serialization algorithm from the previous chapter, use an empty string if custom doesn’t exists
3. Perform another SHA-256 hash by concatenating the hash from step 1 with the serialized custom data of the signature: sha256(dataHash + serializedCustomData) and return this hash as a hex encoded string
4. Use the hash from step 3 as the signature digest

Here is an example implementation in nodeJs:

export function createSignatureDigest(
  dataHash: string,
  signatureCustom?: Record<string, any>,
): string {
  // Serialize the custom data, if it exists
  const serializedCustomData = signatureCustom ?
    serializeData(signatureCustom) :
    ''
 
  // Create a hash by concatenating the data hash
  // with serialized custom data
  return crypto
    .createHash(HASHING_ALGORITHM)
    .update(dataHash + serializedCustomData)
    .digest('hex')
}

Signing

We can now sign ledger requests by signing the digest we calculated previously with our private key using ed25519 compatible implementation. In nodeJS we can perform this by using the standard node crypto package:

import crypto from 'crypto'
 
const digestBuffer = Buffer.from(digest, 'hex')
 
// This assumes you have a private key in DER format,
// it may be necessary to convert keys if that is not
// the case
const key = crypto.createPrivateKey({
  format: 'der',
  type: 'pkcs8',
  key: keyDer,
})
 
// The first argument must be undefined for ed25519, it defines a digest
// algorithm, but ed25519 makes a sha512 digest as part of the algorithm.
// Signing and verification with ed25119 doesn't work correctly if that
// argument is provided. Also, most crypto examples use createSign(algorithm)
// to first create a Sign class and use that for signing, but that way also
// doesn't work with ed25519.
// see: https://github.com/mscdex/io.js/commit/7d0e50dcfef98ca56715adf74678bcaf4aa08796
const result = crypto.sign(undefined, digestBuffer, key).toString('base64')