Support reproducible builds using SOURCE_DATE_EPOCH

[nicholasdille]

Description

Right now, builds are not reproducible. An important step towards that goal would be supporting timestamps as per reproducible-builds.org and the Docker blog.

I propose a new field called source-date-epoch with the following behaviour:

• If unset, builds are performed as before for backwards compatibility
• If set, the env var SOURCE_DATE_EPOCH is set to the specified value and is passed on to docker

[crazy-max]

It should already be possible using the env property:

      -
        name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: user/app:latest
        env:
          SOURCE_DATE_EPOCH: 0

@dvdksn Maybe we could have a new section in https://docs.docker.com/build/ci/github-actions/ about reproducible builds?

[dvdksn]

Yeah sounds like a good addition. Noted

[huxuan]

Repost from #1043

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug, and:

• The documentation does not mention anything about my problem
• There are no open or closed issues that are related to my problem

Description

The logs show that the sha256 for all layers for the registry cache are the same except for config.

Expected behaviour

The sha256 for the registry cache config should be the same.

Actual behaviour

• The sha256 for the registry cache config are mismatched which leads to untaged packages.

Repository URL

https://github.com/huxuan/ss-python

Workflow run URL

/actions/runs/7709174339/job/21009975797

YAML workflow

container-publish:
    needs: release-publish
    permissions:
      contents: read
      packages: write
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          password: ${{ secrets.GITHUB_TOKEN }}
          registry: ghcr.io
          username: ${{ github.actor }}
      - env:
          SOURCE_DATE_EPOCH: 0
        uses: docker/build-push-action@v5
        with:
          build-args: |
            PYTHON_VERSION=${{ matrix.python-version }}
          cache-from: |
            type=registry,ref=ghcr.io/${{ github.repository }}:build-cache-dev-py${{ matrix.python-version }}
          cache-to: type=registry,ref=ghcr.io/${{ github.repository }}:build-cache-dev-py${{ matrix.python-version }},mode=max
          context: .
          file: .devcontainer/prebuild/.devcontainer/Dockerfile
          provenance: false
          push: true
          tags: |
            ghcr.io/${{ github.repository }}:dev-py${{ matrix.python-version }}
            ghcr.io/${{ github.repository }}:dev-py${{ matrix.python-version }}-${{ github.ref_name }}
          target: dev
      - env:
          SOURCE_DATE_EPOCH: 0
        uses: docker/build-push-action@v5
        with:
          build-args: |
            PYTHON_VERSION=${{ matrix.python-version }}
            PDM_BUILD_SCM_VERSION=${{ github.ref_name }}
          cache-from: |
            type=registry,ref=ghcr.io/${{ github.repository }}:build-cache-dev-py${{ matrix.python-version }}
            type=registry,ref=ghcr.io/${{ github.repository }}:build-cache-prod-py${{ matrix.python-version }}
          cache-to: type=registry,ref=ghcr.io/${{ github.repository }}:build-cache-prod-py${{ matrix.python-version }},mode=max
          context: .
          file: .devcontainer/prebuild/.devcontainer/Dockerfile
          provenance: false
          push: true
          tags: |
            ghcr.io/${{ github.repository }}:prod-py${{ matrix.python-version }}
            ghcr.io/${{ github.repository }}:prod-py${{ matrix.python-version }}-${{ github.ref_name }}
          target: prod
    strategy:
      matrix:
        python-version:
          - '3.8'
          - '3.9'
          - '3.10'
          - '3.11'
          - '3.12'

Workflow logs

Attempt 1: https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009847395

#22 exporting cache to registry
#22 preparing build cache for export
#22 writing layer sha256:0a329b671abfc277cf19a2670d019384c564e146c9c5ed1bb97d173[377](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009847395#step:6:382)497d6e
#22 writing layer sha256:0a329b671abfc277cf19a2670d019384c564e146c9c5ed1bb97d173377497d6e 0.2s done
#22 writing layer sha256:1b13d4e1a46e5e969702ec92b7c787c1b6891bff7c21ad[378](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009847395#step:6:383)ff6dbc9e751d5d4
#22 writing layer sha256:1b13d4e1a46e5e969702ec92b7c787c1b6891bff7c21ad378ff6dbc9e751d5d4 0.1s done
#22 writing layer sha256:1c74526957fc2157e8b0989072dc99b9582b398c12d1dcd40270fd76231bab0c
#22 writing layer sha256:1c74526957fc2157e8b0989072dc99b9582b398c12d1dcd40270fd76231bab0c 0.1s done
#22 writing layer sha256:20f18e486fc01f87b6b311be20ae9db9cb82df60c14ad1783b86d5fa2ba623f8
#22 writing layer sha256:20f18e486fc01f87b6b311be20ae9db9cb82df60c14ad1783b86d5fa2ba623f8 0.2s done
#22 writing layer sha256:26916576c92c435f7441cb7490dd537040d28b42ab2719ea02ba909f0985a57c
#22 writing layer sha256:26916576c92c435f7441cb7490dd537040d28b42ab2719ea02ba909f0985a57c 0.2s done
#22 writing layer sha256:2878b437a07192a6cf2b2075786d0a7922b16f1cee9dad8a5f305deef52e6253
#22 writing layer sha256:2878b437a07192a6cf2b2075786d0a7922b16f1cee9dad8a5f305deef52e6253 1.3s done
#22 writing layer sha256:2f44b7a888fa005d07c031d3cfad2a1c0344207def2ab9dbb97712425ff812c1
#22 writing layer sha256:2f44b7a888fa005d07c031d3cfad2a1c0344207def2ab9dbb97712425ff812c1 0.1s done
#22 writing layer sha256:30d85599795460b2d9d24c6b87c53ec60555b601705cc83bea31632240500980
#22 writing layer sha256:30d85599795460b2d9d24c6b87c53ec60555b601705cc83bea31632240500980 0.1s done
#22 writing layer sha256:382cd2159c71b198be5533c09c741cdfecf7dc5618feb5637bbe5f036f73d89f
#22 writing layer sha256:382cd2159c71b198be5533c09c741cdfecf7dc5618feb5637bbe5f036f73d89f 0.1s done
#22 writing layer sha256:396f0fee52796a6a98fc294805b23c509beb8cd0b4556e5568b4298a0be6e3c8
#22 writing layer sha256:396f0fee52796a6a98fc294805b23c509beb8cd0b4556e5568b4298a0be6e3c8 0.1s done
#22 writing layer sha256:3f00b3697662aa214d22bb76bbbf5fa4d80f7ba9ca9f9076a9440d53bf529b83
#22 writing layer sha256:3f00b3697662aa214d22bb76bbbf5fa4d80f7ba9ca9f9076a9440d53bf529b83 0.2s done
#22 writing layer sha256:4acda0384fa4980f4c24482858b8894f5711c36ca9269d50fbbd88bbf74de8a4
#22 writing layer sha256:4acda0384fa4980f4c24482858b8894f5711c36ca9269d50fbbd88bbf74de8a4 0.1s done
#22 writing layer sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
#22 writing layer sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 0.1s done
#22 writing layer sha256:69c6a95e6fa75b2a15dcc1d93e96f2b16e96be599a4c74574ab1c63126e25f97
#22 writing layer sha256:69c6a95e6fa75b2a15dcc1d93e96f2b16e96be599a4c74574ab1c63126e25f97 0.1s done
#22 writing layer sha256:75e2b45cbee50cea4b3ed4f4fe167ad5994622d77a54adde89adcfeefa3c229a
#22 writing layer sha256:75e2b45cbee50cea4b3ed4f4fe167ad5994622d77a54adde89adcfeefa3c229a 0.1s done
#22 writing layer sha256:76530ff9b004d5b0ad94cc95e54ad2749c0deb9d3d105d15b84049b006f9abed
#22 writing layer sha256:76530ff9b004d5b0ad94cc95e54ad2749c0deb9d3d105d15b84049b006f9abed 0.1s done
#22 writing layer sha256:76c111f84668456d1a37f894e2a1a6fc655cecf002d8b3f7934f4ffc14231f72
#22 writing layer sha256:76c111f84668456d1a37f894e2a1a6fc655cecf002d8b3f7934f4ffc14231f72 0.1s done
#22 writing layer sha256:8ea4555671ddb9b06b855c89880e2946201b5ced31dbf1ce8f5b872b55b224e7
#22 writing layer sha256:8ea4555671ddb9b06b855c89880e2946201b5ced31dbf1ce8f5b872b55b224e7 0.1s done
#22 writing layer sha256:ad5739181616b815fae7edc6bba689496674acbcf44e48a57fc7cc13a[379](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009847395#step:6:384)b3a2
#22 writing layer sha256:ad57[391](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009847395#step:6:396)81616b815fae7edc6bba689496674acbcf44e48a57fc7cc13a379b3a2 0.1s done
#22 writing layer sha256:b3ba329ce47347173b038e63b8c9a78f7cf20324c2377dc3a0904837cfbe2e23
#22 writing layer sha256:b3ba329ce47347173b038e63b8c9a78f7cf20324c2377dc3a0904837cfbe2e23 0.1s done
#22 writing layer sha256:c12af4d99b224c7583539128a04c9[395](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009847395#step:6:400)7b1908684e50013d8b58a8e2988d7b9f
#22 writing layer sha256:c12af4d99b224c7583539128a04c93957b1908684e50013d8b58a8e2988d7b9f 0.1s done
#22 writing layer sha256:f4cb18646a15052fbff9e6a7f2af27b73ce3033e9e973709fedf06058817f092
#22 writing layer sha256:f4cb18646a15052fbff9e6a7f2af27b73ce3033e9e973709fedf06058817f092 0.1s done
#22 writing layer sha256:fa8b5ed51b617f793487eb8b5c0d947553c6d649eabd5615e35795e71c70feb2
#22 writing layer sha256:fa8b5ed51b617f793487eb8b5c0d947553c6d649eabd5615e35795e71c70feb2 0.1s done
#22 writing layer sha256:ffaf3b259c1baad7a193f840f1f0506f9debd25f0ba8d9de0033a61553d0f53d
#22 writing layer sha256:ffaf3b259c1baad7a193f840f1f0506f9debd25f0ba8d9de0033a61553d0f53d 0.1s done
#22 writing config sha256:20bd3b4cbe80bae9a81f88dade1f0f6a84ff8ae28a22ceaeec893b4ad92d21a6
#22 writing config sha256:20bd3b4cbe80bae9a81f88dade1f0f6a84ff8ae28a22ceaeec893b4ad92d21a6 0.9s done
#22 writing cache manifest sha256:11dd2cfbbd2816313b0fe862ee8c8cead80984e3d1a8e7b274e6197877ae815b
#22 preparing build cache for export 6.8s done
#22 writing cache manifest sha256:11dd2cfbbd2816313b0fe862ee8c8cead80984e3d1a8e7b274e6197877ae815b 0.7s done
#22 DONE 6.8s

Attempt 2: https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009975797

#22 exporting cache to registry
#22 preparing build cache for export
#22 writing layer sha256:0a329b671abfc277cf19a2670d019384c564e146c9c5ed1bb97d173377497d6e
#22 writing layer sha256:0a329b671abfc277cf19a2670d019384c564e146c9c5ed1bb97d173377497d6e 0.1s done
#22 writing layer sha256:1b13d4e1a46e5e969702ec92b7c787c1b6891bff7c21ad378ff6dbc9e751d5d4 0.1s done
#22 writing layer sha256:1c74526957fc2157e8b0989072dc99b9582b398c12d1dcd40270fd76231bab0c
#22 writing layer sha256:1c74526957fc2157e8b0989072dc99b9582b398c12d1dcd40270fd76231bab0c 0.1s done
#22 writing layer sha256:20f18e486fc01f87b6b311be20ae9db9cb82df60c14ad1783b86d5fa2ba623f8 0.1s done
#22 writing layer sha256:26916576c92c435f7441cb7490dd537040d28b42ab2719ea02ba909f0985a57c
#22 writing layer sha256:26916576c92c435f7441cb7490dd537040d28b42ab2719ea02ba909f0985a57c 0.1s done
#22 writing layer sha256:2878b437a07192a6cf2b2075786d0a7922b16f1cee9dad8a5f305deef52e6253 0.1s done
#22 writing layer sha256:2f44b7a888fa005d07c031d3cfad2a1c0344207def2ab9dbb97712425ff812c1
#22 writing layer sha256:2f44b7a888fa005d07c031d3cfad2a1c0344207def2ab9dbb97712425ff812c1 0.1s done
#22 writing layer sha256:30d85599795460b2d9d24c6b87c53ec60555b601705cc83bea31632240500980 0.0s done
#22 writing layer sha256:382cd2159c71b198be5533c09c741cdfecf7dc5618feb5637bbe5f036f73d89f
#22 writing layer sha256:382cd2159c71b198be5533c09c741cdfecf7dc5618feb5637bbe5f036f73d89f 0.1s done
#22 writing layer sha256:396f0fee52796a6a98fc294805b23c509beb8cd0b4556e5568b4298a0be6e3c8 0.1s done
#22 writing layer sha256:3f00b3697662aa[214](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009975797#step:6:219)d22bb76bbbf5fa4d80f7ba9ca9f9076a9440d53bf529b83
#22 writing layer sha256:3f00b3697662aa214d22bb76bbbf5fa4d80f7ba9ca9f9076a9440d53bf529b83 0.1s done
#22 writing layer sha256:4acda0384fa4980f4c24482858b8894f5711c36ca9269d50fbbd88bbf74de8a4 0.0s done
#22 writing layer sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
#22 writing layer sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 0.1s done
#22 writing layer sha256:69c6a95e6fa75b2a15dcc1d93e96f2b16e96be599a4c74574ab1c63126e25f97 0.1s done
#22 writing layer sha256:75e2b45cbee50cea4b3ed4f4fe167ad5994622d77a54adde89adcfeefa3c229a
#22 writing layer sha256:75e2b45cbee50cea4b3ed4f4fe167ad5994622d77a54adde89adcfeefa3c229a 0.1s done
#22 writing layer sha256:76530ff9b004d5b0ad94cc95e54ad2749c0deb9d3d105d15b84049b006f9abed 0.1s done
#22 writing layer sha256:76c111f84668456d1a37f894e2a1a6fc655cecf002d8b3f7934f4ffc14231f72
#22 writing layer sha256:76c111f84668456d1a37f894e2a1a6fc655cecf002d8b3f7934f4ffc14231f72 0.1s done
#22 writing layer sha256:8ea4555671ddb9b06b855c89880e2946201b5ced31dbf1ce8f5b872b55b224e7 0.1s done
#22 writing layer sha256:ad5739181616b815fae7edc6bba689496674acbcf44e48a57fc7cc13a379b3a2
#22 writing layer sha256:ad5739181616b815fae7edc6bba689496674acbcf44e48a57fc7cc13a379b3a2 0.1s done
#22 writing layer sha256:b3ba329ce47347173b038e63b8c9a78f7cf20324c2377dc3a0904837cfbe2e23 0.1s done
#22 writing layer sha256:c12af4d99b224c7583539128a04c93957b1908684e50013d8b58a8e2988d7b9f
#22 writing layer sha256:c12af4d99b224c7583539128a04c93957b1908684e50013d8b58a8e2988d7b9f 0.1s done
#22 writing layer sha256:f4cb18646a15052fbff9e6a7f2af27b73ce3033e9e973709fedf06058817f092 0.1s done
#22 writing layer sha256:fa8b5ed51b617f793487eb8b5c0d947553c6d649eabd5615e35795e71c70feb2
#22 writing layer sha256:fa8b5ed51b617f793487eb8b5c0d947553c6d649eabd5615e35795e71c70feb2 0.1s done
#22 writing layer sha256:ffaf3b259c1baad7a193f840f1f0506f9debd25f0ba8d9de0033a61553d0f53d 0.1s done
#22 writing config sha256:336be0cc3b08f2c94c32c45c4234ae9075cb071431e19c4787dc7e51e1a5b54d
#22 writing config sha256:336be0cc3b08f2c94c32c45c4234ae9075cb071431e19c4787dc7e51e1a5b54d 0.8s done
#22 writing cache manifest sha256:fdd22d7352e00b75d70666789bd229fa24f5afebb9a[220](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009975797#step:6:225)def5db47f648d3f50c
#22 preparing build cache for export 3.6s done
#22 writing cache manifest sha256:fdd22d7352e00b75d70666789bd[229](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009975797#step:6:234)fa24f5afebb9a220def5db47f648d3f50c 0.9s done
#22 DONE 3.6s

BuildKit logs

No response

Additional info

1. To make everything reproducible as much as possible, both

   env:
     SOURCE_DATE_EPOCH: 0

   and

   with:
     provenance: false

   are set according to Support reproducible builds using SOURCE_DATE_EPOCH #994 and Untagged images produced for multi platform builds #894
2. The sha256 of layers and config for single-stage are the same both for the resulting image and registry cache.
3. The sha256 of layers and config for multi-stage are the same only for the resulting image.
4. The cache overall works as expected, the only side effects are untaged packages for the registry caches.