Skip to content

Conversation

@nuke-web3
Copy link
Member

@nuke-web3 nuke-web3 commented Feb 11, 2025

Fixes #19

Adds a build container with a few caching stages in it to produce a release artifact of eq_service to run.

Testing using Ubuntu, but podman 4 required for BuildKit cache:

just docker-build
just docker-run

@nuke-web3 nuke-web3 self-assigned this Feb 11, 2025
@nuke-web3
Copy link
Member Author

nuke-web3 commented Feb 12, 2025

Service builds and runs with 0c3be20 but it hangs on trying to request a ZK proof remotely, my guess is #46 related as there might be a networking issue in the container? need to diagnose.

confirmed from full RUST_LOG=debug

[2025-02-12T22:04:56Z WARN  sp1_sdk::network::retry] Transient transport error when getting program: transport error, retrying...
[2025-02-12T22:04:56Z ERROR eq_service::internal::inclusion] Job { height: 4409088, namespace: "XSUTEfJbE6VJ4A==", commitment: "DYoAZpU7FrviV7Ui/AjQv0BpxCwexPWaOW/hQVpEl/s=" } failed progressing DataAvailable: Unhandled Error: transport error PLEASE REPORT

so sp1 client retries for about 30 seconds, but eventually times out and the service does the right thing, and fails such that the next gRPC request fires the client again.

here is the log
https://github.com/succinctlabs/sp1/blob/6a921fb256ce949c57bae7c83c49f8aaa6c35f3c/crates/sdk/src/network/retry.rs#L85-L100

@nuke-web3
Copy link
Member Author

lmao it's due to the FROM debian:bookworm-slim NOT including curl and doesn't fail if it's missing - discovered by accident as I needed to install curl. Will need to file a bug upstream somewhere 🤕

Details apt install curl -y Reading package lists... Done Building dependency tree... Done Reading state information... Done The following additional packages will be installed: ca-certificates libbrotli1 libcurl4 libldap-2.5-0 libldap-common libpsl5 librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db libssh2-1 openssl publicsuffix Suggested packages: libsasl2-modules-gssapi-mit | libsasl2-modules-gssapi-heimdal libsasl2-modules-ldap libsasl2-modules-otp libsasl2-modules-sql The following NEW packages will be installed: ca-certificates curl libbrotli1 libcurl4 libldap-2.5-0 libldap-common libpsl5 librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db libssh2-1 openssl publicsuffix 0 upgraded, 14 newly installed, 0 to remove and 1 not upgraded. Need to get 3345 kB of archives. After this operation, 7067 kB of additional disk space will be used. Get:1 http://deb.debian.org/debian bookworm/main amd64 openssl amd64 3.0.15-1~deb12u1 [1427 kB] Get:2 http://deb.debian.org/debian bookworm/main amd64 ca-certificates all 20230311 [153 kB] Get:3 http://deb.debian.org/debian bookworm/main amd64 libbrotli1 amd64 1.0.9-2+b6 [275 kB] Get:4 http://deb.debian.org/debian bookworm/main amd64 libsasl2-modules-db amd64 2.1.28+dfsg-10 [20.3 kB] Get:5 http://deb.debian.org/debian bookworm/main amd64 libsasl2-2 amd64 2.1.28+dfsg-10 [59.7 kB] Get:6 http://deb.debian.org/debian bookworm/main amd64 libldap-2.5-0 amd64 2.5.13+dfsg-5 [183 kB] Get:7 http://deb.debian.org/debian bookworm/main amd64 libpsl5 amd64 0.21.2-1 [58.7 kB] Get:8 http://deb.debian.org/debian bookworm/main amd64 librtmp1 amd64 2.4+20151223.gitfa8646d.1-2+b2 [60.8 kB] Get:9 http://deb.debian.org/debian bookworm/main amd64 libssh2-1 amd64 1.10.0-3+b1 [179 kB] Get:10 http://deb.debian.org/debian bookworm/main amd64 libcurl4 amd64 7.88.1-10+deb12u8 [390 kB] Get:11 http://deb.debian.org/debian bookworm/main amd64 curl amd64 7.88.1-10+deb12u8 [315 kB] Get:12 http://deb.debian.org/debian bookworm/main amd64 libldap-common all 2.5.13+dfsg-5 [29.3 kB] Get:13 http://deb.debian.org/debian bookworm/main amd64 libsasl2-modules amd64 2.1.28+dfsg-10 [66.6 kB] Get:14 http://deb.debian.org/debian bookworm/main amd64 publicsuffix all 20230209.2326-1 [126 kB] Fetched 3345 kB in 0s (8279 kB/s) debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package openssl. (Reading database ... 6275 files and directories currently installed.) Preparing to unpack .../00-openssl_3.0.15-1~deb12u1_amd64.deb ... Unpacking openssl (3.0.15-1~deb12u1) ... Selecting previously unselected package ca-certificates. Preparing to unpack .../01-ca-certificates_20230311_all.deb ... Unpacking ca-certificates (20230311) ... Selecting previously unselected package libbrotli1:amd64. Preparing to unpack .../02-libbrotli1_1.0.9-2+b6_amd64.deb ... Unpacking libbrotli1:amd64 (1.0.9-2+b6) ... Selecting previously unselected package libsasl2-modules-db:amd64. Preparing to unpack .../03-libsasl2-modules-db_2.1.28+dfsg-10_amd64.deb ... Unpacking libsasl2-modules-db:amd64 (2.1.28+dfsg-10) ... Selecting previously unselected package libsasl2-2:amd64. Preparing to unpack .../04-libsasl2-2_2.1.28+dfsg-10_amd64.deb ... Unpacking libsasl2-2:amd64 (2.1.28+dfsg-10) ... Selecting previously unselected package libldap-2.5-0:amd64. Preparing to unpack .../05-libldap-2.5-0_2.5.13+dfsg-5_amd64.deb ... Unpacking libldap-2.5-0:amd64 (2.5.13+dfsg-5) ... Selecting previously unselected package libpsl5:amd64. Preparing to unpack .../06-libpsl5_0.21.2-1_amd64.deb ... Unpacking libpsl5:amd64 (0.21.2-1) ... Selecting previously unselected package librtmp1:amd64. Preparing to unpack .../07-librtmp1_2.4+20151223.gitfa8646d.1-2+b2_amd64.deb ... Unpacking librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2+b2) ... Selecting previously unselected package libssh2-1:amd64. Preparing to unpack .../08-libssh2-1_1.10.0-3+b1_amd64.deb ... Unpacking libssh2-1:amd64 (1.10.0-3+b1) ... Selecting previously unselected package libcurl4:amd64. Preparing to unpack .../09-libcurl4_7.88.1-10+deb12u8_amd64.deb ... Unpacking libcurl4:amd64 (7.88.1-10+deb12u8) ... Selecting previously unselected package curl. Preparing to unpack .../10-curl_7.88.1-10+deb12u8_amd64.deb ... Unpacking curl (7.88.1-10+deb12u8) ... Selecting previously unselected package libldap-common. Preparing to unpack .../11-libldap-common_2.5.13+dfsg-5_all.deb ... Unpacking libldap-common (2.5.13+dfsg-5) ... Selecting previously unselected package libsasl2-modules:amd64. Preparing to unpack .../12-libsasl2-modules_2.1.28+dfsg-10_amd64.deb ... Unpacking libsasl2-modules:amd64 (2.1.28+dfsg-10) ... Selecting previously unselected package publicsuffix. Preparing to unpack .../13-publicsuffix_20230209.2326-1_all.deb ... Unpacking publicsuffix (20230209.2326-1) ... Setting up libpsl5:amd64 (0.21.2-1) ... Setting up libbrotli1:amd64 (1.0.9-2+b6) ... Setting up libsasl2-modules:amd64 (2.1.28+dfsg-10) ... Setting up libldap-common (2.5.13+dfsg-5) ... Setting up libsasl2-modules-db:amd64 (2.1.28+dfsg-10) ... Setting up librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2+b2) ... Setting up libsasl2-2:amd64 (2.1.28+dfsg-10) ... Setting up libssh2-1:amd64 (1.10.0-3+b1) ... Setting up openssl (3.0.15-1~deb12u1) ... Setting up publicsuffix (20230209.2326-1) ... Setting up libldap-2.5-0:amd64 (2.5.13+dfsg-5) ... Setting up ca-certificates (20230311) ... debconf: unable to initialize frontend: Dialog debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 78.) debconf: falling back to frontend: Readline debconf: unable to initialize frontend: Readline debconf: (Can't locate Term/ReadLine.pm in @inc (you may need to install the Term::ReadLine module) (@inc contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.36.0 /usr/local/share/perl/5.36.0 /usr/lib/x86_64-linux-gnu/perl5/5.36 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl-base /usr/lib/x86_64-linux-gnu/perl/5.36 /usr/share/perl/5.36 /usr/local/lib/site_perl) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.) debconf: falling back to frontend: Teletype Updating certificates in /etc/ssl/certs... 140 added, 0 removed; done. Setting up libcurl4:amd64 (7.88.1-10+deb12u8) ... Setting up curl (7.88.1-10+deb12u8) ... Processing triggers for libc-bin (2.36-9+deb12u9) ... Processing triggers for ca-certificates (20230311) ... Updating certificates in /etc/ssl/certs... 0 added, 0 removed; done. Running hooks in /etc/ca-certificates/update.d... done.

@nuke-web3
Copy link
Member Author

ChatGPT suggests we could package a bit slimmer even, if we want to:

gcr.io/distroless/base-debian12 (Ultra-Minimal)

If you need a smaller image but want glibc, use Distroless:

FROM gcr.io/distroless/base-debian12

COPY --from=debian:bookworm-slim /usr/lib/x86_64-linux-gnu/libcurl.so.4 /usr/lib/x86_64-linux-gnu/libcurl.so.4

This pulls libcurl.so.4 from Debian and adds it to a very minimal base image.
No package manager included—only necessary libraries.

@nuke-web3 nuke-web3 marked this pull request as ready for review February 12, 2025 23:08
@nuke-web3 nuke-web3 requested review from Bidon15 and S1nus February 12, 2025 23:08
Comment on lines +9 to +12
# DO NOT wrap variables in quotes (THING=stuff is correct, not THING="stuff")
# Otherwise passing them to docker/podman may fail.
#
# ALL VARIABLES need to be \n terminated, no whitespace trimming happens
Copy link
Member Author

@nuke-web3 nuke-web3 Feb 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sad but true, docker forces more restrictive stufff here than in normal shell context.

@smuu smuu self-requested a review February 17, 2025 12:56
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nuke-web3 nuke-web3 changed the title Add Dockerfile Add Dockerfile with CI Build & Publish Workflow Feb 17, 2025
@nuke-web3 nuke-web3 requested a review from smuu February 18, 2025 10:22
Copy link

@smuu smuu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see if the docker build pipeline works 😎

@nuke-web3 nuke-web3 merged commit 9af9b86 into main Feb 18, 2025
@nuke-web3 nuke-web3 deleted the n/dockerfile branch February 18, 2025 16:12
@nuke-web3
Copy link
Member Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Provide Docker support

4 participants