Skip to content

Conversation

@jameslamb
Copy link
Member

@jameslamb jameslamb commented Dec 18, 2025

Contributes to #832

Removes reliance on https://hub.docker.com/r/rapidsai/ci-conda in this repo. It was only being used in 2 places, both of which could be run in simpler environments:

  • in a compute-matrix job that just needs bash, jq and yq --> regular old ubuntu-latest GitHub Actions VM is fine
  • in a part of a multi-stage build that just needs bash, git, pip, and a Python interpreter --> python:{version} container is fine

Notes for Reviewers

This was the easy part

To be clear, this project still heavily relies on rapidsai/miniforge-cuda (the base image for rapidsai/ci-conda. So there's more work to do for #832.

But this at least means we no longer need to care about anything in these lines breaking the things in this repo 😁 :

https://github.com/rapidsai/ci-imgs/blob/55b24734b7d594e768599771dcae2e718af52f9a/ci-conda.Dockerfile#L183-L313

compute-matrix:
runs-on: ubuntu-latest
container:
image: rapidsai/ci-conda:latest
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just some string manipulation in bash + jq + yq... I don't think we need to pay the cost of pulling a container image, and I think we can pretty safely just accept whatever jq / yq version GitHub is shipping in its images (ref: https://github.com/actions/runner-images/tree/main/images/ubuntu).

*.tar.gz
*.tgz
*.whl
*.zip
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Saw .ruff_cache not .gitignore'd and that led me to updating this file which led me to adding a few more things just for extra safety.

--prefer-binary \
--upgrade \
'conda-merge==0.3.*' \
'rapids-dependency-file-generator==1.20.*'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pinning these to the version ranges we're currently getting just for a bit of extra safety (since I'm touching the line anyway).

@jameslamb
Copy link
Member Author

I think the test jobs are failing because we're pulling in 25.12 conda packages but using the 26.02 development branches of the repos.

I think it's probably this 25.12.00b tag being after 26.02.00a:

git checkout main
git pull upstream main
git describe --tags --abbrev=0
# v25.12.00b

Deleting it should fix this, but I asked RAPIDS Ops offline to confirm before I do that.

@jameslamb
Copy link
Member Author

good news: removing that 25.12.00b tag worked 🎉

bad news: the builds are failing because some libraries were missed in rapidsai/build-planning#233

#13 16.23        │  │  ├─ cuxfilter 26.02.00a5 would require
#13 16.23        │  │  │  └─ python_abi =3.12 *_cp312, which requires
#13 16.23        │  │  │     └─ python =3.12 *_cpython, which conflicts with any installable versions previously reported;
#13 16.23        │  │  └─ cuxfilter 26.02.00a5 would require
#13 16.23        │  │     └─ python_abi =3.13 *_cp313 but there are no viable options
#13 16.23        │  │        ├─ python_abi 3.13 would require
#13 16.23        │  │        │  └─ python =3.13 *_cpython, which conflicts with any installable versions previously reported;
#13 16.23        │  │        └─ python_abi 3.13 would require
#13 16.23        │  │           └─ python =3.13 *_cp313, which conflicts with any installable versions previously reported;
#13 16.23        │  └─ dask-cuda =26.2 *, which requires
#13 16.23        │     └─ numba-cuda >=0.22.1,<0.23.0 * but there are no viable options
#13 16.23        │        ├─ numba-cuda 0.22.1 conflicts with any installable versions previously reported;
#13 16.23        │        ├─ numba-cuda 0.22.1 would require
#13 16.23        │        │  └─ python_abi =3.11 *_cp311 but there are no viable options
#13 16.23        │        │     ├─ python_abi 3.11, which cannot be installed (as previously explained);
#13 16.23        │        │     └─ python_abi 3.11 conflicts with any installable versions previously reported;
#13 16.23        │        ├─ numba-cuda 0.22.1 would require
#13 16.23        │        │  └─ python_abi =3.12 *_cp312, which cannot be installed (as previously explained);
#13 16.23        │        └─ numba-cuda 0.22.1 would require
#13 16.23        │           └─ python_abi =3.13 *_cp313, which cannot be installed (as previously explained);
#13 16.23        └─ rapids 26.02.00a10 would require
#13 16.23           └─ cuda-version >=13,<14.0a0 *, which conflicts with any installable versions previously reported.

(build link)

I'll go update them and come back to this.

@jameslamb
Copy link
Member Author

jameslamb commented Dec 23, 2025

The tagging issue and cuxfilter issue have been resolved.

Next, CI's failing because cugraph hasn't had successful nightlies in a while and the older ones aren't usable alongside newer ones of other RAPIDS packages.

ImportError: /opt/conda/lib/python3.12/site-packages/cugraph/structure/../../../../libcugraph.so: undefined symbol: _ZN3rmm13device_bufferC1EPKvmNS_16cuda_stream_viewENS_6detail23cccl_async_resource_refIN4cuda2mr3__418basic_resource_refILNS8_10_AllocTypeE1EJNS8_17device_accessibleEEEEEE

(build link)

I think rapidsai/cugraph#5379 is trying to fix that. yes, fixed

@jameslamb jameslamb changed the title WIP: remove reliance on rapidsai/ci-conda remove reliance on rapidsai/ci-conda Dec 31, 2025
@jameslamb jameslamb marked this pull request as ready for review December 31, 2025 21:40
@jameslamb jameslamb requested a review from a team as a code owner December 31, 2025 21:40
@jameslamb jameslamb requested a review from msarahan December 31, 2025 21:40
Copy link
Contributor

@bdice bdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple cleanup suggestions.

ARG RAPIDS_BRANCH="main"

SHELL ["/bin/bash", "-euo", "pipefail", "-c"]
ARG YQ_VER=4.49.2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we make this use a versions.yaml like in ci-imgs? https://github.com/rapidsai/ci-imgs/blob/main/versions.yaml

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I can expand this PR to include that.

RUN <<EOF
apt-get update
apt-get install -y --no-install-recommends rsync
apt-get install \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally we keep the flags on the first line (it's not that long) and we also avoid indenting the \ characters. Example: https://github.com/rapidsai/ci-imgs/blob/55b24734b7d594e768599771dcae2e718af52f9a/ci-conda.Dockerfile#L208-L215

Let's follow that style here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, no problem. This style of indenting the \ over is something I've grown used to looking at lots of RAPIDS devcontainers code recently (like this), I kind of like it.

But don't feel strongly at all, I'll change this to match what we do in other Dockerfiles, no problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants