Skip to content

Add --deps-only flag to separate dependency fetching from source builds#1336

Open
sbryngelson wants to merge 4 commits intoMFlowCode:masterfrom
sbryngelson:nodeps
Open

Add --deps-only flag to separate dependency fetching from source builds#1336
sbryngelson wants to merge 4 commits intoMFlowCode:masterfrom
sbryngelson:nodeps

Conversation

@sbryngelson
Copy link
Member

Summary

  • Adds --deps-only CLI flag to ./mfc.sh build that only fetches and builds dependencies (FFTW, HDF5, etc.) without building MFC source targets
  • Already-configured dependencies are skipped during regular builds, preventing network access in the source build step
  • Frontier and Frontier AMD CI now fetch deps on login nodes (internet access) and build source on compute nodes (no network needed)

Test plan

  • Verify ./mfc.sh build --deps-only -j 8 builds only dependency targets
  • Verify regular ./mfc.sh build -j 8 skips already-configured deps
  • Verify Frontier CI: deps fetched on login node, source built + tested on compute node
  • Verify Frontier AMD CI: same pattern via symlinked build.sh
  • Verify Phoenix CI is unaffected (no login-node build step)
  • Verify bench.yml Frontier/Frontier AMD jobs work with updated build_script args

This allows CI to fetch and build dependencies (FFTW, HDF5, etc.) on
login nodes with internet access, then build MFC source code on compute
nodes that may have no network connectivity.

Key changes:
- New --deps-only CLI flag for ./mfc.sh build
- Already-configured dependencies are skipped entirely during regular
  builds, guaranteeing no network access in the source build step
- Frontier and Frontier AMD now follow the pattern: deps on login node,
  source build + test on compute node
@sbryngelson sbryngelson marked this pull request as ready for review March 25, 2026 14:55
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 25, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b82e2aac-df52-4afb-83cf-cd069bb747b9

📥 Commits

Reviewing files that changed from the base of the PR and between 90d4b6e and a22f206.

📒 Files selected for processing (7)
  • .github/workflows/bench.yml
  • .github/workflows/common/bench.sh
  • .github/workflows/common/test.sh
  • .github/workflows/frontier/build.sh
  • .github/workflows/test.yml
  • toolchain/mfc/build.py
  • toolchain/mfc/cli/commands.py

📝 Walkthrough

Walkthrough

This pull request refactors the CI/CD build workflows to separate dependency fetching from source building. Changes include removing the bench argument from Frontier build commands in the benchmarking workflow, updating workflow step labels from "Setup & Build" to "Fetch Dependencies" and "Test" to "Build & Test", removing conditional build-directory checks to enable unconditional builds, simplifying the Frontier build script to only fetch dependencies without the run_bench parameter, and adding a new --deps-only flag to the Python build CLI command that resolves and builds only dependencies without building MFC targets. The toolchain's build module now includes early-exit logic for already-configured dependencies and a dedicated control flow for the deps_only mode.

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description includes a clear summary, test plan with verification items, and addresses the motivation. However, it lacks the required template sections like 'Type of change', proper 'Testing' section structure, and the checklist items are incomplete. Complete the description template by adding 'Type of change' section (mark New feature), expand 'Testing' with actual test results, and check off the verification items or provide test evidence.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: adding a --deps-only flag to separate dependency fetching from source builds, which is the core feature across all modified files.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link

codecov bot commented Mar 25, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 47.42%. Comparing base (6120810) to head (7317ad5).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1336      +/-   ##
==========================================
+ Coverage   45.01%   47.42%   +2.41%     
==========================================
  Files          70       70              
  Lines       20562    19202    -1360     
  Branches     1962     1634     -328     
==========================================
- Hits         9255     9106     -149     
+ Misses      10179     9229     -950     
+ Partials     1128      867     -261     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

When __build_target recursively called build() to resolve sub-deps
(e.g. SILO depends on HDF5), the --deps-only guard intercepted
the recursive call and only built deps-of-deps (none for HDF5),
never building HDF5 itself. This caused SILO's configure to fail
with 'HDF5 was not found'.

Fix: only enter the --deps-only path on the top-level call
(history is empty), letting recursive calls proceed normally.
@github-actions
Copy link

Claude Code Review

Head SHA: 8d39c7b

Files changed:

  • 7
  • .github/workflows/bench.yml
  • .github/workflows/common/bench.sh
  • .github/workflows/common/test.sh
  • .github/workflows/frontier/build.sh
  • .github/workflows/test.yml
  • toolchain/mfc/build.py
  • toolchain/mfc/cli/commands.py

Findings:

  • frontier_amd/build.sh not updated to match new interface: bench.yml removes the bench positional argument from the frontier_amd build script call (changing "bash .github/workflows/frontier_amd/build.sh gpu omp bench" to "bash .github/workflows/frontier_amd/build.sh gpu omp"), but only .github/workflows/frontier/build.sh is present in the changed files — frontier_amd/build.sh is absent. If frontier_amd/build.sh still contains the old run_bench=$3 / if [ "$run_bench" == "bench" ] branching logic (mirroring what was removed from frontier/build.sh), dropping the third argument makes $run_bench empty. The condition "$run_bench" == "bench" then evaluates false, causing it to fall through to the else branch — which in the old code ran mfc.sh test -v -a --dry-run instead of build --deps-only. The benchmark workflow on Frontier AMD would then execute the wrong build command silently, or fail.

Frontier/Frontier AMD now follow the same deps-on-login, source-on-compute
pattern for case optimization tests. Previously, prebuild-case-optimization.sh
built deps + source on the login node. Now:
- Login node: build.sh fetches deps via --deps-only
- Compute node: run_case_optimization.sh builds case-optimized binaries
  then runs them

Phoenix is unchanged (prebuild + run both in SLURM).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant