copy: add option to strip sparse manifest lists by aguidirh · Pull Request #655 · containers/container-libs

aguidirh · 2026-02-11T10:28:39Z

Add option to strip sparse manifest lists when copying specific images

This PR adds the ability to strip non-copied instances from manifest lists when using CopySpecificImages, implementing the functionality proposed in containers/image#1707.

Motivation

Closes #227

When copying a subset of images from a multi-architecture manifest list using CopySpecificImages, the current behavior always produces a sparse manifest list - a manifest list that references platforms that weren't actually copied. While this maintains the original digest, many registries reject sparse manifests with "blob unknown to registry" errors.

This PR adds a new SparseManifestListAction option that gives users control over this behavior:

KeepSparseManifestList (default, existing behavior) - keeps all references even for non-copied images
StripSparseManifestList (new) - removes non-copied instances and generates a manifest list with only copied instances

This is particularly important for use cases like selective mirroring where only specific architectures need to be copied to registries that don't support sparse manifests.

Changes

1. New `SparseManifestListAction` option in `image/copy`

Added SparseManifestListAction enum with two values:
- KeepSparseManifestList (default): preserves existing behavior - keeps all manifest references
- StripSparseManifestList (new): removes non-copied instances from the manifest list
Integrated into Options struct to give users explicit control

2. New `ListOpDelete` operation in `image/internal/manifest`

Extended ListEdit with ListOpDelete operation and DeleteIndex field to support removing instances
Implemented deletion logic in both OCI1Index and Schema2List
Uses index-based deletion (not digest-based) to handle cases where multiple platforms share the same digest (e.g., amd64 variants v2/v3)
Requires deletion from highest to lowest index to prevent index shifting issues

3. Sparse manifest stripping in `copyMultipleImages` (`image/copy/multiple.go`)

Implemented stripping logic that activates when SparseManifestListAction == StripSparseManifestList
Identifies which instances were not copied by comparing against the original manifest list
Removes skipped instances using the new EditInstances API with ListOpDelete

4. Comprehensive test coverage

Added TestListDeleteInstances covering both OCI and Docker manifest formats
Tests both successful deletion and error handling for invalid indices
Validates high-to-low deletion order requirement

Testing

All existing tests pass. New tests verify:

✅ Successful deletion of instances by index
✅ Error handling for invalid indices
✅ Both OCI Index and Docker Schema2 List formats
✅ Manually tested with oc-mirror and local registry

Credits

This implementation is based on the original work by @bertbaron and @mtrmac in containers/image#1707, adapted for the container-libs monorepo structure with the following changes:

Moved from manifest package to internal/manifest (per container-libs directory structure)
Switched from digest-based to index-based deletion (per review feedback in original PR about platform variants)
Simplified test coverage while maintaining comprehensive validation of both OCI and Docker formats
Integrated with the existing EditInstances API pattern used throughout the codebase

packit-as-a-service · 2026-02-11T10:30:58Z

Packit jobs failed. @containers/packit-build please check.

mtrmac

Thanks! Just a brief first look: generally the structure and choice of affected sub packages look correct.

This is impressively small, I was expecting a that a larger change to internal/manifest would be necessary.

image/copy/multiple.go

image/internal/manifest/docker_schema2_list.go

image/internal/manifest/list_test.go

image/copy/multiple.go

image/copy/copy.go

copy: implement code review feedback for sparse manifest stripping Address feedback from mtrmac's review on PR containers#655: 1. Simplify deletion logic by combining EditInstances calls - Moved deletion tracking into prepareInstanceCopies function - Combined update and delete operations into single EditInstances call - Removed separate digest-based tracking and redundant loops 2. Use slices.Delete for manifest entry removal - Replaced manual slice manipulation with slices.Delete in both docker_schema2_list.go and oci_index.go for cleaner code 3. Integrate deletion tests into existing test methods - Removed standalone TestListDeleteInstances function - Added deletion test cases to TestSchema2ListEditInstances and TestOCI1EditInstances for better test organization 4. Move deletion logic into prepareInstanceCopies (most critical fix) - Modified prepareInstanceCopies to return ([]instanceCopy, []int, error) - Track indices to delete when SparseManifestListAction == StripSparseManifestList - Properly handles duplicate digests by using instance indices instead of digest-based lookup, fixing edge case where same digest appears multiple times in manifest list 5. Use slices.Backward for reverse iteration - Simplified deletion loop using slices.Backward iterator - Maintains high-to-low deletion order to avoid index shifting 6. Move SparseManifestListAction field for better organization - Relocated field from line 150 to line 116 in copy.go - Now positioned near related ImageListSelection and Instances fields - Improves logical grouping of manifest list options All tests pass. The implementation now correctly handles: - Duplicate digests in manifest lists - Single EditInstances call for efficiency - Cleaner separation of concerns - Better code organization Fixes feedback on containers#655 Signed-off-by: Alex Guidi <aguidi@redhat.com>

aguidirh · 2026-02-17T16:56:04Z

@mtrmac when using StripSparseManifestList it will fail with Manifest list was edited, but we cannot modify it: "Would invalidate signatures" should we make mandatory the usage of --remove-signatures when using StripSparseManifestList ?

mtrmac

Thanks!

Just a very quick look for now.

image/copy/multiple.go

mtrmac · 2026-02-17T17:36:36Z

should we make mandatory the usage of --remove-signatures when using StripSparseManifestList ?

Hum… it’s a thought.

We currently ask the user to explicitly use --remove-signatures for all other edits (e.g. converting the manifest format or changing compression), so I think also requiring an explicit ACK that the signatures are going to be broken would be consistent here.

Actually, c/image only cares about the signatures of the per-platform instances, it never validates the signature on the list (but it does create it). So, ideally, we should allow stripping the list’s signature but keep the per-instance signatures, so that users can create sparse mirrors but still validate image integrity. I think that would be a new option in copy.Options — and maybe some mirroring tools (in some situations?) would enable it by default when the user explicitly asks for sparse copies.

E.g., purely hypothetically, it might make sense to hard-code this to on when mirroring the OpenShift product payload, because we know how signatures are validated there; and maybe not hard-code it for other images, because those might be validated using some other software like cosign.

TomSweeneyRedHat · 2026-02-25T21:58:27Z

LGTM
and happy green test buttons.

Add StripOnlyListSignatures option to strip manifest list signatures while preserving per-instance signatures when using CopySpecificImages with StripSparseManifestList. This implements "Option 2" from PR containers#655. Previously, users had to use RemoveSignatures which removed all signatures, including per-instance ones that would still be valid after stripping the manifest list. The option requires explicit validation: - Only valid with CopySpecificImages + StripSparseManifestList - Validates at multiple points in copy.Image() and copyMultipleImages() - When used with signed images, requires explicit signature handling Per-instance signatures are preserved because copySingleImage handles each instance independently. Signed-off-by: Alex Guidi <aguidi@redhat.com>

aguidirh · 2026-02-26T15:59:52Z

We currently ask the user to explicitly use --remove-signatures for all other edits (e.g. converting the manifest format or changing compression), so I think also requiring an explicit ACK that the signatures are going to be broken would be consistent here.

I added a validation which requires the user to use RemoveSignatures or the new option StripOnlyListSignatures when SparseManifestListAction == StripSparseManifestList

Actually, c/image only cares about the signatures of the per-platform instances, it never validates the signature on the list (but it does create it). So, ideally, we should allow striping the list’s signature but keep the per-instance signatures, so that users can create sparse mirrors but still validate image integrity. I think that would be a new option in copy.Options — and maybe some mirroring tools (in some situations?) would enable it by default when the user explicitly asks for sparse copies.

I added an option StripOnlyListSignatures where it removes the signature of the manifest list while it keeps the signature of the instances requested by the user.

E.g., purely hypothetically, it might make sense to hard-code this to on when mirroring the OpenShift product payload, because we know how signatures are validated there; and maybe not hard-code it for other images, because those might be validated using some other software like cosign.

I'm not sure if I understood this last comment, could you please explain it to me?

I just pushed a commit with the last changes that I mentioned above.

mtrmac · 2026-02-26T21:04:18Z

purely hypothetically, it might make sense to hard-code this to on when mirroring the OpenShift product payload, because we know how signatures are validated there; and maybe not hard-code it for other images, because those might be validated using some other software like cosign.

I'm not sure if I understood this last comment, could you please explain it to me?

Software that knows how the images is going to be used can hard-code option values so that users don’t have to care; generic software like skopeo should probably default to doing the safe thing, and not discard potentially-valuable data or break signatures without the user understanding the consequence.

mtrmac

Thanks!

image/copy/copy.go

image/copy/multiple.go

image/internal/manifest/docker_schema2_list_test.go

image/copy/multiple.go

image/copy/multiple_test.go

image/copy/multiple.go

image/copy/multiple_test.go

copy: implement code review feedback for sparse manifest stripping Address feedback from mtrmac's review on PR containers#655: 1. Simplify deletion logic by combining EditInstances calls - Moved deletion tracking into prepareInstanceCopies function - Combined update and delete operations into single EditInstances call - Removed separate digest-based tracking and redundant loops 2. Use slices.Delete for manifest entry removal - Replaced manual slice manipulation with slices.Delete in both docker_schema2_list.go and oci_index.go for cleaner code 3. Integrate deletion tests into existing test methods - Removed standalone TestListDeleteInstances function - Added deletion test cases to TestSchema2ListEditInstances and TestOCI1EditInstances for better test organization 4. Move deletion logic into prepareInstanceCopies (most critical fix) - Modified prepareInstanceCopies to return ([]instanceCopy, []int, error) - Track indices to delete when SparseManifestListAction == StripSparseManifestList - Properly handles duplicate digests by using instance indices instead of digest-based lookup, fixing edge case where same digest appears multiple times in manifest list 5. Use slices.Backward for reverse iteration - Simplified deletion loop using slices.Backward iterator - Maintains high-to-low deletion order to avoid index shifting 6. Move SparseManifestListAction field for better organization - Relocated field from line 150 to line 116 in copy.go - Now positioned near related ImageListSelection and Instances fields - Improves logical grouping of manifest list options All tests pass. The implementation now correctly handles: - Duplicate digests in manifest lists - Single EditInstances call for efficiency - Cleaner separation of concerns - Better code organization Fixes feedback on containers#655 Signed-off-by: Alex Guidi <aguidi@redhat.com>

Add StripOnlyListSignatures option to strip manifest list signatures while preserving per-instance signatures when using CopySpecificImages with StripSparseManifestList. This implements "Option 2" from PR containers#655. Previously, users had to use RemoveSignatures which removed all signatures, including per-instance ones that would still be valid after stripping the manifest list. The option requires explicit validation: - Only valid with CopySpecificImages + StripSparseManifestList - Validates at multiple points in copy.Image() and copyMultipleImages() - When used with signed images, requires explicit signature handling Per-instance signatures are preserved because copySingleImage handles each instance independently. Signed-off-by: Alex Guidi <aguidi@redhat.com>

image/copy/copy.go

image/copy/multiple.go

mtrmac · 2026-03-03T19:36:40Z

image/copy/multiple_test.go

+
+// TestStripOnlyListSignaturesValidation tests the validation logic for StripOnlyListSignatures
+// by actually calling copy.Image() with various option combinations.
+func TestStripOnlyListSignaturesValidation(t *testing.T) {


I really didn’t mean to ask for the in-memory test harness here.

It was more of… “if there is no plausible unit test, it is acceptable to leave the code untested here instead of writing thousands of lines”.

I would actually like to ask for an added test — but not here. At least one integration test in the Skopeo repo, perhaps exercising the platform filtering and sparse creation together. (Integration tests in Skopeo are where we run the top-level skopeo copy commands, and where we have local registries, other infrastructure, and license to take more than a few hundred milliseconds.) That would, though, mean also wiring up the new features as Skopeo options, which might not be in scope — so, non-blocking as well.

mtrmac · 2026-03-03T19:41:30Z

image/copy/copy.go

+		}
+		single, err := c.copySingleImage(ctx, singleInstance, nil, copySingleImageOptions{requireCompressionFormatMatch: requireCompressionFormatMatch})
 		if err != nil {
-			return nil, fmt.Errorf("copying system image from manifest list: %w", err)


Hum, we probably want this context (at least Skopeo defaults to CopySystemImage, and users might not realize that). That’s a bit awkward (singleInstanceErrorWrapping?), but on balance, consolidating the singleInstance code paths is probably still worth it.

Fixed at commit 4a6817d.

I’m sorry, I by “this context” I meant “system image from manifest list”.

image/copy/multiple_test.go

@bertbaron

Add the ability to strip non-copied instances from manifest lists when using CopySpecificImages. This implements the functionality originally proposed in containers/image#1707. When copying a subset of images from a multi-architecture manifest list using CopySpecificImages, the current behavior always produces a sparse manifest list - a manifest list that references platforms that weren't actually copied. While this maintains the original digest, many registries reject sparse manifests with "blob unknown to registry" errors. This commit adds a new SparseManifestListAction option that gives users control over this behavior: - KeepSparseManifestList (default): preserves existing behavior - StripSparseManifestList (new): removes non-copied instances The implementation includes: 1. New SparseManifestListAction enum in image/copy 2. New ListOpDelete operation in image/internal/manifest with support for both OCI1Index and Schema2List formats 3. Index-based deletion (not digest-based) to handle platform variants that share the same digest 4. Stripping logic in copyMultipleImages that activates when StripSparseManifestList is set 5. Comprehensive test coverage for deletion operations Based on original work by @bertbaron and @mtrmac in containers/image#1707, adapted for the container-libs monorepo structure. Relates to containers#227 Signed-off-by: Alex Guidi <aguidi@redhat.com>

Remove 'orthogonal' emphasis from RemoveListSignatures comment. API consumers should assume options don't interact unless documented. Signed-off-by: Alex Guidi <aguidi@redhat.com>

Remove redundant comment about single vs multiple image copying. The code logic with singleInstance variable is self-explanatory. Signed-off-by: Alex Guidi <aguidi@redhat.com>

Restore inline comment for ChooseInstanceByCompression. Clarifies that SourceCtx is used to pick matching instance. Signed-off-by: Alex Guidi <aguidi@redhat.com>

Remove redundant 'Single-image copy path' comment. The condition and variable name are self-documenting. Signed-off-by: Alex Guidi <aguidi@redhat.com>

Remove redundant 'Multi-image copy path' comment. The else block context makes this self-evident. Signed-off-by: Alex Guidi <aguidi@redhat.com>

Add validation to reject RemoveListSignatures in single instance mode. RemoveListSignatures only applies when copying manifest lists. Signed-off-by: Alex Guidi <aguidi@redhat.com>

Return -1 instead of 0 on error paths in prepareInstanceOps. Makes it more obvious if the count is accidentally used despite an error. Signed-off-by: Alex Guidi <aguidi@redhat.com>

Remove redundant comment about delete operation. The condition and debug text are self-explanatory. Signed-off-by: Alex Guidi <aguidi@redhat.com>

Refactor signature validation per mtrmac's suggestion. Move validation into prepareInstanceOps where delete operations occur. Simplify error handling by removing string matching in consumer code. Signed-off-by: Alex Guidi <aguidi@redhat.com>

Add error context for single image copy. Helps users understand when copying single instance from manifest list. Signed-off-by: Alex Guidi <aguidi@redhat.com>

Remove unused ociManifestFile constant from tests. Signed-off-by: Alex Guidi <aguidi@redhat.com>

Fix gofumpt formatting by using explicit octal prefix (0o644 instead of 0644) for file permissions in os.WriteFile calls Signed-off-by: Alex Guidi <aguidi@redhat.com>

mtrmac · 2026-03-06T14:29:49Z

image/copy/multiple.go

-		cannotModifyManifestListReason = "Would invalidate signatures"
+		// If RemoveListSignatures is set, we allow modification of the manifest list
+		// by stripping only the list-level signature while preserving per-instance signatures.
+		// This is handled below by setting sigs to empty (similar to RemoveSignatures).


This line of the comment refers to removed code.

mtrmac · 2026-03-06T14:30:25Z

image/copy/multiple.go

 		// If we can't just use the original value, but we have to change it, flag an error.
 		if !bytes.Equal(attemptedManifestList, originalManifestList) {
 			if cannotModifyManifestListReason != "" {
+				if cannotModifyManifestListReason == "Would invalidate signatures" {


This is now dead code

mtrmac · 2026-03-06T14:32:41Z

image/copy/multiple.go

+	// Add delete operations in reverse order (highest to lowest index) to avoid shifting
+	slices.Reverse(deleteOps)
+	copyLen := len(res) // Count copy/clone operations before appending deletes
+	res = append(res, deleteOps...)


LLM points out: If the outcome is to copy nothing and delete all entries in a list, we probably want to fail. (If we copy nothing but copy a sparse manifest list, that’s still, perhaps, useful for someone.)

github-actions bot added the image Related to "image" package label Feb 11, 2026

mtrmac reviewed Feb 11, 2026

View reviewed changes

mtrmac reviewed Feb 17, 2026

View reviewed changes

image/copy/multiple.go Outdated Show resolved Hide resolved

image/copy/multiple.go Outdated Show resolved Hide resolved

mtrmac reviewed Feb 26, 2026

View reviewed changes

aguidirh changed the title ~~copy: add option to strip sparse manifest lists~~ WIP: copy: add option to strip sparse manifest lists Feb 27, 2026

aguidirh force-pushed the sparse-manifest-strip-pr1707 branch from 9889bbf to 586fc62 Compare March 3, 2026 13:52

aguidirh changed the title ~~WIP: copy: add option to strip sparse manifest lists~~ copy: add option to strip sparse manifest lists Mar 3, 2026

mtrmac reviewed Mar 3, 2026

View reviewed changes

aguidirh mentioned this pull request Mar 5, 2026

Support copying images for specific platforms/architectures #227

Open

aguidirh added 11 commits March 5, 2026 15:12

fixup! copy: add option to strip sparse manifest lists

f98bc6d

Remove 'orthogonal' emphasis from RemoveListSignatures comment. API consumers should assume options don't interact unless documented. Signed-off-by: Alex Guidi <aguidi@redhat.com>

fixup! copy: add option to strip sparse manifest lists

a61606f

Remove redundant comment about single vs multiple image copying. The code logic with singleInstance variable is self-explanatory. Signed-off-by: Alex Guidi <aguidi@redhat.com>

fixup! copy: add option to strip sparse manifest lists

b28ed67

Restore inline comment for ChooseInstanceByCompression. Clarifies that SourceCtx is used to pick matching instance. Signed-off-by: Alex Guidi <aguidi@redhat.com>

fixup! copy: add option to strip sparse manifest lists

1e6e66a

Remove redundant 'Single-image copy path' comment. The condition and variable name are self-documenting. Signed-off-by: Alex Guidi <aguidi@redhat.com>

fixup! copy: add option to strip sparse manifest lists

1ae335d

Remove redundant 'Multi-image copy path' comment. The else block context makes this self-evident. Signed-off-by: Alex Guidi <aguidi@redhat.com>

fixup! copy: add option to strip sparse manifest lists

b19e2fe

Add validation to reject RemoveListSignatures in single instance mode. RemoveListSignatures only applies when copying manifest lists. Signed-off-by: Alex Guidi <aguidi@redhat.com>

fixup! copy: add option to strip sparse manifest lists

ef0a1a2

Return -1 instead of 0 on error paths in prepareInstanceOps. Makes it more obvious if the count is accidentally used despite an error. Signed-off-by: Alex Guidi <aguidi@redhat.com>

fixup! copy: add option to strip sparse manifest lists

86f07c9

Remove redundant comment about delete operation. The condition and debug text are self-explanatory. Signed-off-by: Alex Guidi <aguidi@redhat.com>

fixup! copy: add option to strip sparse manifest lists

4a6817d

Add error context for single image copy. Helps users understand when copying single instance from manifest list. Signed-off-by: Alex Guidi <aguidi@redhat.com>

aguidirh added 2 commits March 5, 2026 16:50

fixup! copy: add option to strip sparse manifest lists

fc4364b

Remove unused ociManifestFile constant from tests. Signed-off-by: Alex Guidi <aguidi@redhat.com>

fixup! copy: add option to strip sparse manifest lists

ab225b6

Fix gofumpt formatting by using explicit octal prefix (0o644 instead of 0644) for file permissions in os.WriteFile calls Signed-off-by: Alex Guidi <aguidi@redhat.com>

aguidirh force-pushed the sparse-manifest-strip-pr1707 branch from 586fc62 to ab225b6 Compare March 6, 2026 13:38

mtrmac reviewed Mar 6, 2026

View reviewed changes

Conversation

aguidirh commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Add option to strip sparse manifest lists when copying specific images

Motivation

Changes

1. New SparseManifestListAction option in image/copy

2. New ListOpDelete operation in image/internal/manifest

3. Sparse manifest stripping in copyMultipleImages (image/copy/multiple.go)

4. Comprehensive test coverage

Testing

Credits

Uh oh!

packit-as-a-service bot commented Feb 11, 2026

Uh oh!

mtrmac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aguidirh commented Feb 17, 2026

Uh oh!

mtrmac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mtrmac commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TomSweeneyRedHat commented Feb 25, 2026

Uh oh!

aguidirh commented Feb 26, 2026

Uh oh!

mtrmac commented Feb 26, 2026

Uh oh!

mtrmac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mtrmac Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

mtrmac Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

aguidirh Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

mtrmac Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mtrmac Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

mtrmac Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

aguidirh commented Feb 11, 2026 •

edited

Loading

1. New `SparseManifestListAction` option in `image/copy`

2. New `ListOpDelete` operation in `image/internal/manifest`

3. Sparse manifest stripping in `copyMultipleImages` (`image/copy/multiple.go`)

mtrmac commented Feb 17, 2026 •

edited

Loading