Skip to content

Use jsonargparse#9336

Closed
ThomasWaldmann wants to merge 12 commits intoborgbackup:masterfrom
ThomasWaldmann:use-jsonargparse
Closed

Use jsonargparse#9336
ThomasWaldmann wants to merge 12 commits intoborgbackup:masterfrom
ThomasWaldmann:use-jsonargparse

Conversation

@ThomasWaldmann
Copy link
Member

WIP

@codecov
Copy link

codecov bot commented Feb 15, 2026

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
1623 2 1621 346
View the top 2 failed test(s) by shortest run time
src.borg.testsuite.archiver.create_cmd_test::test_create_pattern_root[archiver]
Stack Traces | 0.338s run time
archivers = 'archiver'
request = <FixtureRequest for <Function test_create_pattern_root[archiver]>>

    def test_create_pattern_root(archivers, request):
        """test create with only a root pattern"""
        archiver = request.getfixturevalue(archivers)
        cmd(archiver, "repo-create", RK_ENCRYPTION)
        create_regular_file(archiver.input_path, "file1", size=1024 * 80)
        create_regular_file(archiver.input_path, "file2", size=1024 * 80)
>       output = cmd(archiver, "create", "test", "-v", "--list", "--pattern=R input")
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.../testsuite/archiver/create_cmd_test.py:434: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

archiver = <borg.conftest.ArchiverSetup object at 0x7fca7cb86170>
args = ('create', 'test', '-v', '--list', '--pattern=R input'), kw = {}
exit_code = 0, fork = False, binary_output = False, ret = 2
output = 'usage: -c [--cockpit] [--critical] [--error] [--warning] [--info] [--debug]\n          [--debug-topic TOPIC] [-p] [--...nchmark,debug,key} ...\ntip: For details of accepted options run: -c --help\nerror: Need at least one PATH argument.\n'
@py_assert1 = False, @py_format3 = '2 == 0', @py_format5 = 'assert 2 == 0'

    def cmd(archiver, *args, **kw):
        exit_code = kw.pop("exit_code", 0)
        fork = kw.pop("fork", None)
        binary_output = kw.get("binary_output", False)
        if fork is None:
            fork = archiver.FORK_DEFAULT
        ret, output = exec_cmd(
            f"--repo={archiver.repository_location}", *args, archiver=archiver.archiver, fork=fork, exe=archiver.EXE, **kw
        )
        if ret != exit_code:
            print(output)
>       assert ret == exit_code
E       assert 2 == 0

.../testsuite/archiver/__init__.py:152: AssertionError
src.borg.testsuite.archiver.create_cmd_test::test_create_pattern_root[remote_archiver]
Stack Traces | 1.45s run time
archivers = 'remote_archiver'
request = <FixtureRequest for <Function test_create_pattern_root[remote_archiver]>>

    def test_create_pattern_root(archivers, request):
        """test create with only a root pattern"""
        archiver = request.getfixturevalue(archivers)
        cmd(archiver, "repo-create", RK_ENCRYPTION)
        create_regular_file(archiver.input_path, "file1", size=1024 * 80)
        create_regular_file(archiver.input_path, "file2", size=1024 * 80)
>       output = cmd(archiver, "create", "test", "-v", "--list", "--pattern=R input")
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.../testsuite/archiver/create_cmd_test.py:434: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

archiver = <borg.conftest.ArchiverSetup object at 0x7fca781ec870>
args = ('create', 'test', '-v', '--list', '--pattern=R input'), kw = {}
exit_code = 0, fork = False, binary_output = False, ret = 2
output = 'usage: -c [--cockpit] [--critical] [--error] [--warning] [--info] [--debug]\n          [--debug-topic TOPIC] [-p] [--...nchmark,debug,key} ...\ntip: For details of accepted options run: -c --help\nerror: Need at least one PATH argument.\n'
@py_assert1 = False, @py_format3 = '2 == 0', @py_format5 = 'assert 2 == 0'

    def cmd(archiver, *args, **kw):
        exit_code = kw.pop("exit_code", 0)
        fork = kw.pop("fork", None)
        binary_output = kw.get("binary_output", False)
        if fork is None:
            fork = archiver.FORK_DEFAULT
        ret, output = exec_cmd(
            f"--repo={archiver.repository_location}", *args, archiver=archiver.archiver, fork=fork, exe=archiver.EXE, **kw
        )
        if ret != exit_code:
            print(output)
>       assert ret == exit_code
E       assert 2 == 0

.../testsuite/archiver/__init__.py:152: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@ThomasWaldmann ThomasWaldmann mentioned this pull request Feb 16, 2026
Replace stdlib argparse with jsonargparse for enhanced argument parsing,
using a compatibility layer in _argparse.py.

Key changes:

- Add _argparse.py: ArgumentParser subclass bridging Borg's argparse
  patterns with jsonargparse (type+action handling, namespace flattening)

- Switch from add_subparsers/add_parser to add_subcommands/add_subcommand
  across all command modules

- Replace set_defaults(func=) with COMMAND_DISPATCH table and get_func()

- Highlander: handle type conversion internally (BoundHighlander pattern)
  since jsonargparse forbids combining type= and action=

- TypeConvertingAction: wrapper for standard actions (append, store) that
  need type conversion

- Pattern actions: handle None paths/patterns from jsonargparse

- flatten_namespace: convert jsonargparse's nested subcommand namespaces
  to the flat namespace Borg's command handlers expect

- Update argparsing_test.py for dispatch table (get_func vs args.func)

Implemented by AI: Claude Opus 4.6 (Thinking).
Refactor jap_wrapper.py to explicitly define a custom ArgumentGroup class instead of relying on jsonargparse's dynamic subclass generation.

jsonargparse attempts to create an ArgumentGroup subclass that inherits add_argument from the parser by reading source code with inspect.getsource(). This mechanism is fragile and can fail on Windows CI (and likely frozen binaries), causing it to fall back to the standard ArgumentGroup.

When falling back, the standard ArgumentGroup lacks our add_argument wrapper that strips type when action is present. This results in ValueError: Providing both type and action not allowed because jsonargparse forbids this combination.

By explicitly defining ArgumentGroup with our mixin and setting self._group_class, we ensure our compatibility logic is always used.
- Remove Highlander and MakePathSafeAction form jap_wrapper.py and parseformat.py.

- Replace usage with standard argparse actions and new type validators (SafePathSpec, compression_spec_validator).

- Make validators (compression_spec_validator, location_validator, positive_int_validator, parse_file_size, timestamp) idempotent and compatible with jsonargparse validation logic.

- change default value for --first and --last to None (0 is considered invalid by the validator)
When using --patterns-from with R (root) lines alongside paths on the
command line, the positional paths argument would overwrite the roots
parsed from the pattern file.

Fix by storing pattern file roots in a class-level list on
ArgparsePatternFileAction during parsing, then merging them into
args.paths in flatten_namespace() after argument parsing is complete.

Add test_create_pattern_file_mixed_roots to verify that roots from a
pattern file and roots from the command line are properly merged and
that exclude patterns from the pattern file still apply correctly.
@mauvilsa
Copy link

FIY, with pull request omni-us/jsonargparse#845 now it is possible to add arguments with type and action. Not yet in a release, but the dependency could be set to install directly from the github main branch. I haven't yet looked at the pull request in detail.

@ThomasWaldmann
Copy link
Member Author

@mauvilsa Ah, that's great, thanks for the info.

That limitation was one of the major pain points in switching to jsonargparse.

@ThomasWaldmann
Copy link
Member Author

superceded by #9413

@ThomasWaldmann ThomasWaldmann deleted the use-jsonargparse branch February 25, 2026 14:48
@ThomasWaldmann
Copy link
Member Author

@mauvilsa that type+action change was very helpful for borg, thanks.

doing #9413 was quite a bit easier than this first try.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants