optimize management api calls

[ajha-ss]

We are having redundant API calls to management API for the DOWNLOAD MODEL command.
We are making 3 GET calls for every folder/file , so total GET calls = 3 (for parent folder)+ 3*N (2 checks are redundant as they check if the path is a directory or not but that information is already available in the metadata so these calls can be skipped).

Optimization in this branch -
we make 1 GET call per folder/file so if there is 1 parent folder and N files inside it, total calls now will be 1+N

This reduces the number of calls to 1/3rd or 67% reduction while maintaining the same functionality.

[Copilot]

Pull request overview

This PR optimizes the management API calls for the DOWNLOAD MODEL command by reducing redundant GET requests. Instead of making 3 GET calls per folder/file (one for listing, two for directory checks), the optimization enables passing file metadata along with listings, reducing calls to 1 per folder/file—a 67% reduction while maintaining the same functionality.

Key Changes:

• Added return_objects parameter to listdir methods to optionally return FilesObject instances with metadata instead of just path strings
• Refactored download_folder in FileSpace to use metadata from listings, eliminating redundant is_dir() API calls
• Added internal _download_file method with _skip_dir_check parameter to avoid duplicate directory validation

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.

File	Description
singlestoredb/management/workspace.py	Added return_objects parameter to listdir method with type overloads; download_folder still uses old implementation
singlestoredb/management/files.py	Added return_objects parameter to listdir methods and optimized download_folder to use FilesObject metadata, eliminating redundant API calls
singlestoredb/fusion/handlers/files.py	Updated listdir calls to explicitly pass metadata parameter (contains bug: uses wrong parameter name)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ajha-ss]

@copilot open a new pull request to apply changes based on the comments in this thread

[Copilot]

@ajha-ss I've opened a new pull request, #115, to work on those changes. Once the pull request is ready, I'll request review from you.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

singlestoredb/management/workspace.py:649

• The optimization to reduce API calls is not implemented in this method. Unlike the corresponding method in files.py (lines 1160-1180), this implementation still uses return_objects=False and calls is_dir(f) for each item, which makes an additional GET request per item. This defeats the purpose of the optimization described in the PR.

The method should be updated to match the pattern used in files.py:

1. Use return_objects=True to get FilesObject instances
2. Check entry.type == 'directory' instead of calling is_dir(f)
3. Consider adding an internal _download_file method with _skip_dir_check parameter to avoid the is_dir check in download_file

        for f in self.listdir(stage_path, recursive=True, return_objects=False):
            if self.is_dir(f):
                continue
            target = os.path.normpath(os.path.join(local_path, f))
            os.makedirs(os.path.dirname(target), exist_ok=True)
            self.download_file(f, target, overwrite=overwrite)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[nunogoncalves03]

Thanks for working on this, the overall logic looks good to me. I will leave the Python implementation details to @kesmit13.

What's the test plan for this? Could you describe how you tested it and share the results? If possible, it would also be helpful to add test cases to the existing test suite.

[kesmit13]

The changes look fine. It would be nice if the new option was covered in the unit tests (singlestoredb/tests/test_management.py). There are existing tests for listdir that could be extended. It doesn't have to be a lot of changes, but having one check to make sure the return_objects=True flag actually works.

[ajha-ss]

@nunogoncalves03
Test screen recording -

1. Using main branch sdk locally, it made 12 GET calls for 3files in 1 folder ( 3 calls per file/folder)
2. Using this branch sdk locally, it made 4 GET calls ( 1 call per file/folder)

Screencast.from.2026-01-05.22-31-01.webm

I will add test cases soon to the test suite. Thanks for your review.