Add HNSW ACE build method

[julianmi]

This PR adds a direct hnsw::build API that uses the ACE (Augmented Core Extraction) algorithm to build HNSW indexes on the GPU. ACE enables building HNSW indexes for datasets too large to fit in GPU memory by partitioning the data and building sub-indexes.

CC @tfeher

C++ API

• Added hnsw::build() function with ACE parameters for direct HNSW index construction. This serializes an HNSW index to disk if use_disk is true.
• Added hnsw::graph_build_params::ace_params struct with configurable options:
  • npartitions - number of partitions for parallel build
  • ef_construction - index quality parameter
  • build_dir - directory for disk-based build artifacts
  • use_disk - force disk-based storage mode
• Implemented proper serialization/deserialization for disk-backed HNSW indexes
• Added C++ tests in ann_hnsw_ace.cuh

C API

• Added cuvsHnswBuild function with ACE parameters
• Added C tests in ann_hnsw_ace.cu

Python

• Added hnsw.AceParams class for configuring ACE builds
• Added Python tests in test_hnsw_ace.py

Java

• Added HnswAceParams class
• Added Java tests in HnswAceBuildAndSearchIT.java

Documentation

• Added cuvs_hnsw section to the parameter tuning guide with ACE parameters

Example

• Added hnsw_ace_example.cu demonstrating the build → deserialize → search workflow

[tfeher]

Thanks Julian for the PR! I think it is a nice improvement, but I would still recommend to simplify further along the following lines

• Treat CAGRA parameters as implementation details. We want to keep the usage and the documentation simple for users familiar with HNSW index building.
• Still provide a way to adjust these parameters.
• Clarify that the results are not an exact equivalent to a HNSW graph, but in practice the graph is a good replacement that works well in HNSW search.

[tfeher]

Thanks @julianmi for the updates! There are only two issues remaining:

1. Redundancy with ef_construction parameter (see comments below)
2. Build - deserialize - search workflow

Considering the second point, currently the resulting index can be either in memory or stored in a file. (Although #1604 removes the in-memory ACE path, we might still decide later to enable other in-memory build methods.)

auto hnsw_index = hnsw::build(res, hnsw_params, dataset);
// index is saved to disk at hnsw_index->file_path(), the hnsw_index structure just a wrapper around the file name and index params
hnsw::search(res, search_params, *hnsw_index, queries, indices, distances);

This would fail with the following error message if the index does not fit memory.

Searching HNSW index
terminate called after throwing an instance of 'raft::logic_error'
  what():  RAFT failure at file=/home/scratch.tfeher_gpu_2/cuvs_1597/cpp/src/neighbors/detail/hnsw.hpp line=1140: Cannot search an HNSW index that is stored on disk. The index must be deserialized into memory first using hnsw::deserialize().
Obtained 7 stack frames

Instead, when the index is on disk, then we need to use the following step

auto hnsw_index = hnsw::build(res, hnsw_params, dataset);
hnsw::deserialize(res, hnsw_params, hnsw_index->file_path(), dataset.extent(1), hnsw_params.metric, &hnsw_index_deserialized);
hnsw::search(res, search_params, *hnsw_index_deserialized, queries, indices, distances);

Do we have to dictate such usage patterns? Could we instead let hnsw::search load the index from file if needed?

[tfeher]

Thanks @julianmi for the updates, the PR looks good to me!

[divyegala]

Python approval only