Skip to content

Conversation

@adrianlizarraga
Copy link
Contributor

@adrianlizarraga adrianlizarraga commented Dec 8, 2025

Description

  • Adds C APIs to support pre-packing of const weights for OrtKernelImpl implementations.
    • APIs optionally support sharing of pre-packed weight data (for cpu-accessible memory).
  • Updates example kernel (Mul) to use new pre-packing API. Tested by existing unit test:
    TEST(OrtEpLibrary, KernelPluginEp_Inference) {
    RegisteredEpDeviceUniquePtr example_kernel_ep;
    ASSERT_NO_FATAL_FAILURE(Utils::RegisterAndGetExampleEp(*ort_env, Utils::example_ep_kernel_registry_info,
    example_kernel_ep));
    Ort::ConstEpDevice plugin_ep_device(example_kernel_ep.get());
    // Create session with example kernel-based plugin EP
    Ort::SessionOptions session_options;
    session_options.AddConfigEntry(kOrtSessionOptionsDisableCPUEPFallback, "1"); // Fail if any node assigned to CPU EP.
    std::unordered_map<std::string, std::string> ep_options;
    session_options.AppendExecutionProvider_V2(*ort_env, {plugin_ep_device}, ep_options);
    // This model has Squeeze, Mul, and Relu nodes. The example plugin EP supports all nodes using registered kernels.
    Ort::Session session(*ort_env, ORT_TSTR("testdata/squeeze_mul_relu.onnx"), session_options);

Motivation and Context

The previous PR added the base APIs that support kernel-based plugin EPs. This PR adds an additional feature that was identified as necessary for the port of WebGPU EP.

@adrianlizarraga adrianlizarraga marked this pull request as ready for review December 9, 2025 16:55
fs-eire
fs-eire previously approved these changes Dec 9, 2025
@adrianlizarraga adrianlizarraga marked this pull request as draft December 11, 2025 00:51
@adrianlizarraga adrianlizarraga marked this pull request as ready for review December 17, 2025 18:06
edgchen1
edgchen1 previously approved these changes Dec 24, 2025
Copy link
Contributor

@fs-eire fs-eire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve for the API and examples.

@adrianlizarraga adrianlizarraga enabled auto-merge (squash) December 27, 2025 07:38
@adrianlizarraga adrianlizarraga merged commit 2d1ed5b into main Dec 27, 2025
91 checks passed
@adrianlizarraga adrianlizarraga deleted the adrianl/plugin-ep-kernel-prepack branch December 27, 2025 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants