feat: support sycl gpu on windows

[makejiang]

Feature Description

current on windows platform, cuda and vulkan are supported. is there a roadmap to support sycl?
I think it will improve the performance especially on intel's gpu.

build llama.cpp with sycl:
https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/SYCL.md#windows

The Solution

win-x64-sycl

Considered Alternatives

win-x64-sycl

Additional Context

No response

Related Features to This Feature Request

• Metal support
• CUDA support
• Vulkan support
• Grammar
• Function calling

Are you willing to resolve this issue by submitting a Pull Request?

Yes, I have the time, but I don't know how to start. I would need guidance.

[giladgd]

@makejiang I don't have a machine with an Intel Arc GPU nor have access to one, so I can't add official support for it on node-llama-cpp and maintain support for it.
From my tests, the performance of Vulkan is comparable to the performance of CUDA on the machines that I tested both on, so given that Intel GPUs have decent Vulkan support I suppose the performance of Vulkan would be comparable to SYCL on such machines, but it's worth testing.

If you'd like to add official support for this yourself and open a PR, these are the main components that you'd have to take care of:

detectAvailableComputeLayers.ts

Automatically detect when the current machine has SYCL support.
This is used to only try to load SYCL binaries or try to compile them when support is detected.
Also find the relevant usages of this function and update them appropriately to prioritize SYCL when it's detected.

---

getGpuInfo.cpp

Update all the relevant functions to return the information about the available SYCL GPUs.
Take a look at CUDA for a good example of what you'd need to support (ignore the Vulkan and Apple code there, don't let that confuse you).
You'll probably have to log stuff from the native code there to get it right.
Use fputs("Some text\n", stderr); fflush(stderr); to log things for debugging purposes there, but don't commit it.

---

@node-llama-cpp/win-x64-sycl

Add the relevant packages for the prebuilt binaries for Windows and Linux.
Just duplicate packages/@node-llama-cpp/win-x64-cuda and packages/@node-llama-cpp/linux-x64-cuda and make the relevant changes.
Don't forget to add a reference to this under the main package.json's optionalDependencies as well.

If the binaries get too big (over 10MB), try to find ways to reduce their size if possible.
It's ok if the binaries will only work on modern GPUs, since we always have Vulkan as a fallback.

When consuming node-llama-cpp from its npm package, before it loads a prebuilt binary it first tests whether loading the binary works or crashes the process by loading it in a child process first before actually loading it in the main process, and if the test fails if then it fallbacks to the next supported compute layer (e.g. CUDA, Vulkan, etc.).
If the user will have missing libraries for some reason, it'll then automatically fallback to Vulkan, so it's should be fine to only support model GPUs/SYCL versions.
It's still better to detect support properly first (via detectAvailableComputeLayers) to avoid spending time on something we know wouldn't work in advance.

---

build.yml

Update the build-binaries job to install the libraries required to build the SYCL binaries on both Windows and Linux, and add the commands to actually build the binaries.

---

SYCL.md

Add documentation on how to setup the prerequisites to build node-llama-cpp with SYCL.
Take a look at CUDA.md for inspiration and try to keep the same format.
Add relevant links to this page when relevant in the documentation.

When you encounter errors on your machine when trying to compile with SYCL, I recomment you to document the issues you encountered and their solutions under this document.
For issues that can automatically be solved in the compilation process (like detecting some error message that gets printed and then change some compilation parameters or environment variables), consider adding automatic detection and handling in compileLLamaCpp.ts (take a look at the CUDA issue handlings there for inspiration, as these solve most of the CUDA compilation issues that people encounter automatically)

---

I recommend you to search the source code for CUDA to find where you need to also add support for SYCL.
Read the developer guide to get started with developing node-llama-cpp.
To ensure that everything works properly with SYCL, I recommend you to run this command to run the testsuite against SYCL:

NODE_LLAMA_CPP_GPU=sycl npm run test:vitest

Let me know if you need any additional help with implementing this, or if you get stuck on something.

[makejiang]

@giladgd
Thank you very much for your reply. I will first digest your guidance and then start trying to add the support for SYCL.
I will continue to ask you here if I have any questions.
Thank you.