Add ekore_capi

[AkshatRai07]

A step for #519 ; we explicitly shape our public API along the way #145

In this PR we:

• Add the ekore_capi crate.
• Expose anomalous_dimensions and operator_matrix_elements as C functions.
• Change visibility of functions in ekore crate.

TODO

Create and merge a PR for:

• Deploy of maturin wheels failed #538. Fix Maturin workflow #540
• --release tag. Fix poe compile #541
• No malloc in ekore. Remove mallocs and pass single buffer in run_op_integration #542
• Single buffer passing throughout the integration. Remove mallocs and pass single buffer in run_op_integration #542
• ekore visibility. Changing visibility of ekore and Splitting constants.rs #543
• Split crates/ekore/src/constants.rs #539 Changing visibility of ekore and Splitting constants.rs #543
• spacelike.rs tests. Add tests to spacelike.rs files #546
• Bump Rust and add msrv.yml Add msrv.yml #545
• Define version of a dependency in workspace and make crates point to that. Add centralised workspace dependencies #544
• port N3LO test Check N3LO NS number conservation #547
• Update rust/ekore documentation #550

Only after all that, in this PR:

• Expose necessary constants.
• Add helper functions.
• Add guards in ekore_capi.
• Add tests.
• Add CI
• Add CD.
• Add doc-strings.
• Explore removal of duplicate code.
• Add Changelog entry

[Copilot]

Pull request overview

This PR introduces a new ekore_capi crate to expose ekore’s anomalous dimensions and operator matrix elements through a C ABI, and tightens internal visibility in ekore to keep implementation details private while preserving higher-level APIs.

Changes:

• Added crates/ekore_capi with C-ABI entrypoints for unpolarized/polarized anomalous dimensions and unpolarized space-like OMEs, plus Cache allocation/free helpers.
• Adjusted visibility in ekore (many pub → pub(super) / pub(crate), and pub mod → mod) to reduce exposed internals.
• Added cbindgen/cargo-c metadata and updated the lockfile to include the new crate.

Reviewed changes

Copilot reviewed 29 out of 30 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
crates/ekore/src/operator_matrix_elements/unpolarized/spacelike/as2.rs	Restricts OME helper function visibility to module scope.
crates/ekore/src/operator_matrix_elements/unpolarized/spacelike/as1.rs	Restricts OME helper function visibility to module scope.
crates/ekore/src/operator_matrix_elements/unpolarized/spacelike.rs	Makes as1/as2 internal modules.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4/gqg.rs	Restricts N3LO component function visibility.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4/gps.rs	Restricts N3LO component function visibility.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4/gnsv.rs	Narrows visibility of gamma_nsv.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4/gnsp.rs	Narrows visibility of gamma_nsp.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4/gnsm.rs	Narrows visibility of gamma_nsm.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4/ggq.rs	Restricts N3LO component function visibility.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4/ggg.rs	Restricts N3LO component function visibility.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4.rs	Removes public re-exports and makes singlet entry internal.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as3.rs	Restricts NNLO functions to parent module.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as2.rs	Narrows visibility of NLO functions and singlet helper pieces.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as1aem1.rs	Restricts mixed QCD×QED pieces to parent module.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as1.rs	Narrows visibility of LO helper pieces.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/aem2.rs	Restricts NLO QED pieces to parent module.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/aem1.rs	Restricts LO QED pieces to parent module.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike.rs	Makes most submodules internal; keeps as1/as2 crate-visible.
crates/ekore/src/anomalous_dimensions/polarized/spacelike/as2.rs	Restricts polarized NLO functions to parent module.
crates/ekore/src/anomalous_dimensions/polarized/spacelike/as1.rs	Restricts polarized LO functions to parent module.
crates/ekore/src/anomalous_dimensions/polarized/spacelike.rs	Makes polarized as1/as2 internal modules.
crates/ekore_capi/src/lib.rs	Defines C-facing ComplexF64 and Cache lifecycle API; wires C API modules.
crates/ekore_capi/src/ad_us.rs	Adds C ABI wrappers for unpolarized space-like anomalous dimensions (QCD and QCD×QED).
crates/ekore_capi/src/ad_ps.rs	Adds C ABI wrappers for polarized space-like anomalous dimensions (QCD).
crates/ekore_capi/src/ome_us.rs	Adds C ABI wrappers for unpolarized space-like OME towers.
crates/ekore_capi/README.md	Documents crate purpose and citation policy.
crates/ekore_capi/cbindgen.toml	Configures C header generation.
crates/ekore_capi/Cargo.toml	Declares new crate, crate-type, and cargo-c metadata.
Cargo.lock	Adds ekore_capi to the workspace lockfile.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[AkshatRai07]

For change in visibility of ekore functions, I changed all of them to pub(super) and only the functions in the spacelike.rs files are now pub. Those who caused errors on being pub(super) (because they are called by functions outside their mod but inside the crate), their visibility was changed to pub(crate).

What I will do later:

• Create C, C++, and maybe Fortran files for running tests.
• Add ccompile, ctest, and other relevant tasks in pyproject.toml for poethepoet.
• Add CI/CD files.

There are a few doubts for which I want your opinion:

For ekore:

• What should be the visibility for functions in harmonics?

For ekore_capi:

• Should we expose constants.rs?
• Should we expose harmonics? If yes then which functions?
• In ekore crate, tests are only for the ad_us, should I write tests for all spacelike.rs (as they are there on the python side), and hence write the same tests for capi?
• How do we want to publish this? Do we want to push this crate to crates.io? In pineappl it is pointed out that cargo-c cannot fetch over the internet, and cargo install won't work. What I've been doing is:

cargo cinstall --release -p ekore_capi --destdir=./crates/ekore_capi/dist --prefix=/

We can then publish the dist as a GitHub release (as it is done in pineappl).

• Should we write extensive doc comments, detailing the function's purpose and parameters?

My initial thought is to make only the harmonics::cache::Cache public, expose that through c_api. If required, we can change visibility of others later. We should also expose constants.rs, write more tests, publish the way pineappl does it, and add doc comments.

Other things I thought we could've done for #518, but can be done here:

• Add a --release tag in poethepoet's compile task.
• Remove all the mallocs inside the ekore and eko crate. For eg. replacing all the vec! with constant size arrays.
• Pass a buffer through rust_quad_ker such that it doesn't malloc at every quadrature point, so one buffer for the entire integration loop.

Do let me know your thoughts.

[AkshatRai07]

CoPilot reviews Summary:

• There will be panic if we pass order_qcd as 0 (As you cannot initialize a 0 dimension vec). I don't think you'll ever pass that, there was no such guard in ekore, but still no problem in adding a guard here.
• CoPilot is stating that there might be accidental panics (by say accidentally passing a wrong value), and these panics can cause Undefined Behaviour, so we should add safeguards. This is partially True. This behavior occurred for versions before 1.81. For 1.81+, it results in the crashing of code instead of undefined behaiour. We can either bump the 1.70 to 1.81 in our workspace Cargo.toml or add graceful handling.
• CoPilot says to use checked_mul. This is just noise, there's no way order_qcd * 9 or something will get close to max for usize.

My initial thought is to add safeguards and not bump the Rust version. Please let me know your thoughts too.

[AkshatRai07]

Small note, the data.nnpdf.science site is down, assets.sh is failing locally too, and that is why these tests are failing. Do you know the status of the site?

[felixhekhorn]

Small note, the data.nnpdf.science site is down, assets.sh is failing locally too, and that is why these tests are failing. Do you know the status of the site?

Indeed, I noticed. I hope the relevant people become aware quickly and are able to do something ...

[AkshatRai07]

Should I fix #538 here or in a separate branch?

[felixhekhorn]

• many comments generalize to other cases
• I think it would be a good idea to split cf61e01 into a separate PR (since it affects lot's of files and does effectively nothing) and then put this one with just the C API on top

[felixhekhorn]

Other things I thought we could've done for #518, but can be done here:

* Add a `--release` tag in poethepoet's compile task.

* Remove all the mallocs inside the ekore and eko crate. For eg. replacing all the `vec!` with constant size arrays.

* Pass a buffer through rust_quad_ker such that it doesn't malloc at every quadrature point, so one buffer for the entire integration loop.

Do let me know your thoughts.

Should I fix #538 here or in a separate branch?

all these go into separate PRs; the --release tag and fixing #538 should have priority over this one I would say

[felixhekhorn]

For ekore:
* What should be the visibility for functions in harmonics?
For ekore_capi:
* Should we expose harmonics? If yes then which functions?
My initial thought is to make only the harmonics::cache::Cache public, expose that through c_api. If required, we can change visibility of others later.

I would say, let's follow our established logic, that we first expose as little as possible. In this case we have to expose Cache since it is an explicit parameter, but everything else should be hidden if possible. Since C and Rust should have the same surface, this implies making Rust more private. Since we think here for the first time seriously about visibility, I'm not worried about hiding stuff.

For ekore_capi:
* Should we expose constants.rs?

I think this requires some work, because again I want to expose as little as possible. We need to expose the PID as they are an explicit input, but I want to hide all the other things. I suggest to split the files into several parts corresponding to different math/physics and then we only expose what we need.

* In ekore crate, tests are only for the `ad_us`, should I write tests for all spacelike.rs (as they are there on the python side), and hence write the same tests for capi?

write more tests

yes, please. Please import all tests from the Python side which are not implemented yet. My idea is to use these same tests also in the CI to test the capo.

* How do we want to publish this? Do we want to push this crate to crates.io? In pineappl it is pointed out that cargo-c cannot fetch over the internet, and cargo install won't work. What I've been doing is:

cargo cinstall --release -p ekore_capi --destdir=./crates/ekore_capi/dist --prefix=/

We can then publish the dist as a GitHub release (as it is done in pineappl).

publish the way pineappl does it

yes, let's follow what other people do (and make our life easier)

* Should we write extensive doc comments, detailing the function's purpose and parameters?

and add doc comments

yes, we must. If we want to provide an API we must provide the documentation along side it else it is pointless

[felixhekhorn]

* CoPilot is stating that there might be accidental panics (by say accidentally passing a wrong value), and these panics can cause Undefined Behaviour, so we should add safeguards. This is partially True. This behavior occurred for versions before `1.81`. For `1.81+`, it results in the crashing of code instead of undefined behaiour. We can either bump the `1.70` to `1.81` in our workspace Cargo.toml or add graceful handling.

and not bump the Rust version

We can safely raise the MSRV I would say. Again, let's follow PineAPPL and they are currently at "raised MSRV to 1.91.0
raised Rust edition to 2024" - see Changelog . We also should add an MSRV workflow to our repo

CoPilot reviews Summary:

* There will be `panic` if we pass order_qcd as 0 (As you cannot initialize a 0 dimension vec). I don't think you'll ever pass that, there was no such guard in `ekore`, but still no problem in adding a guard here.

* CoPilot says to use `checked_mul`. This is just noise, there's no way `order_qcd * 9` or something will get close to max for `usize`.

My initial thought is to add safeguards

I agree - I think it is the rust way to add guards

[felixhekhorn]

What I will do later:
* Create C, C++, and maybe Fortran files for running tests.
* Add ccompile, ctest, and other relevant tasks in pyproject.toml for poethepoet.
* Add CI/CD files.

remember that Fortran has very low priority, but all the other stuff is crucial.

Now, looking at the flood of comments above I think this PR is a good start, but we should split some stuff from this PR and address problems that got revealed here first, before pushing further on this front (when we will surely discover more problems). In short, let's try to make a bunch of small PRs first instead of one big one

[AkshatRai07]

let's try to make a bunch of small PRs first instead of one big one

Let me add a TODO as a comment here:

EDIT: moved to top

[felixhekhorn]

Is this relevant for us NNPDF/pineappl@823a4f7 ?

[felixhekhorn]

some of the comments obviously generalise to other cases, if you agree

[felixhekhorn]

Please add more documentation

[felixhekhorn]

After #550 is merged, we can then add a sentence to the C API file, something like For the list of available perturbative orders see [here](ekore::bla::blub)

[felixhekhorn]

and also, please fix

warning: unresolved link to `ad_ps_gamma_ns_qcd_n3lo_len`
  --> crates/ekore_capi/src/ad_ps.rs:34:37
   |
34 | ///   least the value returned by [`ad_ps_gamma_ns_qcd_n3lo_len`].
   |                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^ no item named `ad_ps_gamma_ns_qcd_n3lo_len` in scope