Skip to content

IMAP-Lo systematic error calculation fixes#3315

Merged
tmplummer merged 2 commits into
IMAP-Science-Operations-Center:devfrom
vineetbansal:vb/issue3314
Jul 15, 2026
Merged

IMAP-Lo systematic error calculation fixes#3315
tmplummer merged 2 commits into
IMAP-Science-Operations-Center:devfrom
vineetbansal:vb/issue3314

Conversation

@vineetbansal

@vineetbansal vineetbansal commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

Closes #3314

Overview

  • Updated uncertainty values in geometric factor csvs (v4) to be 1/1000th of previous values (which were in [eV]).
  • calculate_intensities and calculate_backgrounds now propagate the asymmetric GF uncertainty with the exact reciprocal relation (J·G/(G ∓ dG), emit _sys_err_plus / _sys_err_minus bounds, and store the geometric mean sqrt(minus·plus) in the existing _sys_err variable. Applied to ena_intensity, bg_rate, and bg_intensity.
  • Made changes to tests to match the correct formulas.

EDIT: Removed the GF correction for bg_rate_sys_err since it is independent of GF, as per the second commit (and comment below).

@vineetbansal vineetbansal marked this pull request as ready for review June 22, 2026 19:53
@vineetbansal vineetbansal changed the title Vb/issue3314 IMAP-Lo systematic error calculation fixes Jun 22, 2026
@vineetbansal

Copy link
Copy Markdown
Collaborator Author

Please wait on merging this PR till @frahmanif has had a chance to sign off on this ...

@vineetbansal

Copy link
Copy Markdown
Collaborator Author

Also tagging @mitchellshen and @hi1011

@frahmanif

Copy link
Copy Markdown

As we discussed, background rates and associated uncertainty (systematic uncertainty) are read from the hydrogen-background ancillary file. GF values are not involved in the rates calculations and thus the asymmetry in the systematic uncertainty error bars do not affect the bg rates uncertainties. In the next step, when we calculate bg flux and systematic uncertainty, the GF values and uncertainties are then used and we expect the asymmetry to be considered.

The algorithm document (Appendix A, eq 64) specifies the background rate
systematic as the exposure time weighted average of the per-pset systematic
error.

The systematic is a single symmetric value, so _plus and _minus are set equal
to it for now.

Note that the systematic errors are all 0s, since no constant has been
introduced yet to populate this (to be Imap-Lo team supplied) value.
@vineetbansal

vineetbansal commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator Author

@frahmanif, @mitchellshen, @hi1011 - As per Fatemeh's input as well as what I could understand from the algorithm document, I've tweaked the calculation of bg_rate_sys_err (as we're calling it in the code) to match eq 64 of the algorithm document, which calls it systematic uncertainty:

$$\langle \sigma R^B \rangle_{\text{pxl}} = \frac{\sum_i \sigma R^B_i , \tau_i}{\sum_i \tau_i}$$

The delta term = $\langle (\delta R^B)^2 \rangle_{\text{pxl}}$ is bg_rate_stat_uncert in the code/CDFs.

So the terms we're using in the code do have the "stat" or "sys" prefix, but call it the "statistical uncertainty" and "systematic error" respectively, while the document calls the latter "systematic uncertainty". This is a carryover from when the code was written and attributes in the CDF created, so hopefully it is not a big deal.

So the sigma row in files like imap_lo_H_background.csv corresponds to bg_rate_stat_uncert in the product CDFs. The fact that the bg_rate_sys_err in all CDFs is all 0 is because the dropbox pipeline doesn't put the actual values anywhere (ideally there would be a 3rd row after each pair of "rate" and "sigma" row), so the SDC pipeline doesn't have it either.

In fact an existing test even asserts this:

# Systematic error is always zero

    # Systematic error is always zero
    np.testing.assert_array_equal(err.values, 0)

But that's a separate issue and we can tackle it in another PR. This PR is now done.

@vineetbansal vineetbansal requested a review from tmplummer July 14, 2026 21:40

@tmplummer tmplummer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks fine to me.

@tmplummer tmplummer merged commit 0a6de2d into IMAP-Science-Operations-Center:dev Jul 15, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Imap-Lo systematic error calculations

3 participants