Add true runcards for comparefits reports

[andrpie]

Addresses #402

[scarlehoff]

I'm a bit confused at how this addresses the issue?

Does it mean that the lockfile was not enough to reproduce the comparefit?

[andrpie]

Perhaps I've misunderstood what the issue is!
What I thought needed fixing was that the vp-comparefits would save the runcard template as output/input/runcard.yaml, whereas a typical validphys script just copies the specified runcard there. The 'actual' runcard used by vp-comparefits is the template combined with extra arguments (such as the current fit, reference fit, meta etc.). So here, I essentially create this runcard and save in place of the template.

I'm actually unsure what the relationship between the runcard and the lockfile is.

[scarlehoff]

We want to make sure the reports are reproducible. To that end, validphys creates a lockfile which in principle contains everything.
However, the vp-comparefit script has quite a few command line argument, so we want to check the lockfile is actually complete, if yes just close the issue, if not, the missing pieces need to be added.

One should be able to get exactly the same report with validphys lockfile.yaml

[andrpie]

Dear @scarlehoff
I see what you mean now. So there are basically two issues:

1. Making sure that lockfile.yaml reproduces the report. As far as I understand, this works even without this PR, because vp-comparefits takes the template + user arguments and creates the 'full runcard' by combining those two. It is then burned into lockfile.yaml after config and resource checks, ensuring the reproducibility. I also tested this by running validphys lockfile.yaml to see if I reproduce the exact comparison report I had created a few days ago. It worked!
2. vp-comparefits saves only the runcard template in the input folder, rather than the true runcard that is behind the comparison. This PR fixes this issue.

[scarlehoff]

If 1 is correct, then 2 is not necessary.

But the question is whether 1 is correct for "non standard arguments". if it is (and from your msg it looks like that's the case) then no need to do anything else.

[andrpie]

I'll soon follow up with the code snippets that I believe ensure 1 is correct!