Allow symlinks to test cases with other output validator flags and…#276
Allow symlinks to test cases with other output validator flags and…#276Tagl wants to merge 1 commit intoKattis:developfrom
Conversation
…rn if output validators fails to compile.
| self.error(f"Symbolic link '{nicepath}' must have a corresponding link for answer file") | ||
| return False | ||
| if self.reuse_result_from is None: | ||
| if not nicepath.startswith('data'): |
There was a problem hiding this comment.
Isn't the previous check more accurate?
There was a problem hiding this comment.
Sorry for late reply, but:
- this covers e.g.
data2too nicepathis the path prior to symlink resolution
| if res.verdict != 'AC': | ||
| return res | ||
| else: | ||
| self.warning(f"Compilation failed for {val.name}: {compile_res[1]}") |
There was a problem hiding this comment.
Won't this result in a lot of warning spam (one log line per submission per test case)? IIRC we already give an error in an earlier phase.
There was a problem hiding this comment.
I had an error and got no information before adding this. It was very confusing.
I will double check though.
There was a problem hiding this comment.
I've fixed this in a less spammy way in #319 where if validators fail to compile (or the problem uses the default validator and we can't find it), we stop validation before running any submissions.
|
I've reworked the handling of testcase reuse completely in #411. Symlinks now have no special treatment - we determine when we can safely reuse results and do so when test cases are identical. Symlinks are now allow between non-identical test cases (e.g., to save space). Closing this PR, as I feel like the approach in #411 is a better long-term fix. |
3 participants
..warn if output validators fails to compile. Closes #244
Note this may be a breaking change for some problem based on discussion in issue.
However that problem is not being validated "correctly" since the output validator generally should behave differently with different arguments passed to it, rendering reuse impossible.