fix: preserve literal paths in Git change discovery#618
Draft
teilomillet wants to merge 1 commit into
Draft
Conversation
Read NUL-delimited Git output so incremental updates keep non-ASCII and arrow-containing filenames intact while retaining destination-only rename and copy semantics.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked issue
No exact issue exists for these Git discovery functions; the defect was found with Ordeal while auditing the current Python Trending surface. #497 is related Unicode-path work, but it concerns generated host configuration rather than incremental Git output. PR #566 overlaps these subprocess blocks while addressing decode crashes, but its replacement decoding does not preserve Git's quoted paths.
What & why
code-review-graph keeps a map of a repository so its review tools know which files changed and what those changes may affect. When someone edited a valid source file such as
café.py, Git sometimes reported the name in an encoded form. The updater then searched for that encoded text instead of the real filename, so it could silently miss the changed file. This patch reads Git's unambiguous machine format, allowing the updater to find the real file while preserving its existing rename and copy behavior.Before this change, both the working-tree and commit-diff paths produced:
['"caf\\303\\251.py"']instead of:
["café.py"]Downstream incremental parsing then looked up a filename that did not exist and could silently miss the changed source file. The line-based status parser also treated a literal
->inside a filename as a rename delimiter and truncated that path.Git quotes unusual paths in its newline-delimited human-readable output. This change requests NUL-delimited output from
git diff --name-only -zandgit status --porcelain=v1 -z, keeps the subprocess output as bytes, and decodes each complete path withos.fsdecode. Rename and copy records retain the existing destination-only behavior by skipping the following source-path record.The changelog now records both corrected behaviors. Ordinary ASCII paths and the existing rename behavior remain unchanged; no schema or migration change is required.
Ordeal evidence
A contract-specific Ordeal
fuzzproperty generated valid non-ASCII Python filenames and required each Git discovery API to return the literal path.get_staged_and_unstaged: 12 examples, minimized failure{"filename": "café.py"}.get_changed_files: 12 examples, minimized failure{"filename": "café.py"}.['"caf\\303\\251.py"']from both APIs.The semantic-delta check also covers simple paths, destination-only rename and copy records, and a literal
path -> literal.pyfilename.How it was tested
Results:
code-review-graph buildparsed all 179 files, anddetect-changes --base origin/main --briefanalyzed the intended four-file patch.The local run used macOS and Python 3.13. Repository CI will provide the configured Python 3.10-3.13 matrix. The coverage run emitted existing SQLite
ResourceWarnings in unrelated parser, visualization, and wiki tests; they did not fail the suite.Checklist
uv run pytest tests/ --tb=short -quv run ruff check code_review_graph/uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optionalCHANGELOG.md)