From 6b1a778fe10051d779beb22101825b5a72dcddae Mon Sep 17 00:00:00 2001 From: Denis Bilenko Date: Fri, 7 Feb 2025 17:17:50 +0100 Subject: [PATCH] Fix flaky acceptance test (#2310) ## Changes Replace timestamps with fixed string before output is sorted (and before test runner replacements are applied). Otherwise the test sometimes fails with error below. Note, timestamps themselves do not show it, because they were replaced. ``` --- FAIL: TestAccept/bundle/debug (0.78s) acceptance_test.go:404: Diff: --- bundle/debug/out.stderr.parallel.txt +++ /var/folders/5y/9kkdnjw91p11vsqwk0cvmk200000gp/T/TestAcceptbundledebug1859985035/001/out.stderr.parallel.txt @@ -8,8 +8,8 @@ 10:07:59 Debug: ApplyReadOnly pid=12345 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:files_to_sync 10:07:59 Debug: ApplyReadOnly pid=12345 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:folder_permissions 10:07:59 Debug: ApplyReadOnly pid=12345 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:validate_sync_patterns -10:07:59 Debug: Path /Workspace/Users/[USERNAME]/.bundle/debug/default/files has type directory (ID: 0) pid=12345 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:files_to_sync 10:07:59 Debug: non-retriable error: pid=12345 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:files_to_sync sdk=true +10:07:59 Debug: Path /Workspace/Users/[USERNAME]/.bundle/debug/default/files has type directory (ID: 0) pid=12345 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:files_to_sync < {} pid=12345 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:files_to_sync sdk=true < {} pid=12345 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:files_to_sync sdk=true < } pid=12345 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:files_to_sync sdk=true ``` ## Tests Running `hyperfine --min-runs 10 'go test ../.. -run ^TestAccept$/^bundle$/^debug$ -count=1' --show-output` detects flakiness on main but not with this PR. --- acceptance/bundle/debug/script | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/acceptance/bundle/debug/script b/acceptance/bundle/debug/script index 1b4cfc8f0..005a1a341 100644 --- a/acceptance/bundle/debug/script +++ b/acceptance/bundle/debug/script @@ -1,4 +1,4 @@ $CLI bundle validate --debug 2> full.stderr.txt grep -vw parallel full.stderr.txt > out.stderr.txt -grep -w parallel full.stderr.txt | sort_lines.py > out.stderr.parallel.txt +grep -w parallel full.stderr.txt | sed 's/[0-9]/0/g' | sort_lines.py > out.stderr.parallel.txt rm full.stderr.txt