Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(screenshot): reject pixel match process on exit #5167

Merged
merged 1 commit into from
Dec 13, 2023

Conversation

christian-bromann
Copy link
Member

What is the current behavior?

Currently if we for some reason throw an error within the pixel-match process, the test stales because the promise that waits for the process to emit a "done" message never resolves

refs #4866
STENCIL-962

What is the new behavior?

Listen on the exit event and reject the promise. This doesn't completely addresses the issue highlighted in #4866 but fixes one problem of it.

Does this introduce a breaking change?

  • Yes
  • No

Testing

To test the new behavior:

  • check out the example project from mentioned issue
  • run yarn run test: all should be fine
  • change the component and rerun: now the test process stales

After checking out this branch, you will see that the error is now properly propagated.

Other information

I am still not sure why the test fails. It seems like the picture is far larger in byte size than mine created on the Mac. I assume here that screenshots made in the CI machine are just done in a different quality which pixelmatch doesn't accompass for.

Copy link
Contributor

github-actions bot commented Dec 13, 2023

--strictNullChecks error report

Typechecking with --strictNullChecks resulted in 1301 errors on this branch.

Unfortunately, it looks like that's an increase of 1 over main 😞.

Violations Not on `main` (may be more than the count above)
Path Location Error Message
src/screenshot/screenshot-compare.ts (172, 7) TS18047

reports and statistics

Our most error-prone files
Path Error Count
src/dev-server/index.ts 37
src/mock-doc/serialize-node.ts 36
src/dev-server/server-process.ts 32
src/compiler/output-targets/dist-lazy/generate-lazy-module.ts 22
src/compiler/prerender/prerender-main.ts 22
src/testing/puppeteer/puppeteer-element.ts 22
src/runtime/client-hydrate.ts 20
src/screenshot/connector-base.ts 19
src/runtime/vdom/vdom-render.ts 18
src/compiler/config/test/validate-paths.spec.ts 16
src/dev-server/request-handler.ts 15
src/compiler/prerender/prerender-optimize.ts 14
src/compiler/sys/stencil-sys.ts 14
src/compiler/transpile/transpile-module.ts 14
src/runtime/vdom/vdom-annotations.ts 14
src/sys/node/node-sys.ts 14
src/compiler/prerender/prerender-queue.ts 13
src/compiler/sys/in-memory-fs.ts 13
src/runtime/connected-callback.ts 13
src/runtime/set-value.ts 13
Our most common errors
Typescript Error Code Count
TS2345 376
TS2322 373
TS18048 286
TS18047 103
TS2722 37
TS2532 30
TS2531 23
TS2454 14
TS2352 12
TS2790 10
TS2769 8
TS2538 8
TS2344 6
TS2416 6
TS2493 3
TS2488 2
TS18046 2
TS2684 1
TS2430 1

Unused exports report

There are 14 unused exports on this PR. That's the same number of errors on main, so at least we're not creating new ones!

Unused exports
File Line Identifier
src/runtime/bootstrap-lazy.ts 21 setNonce
src/screenshot/screenshot-fs.ts 18 readScreenshotData
src/testing/testing-utils.ts 198 withSilentWarn
src/utils/index.ts 145 CUSTOM
src/utils/index.ts 269 normalize
src/utils/index.ts 7 escapeRegExpSpecialCharacters
src/compiler/app-core/app-data.ts 25 BUILD
src/compiler/app-core/app-data.ts 115 Env
src/compiler/app-core/app-data.ts 117 NAMESPACE
src/compiler/fs-watch/fs-watch-rebuild.ts 123 updateCacheFromRebuild
src/compiler/types/validate-primary-package-output-target.ts 61 satisfies
src/compiler/types/validate-primary-package-output-target.ts 61 Record
src/testing/puppeteer/puppeteer-declarations.ts 485 WaitForEventOptions
src/compiler/sys/fetch/write-fetch-success.ts 7 writeFetchSuccessSync

@christian-bromann christian-bromann marked this pull request as ready for review December 13, 2023 02:16
@christian-bromann christian-bromann requested a review from a team as a code owner December 13, 2023 02:16
@christian-bromann
Copy link
Member Author

With this patch the test is now failing with the right error message propagated:

[47:24.8]  testing e2e files
[47:24.8]  comparing against master screenshots
[47:25.3]  build, bug-stencil-e2e-clip, dev mode, started ...
[47:25.5]  transpile started ...
[47:26.4]  transpile finished in 896 ms
[47:26.4]  copy started ...
[47:26.4]  generate custom elements + source maps started ...
[47:26.4]  generate lazy + source maps started ...
[47:26.5]  copy finished (0 files) in 94 ms
[47:26.5]  generate custom elements + source maps finished in 144
           ms
[47:26.7]  generate lazy + source maps finished in 247 ms
[47:26.7]  build finished in 1.34 s

[47:26.7]  jest args: --e2e --screenshot --max-workers=8
 FAIL  src/components/my-component/my-component.e2e.ts
  ● my-component › looks the same

    Pixelmatch process exited with code 1: node:events:495
          throw er; // Unhandled 'error' event
          ^

    Error: Image data size does not match width/height.

      37 |         throw new Error('Image sizes do not match.');
      38 |
    > 39 |     if (img1.length !== width * height * 4) throw new Error('Image data size does not match width/height.');
         |                                                   ^
      40 |
      41 |     options = Object.assign({}, defaultOptions, options);
      42 |

      at pixelmatch (../../../Users/christian.bromann/Sites/Ionic/projects/stencil/screenshot/pixel-match.js:39:51)
      at exports.PNG.doneReading (../../../Users/christian.bromann/Sites/Ionic/projects/stencil/screenshot/pixel-match.js:2800:34)
      at exports.PNG.<anonymous> (../../../Users/christian.bromann/Sites/Ionic/projects/stencil/screenshot/pixel-match.js:2636:12)
      at ParserAsync._complete (../../../Users/christian.bromann/Sites/Ionic/projects/stencil/screenshot/pixel-match.js:1670:8)
      at module.exports.complete (../../../Users/christian.bromann/Sites/Ionic/projects/stencil/screenshot/pixel-match.js:769:12)
      at Filter._reverseFilterLine (../../../Users/christian.bromann/Sites/Ionic/projects/stencil/screenshot/pixel-match.js:747:10)
      at ChunkStream._processRead (../../../Users/christian.bromann/Sites/Ionic/projects/stencil/screenshot/pixel-match.js:427:13)
      Emitted 'error' event on  instance at:
      at ChunkStream._process (../../../Users/christian.bromann/Sites/Ionic/projects/stencil/screenshot/pixel-match.js:454:10)
      at ChunkStream.write (../../../Users/christian.bromann/Sites/Ionic/projects/stencil/screenshot/pixel-match.js:324:8)
      at Inflate.<anonymous> (../../../Users/christian.bromann/Sites/Ionic/projects/stencil/screenshot/pixel-match.js:1602:9)
      Node.js v18.18.0
      at ChildProcess.<anonymous> (../../../Users/christian.bromann/Sites/Ionic/projects/stencil/testing/index.js:2337:82)

@christian-bromann christian-bromann added this pull request to the merge queue Dec 13, 2023
Merged via the queue into main with commit c2ee40d Dec 13, 2023
120 checks passed
@christian-bromann christian-bromann deleted the cb/fix-pixel-match-fork branch December 13, 2023 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: getMismatchedPixels timeout when using clip options in compareScreenshot
3 participants