New Feature: shared context in beforeAll and afterAll hooks #1770

BlueMona · 2021-08-17T19:20:36Z

Description

This is a draft failing scenario to try and describe what @charlierudolph and
@davidjgoss are describing in #1393

I'd like to work on a fix but thought it would be best to get feedback on the scenario first.

Motivation & context

Fixes #1393

Type of change

New feature (non-breaking change which adds new behaviour)

Checklist:

My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have read the CONTRIBUTING document.
I have added tests to cover my changes.
All new and existing tests passed.

@charlierudolph

This is a draft failing scenario to try and describe what @charlierudolph and @davidjgoss are describing in cucumber#1393 Feedback welcome. Co-authored-by: Matt Wynne <matt@cucumber.io>

mattwynne · 2021-08-17T19:57:41Z

The scenario we have so far doesn't address the issue around immutability. I suggest we add another scenario showing that even if the first scenario mutates the value that was set in the BeforeAll hook, the second scenario gets the original value again. (i.e. we can't use it to leak state between scenarios; each new World instance gets a fresh copy of the testRunContext)

davidjgoss · 2021-08-21T10:18:42Z

@BlueMona thanks for getting this going, and great idea to start with the scenario. I think my only suggestion would be for the stuff that we set in BeforeAll to be set on the world under its own testRunContext property rather than mixed in, so like:

Given('first step', function() {
  expect(this.testRunContext.myVar.foo).to.eql(1) 
})

Then it's like a sibling of parameters and attach that you can access from your world (including a custom one).

@mattwynne seems a good direction re immutability.

Co-authored-by: Matt Wynne <matt@mattwynne.net>

coveralls · 2022-02-24T19:56:49Z

coverage: 98.381%. remained the same
when pulling 4c269b1 on BlueMona:1393-add-world-object-reference-in-beforeall
into 96a65ca on cucumber:main.

BlueMona · 2022-02-24T20:00:17Z

The scenario is now passing, but there are a few questions / things left to do.

is object an ok type for testRunContext?
testRunContext should likely be a property of world, instead of being merged into the root of the object, which would help avoid potential conflicts
how do we implement this for the worker and how is it tested?
do we need a unit test for the testCaseRunner?
what about immutability? Should we add a scenario testing for it?

davidjgoss · 2022-02-24T21:37:26Z

Thanks for the update @BlueMona, this is looking good. To your points:

is object an ok type for testRunContext?

It's kinda the same need as worldParameters, which we type as any. Technically it should be a JSON-serialisable object so any is a bit broad, but there's no nice way to express that in TypeScript at the moment, so I'd say any is good for now.

testRunContext should likely be a property of world, instead of being merged into the root of the object, which would help avoid potential conflicts

I definitely agree, and I'd even go as far as giving each World instance a clone of the original testRunContext so that if users do modify it, the changes won't bleed into other scenarios running.

how do we implement this for the worker and how is it tested?

Good question, I'll get back to you! We run the test run hooks once for each worker which makes things...interesting. Testing of parallel stuff is generally in the feature tests.

what about immutability? Should we add a scenario testing for it?

Yep would be good to test this. Hopefully the cloning idea above would have the desired effect of (enough) immutability. Marking it readonly in TypeScript would be an easy win too.

mattwynne · 2022-02-24T23:42:29Z

@davidjgoss thanks for the quick feedback!

I'm wondering if we could achieve the immutability by copying the testRunContext when we create the merged this object in invokeStep.

i.e. something like

const world = Object.assign(this.world, { testRunContext: { ...this.testRunContext }})

WDYT?

davidjgoss · 2022-02-26T16:50:53Z

@mattwynne yeah that's basically what I was suggesting, although we'd want a deep clone since doing a spread just gives you a shallow clone.

mattwynne · 2022-02-28T18:51:43Z

@BlueMona I've extracted a function to run the BeforeAll hooks in #1933. We'll want to merge that into this PR before proceeding.

One slightly tricky part of that merge will be sharing the thisArg / testRunContext with the new function. I suggest adding it as a new argument to the function, but you could also consider making it be the return value, I suppose.

kamelbou · 2022-04-07T13:13:26Z

Hello,

Is it plan to merge this great feature soon ? :)

Thank you

….com:BlueMona/cucumber-js into 1393-add-world-object-reference-in-beforeall

davidjgoss

Looking fantastic, thanks for taking this on Mona. And sorry for the delayed review.

One thing I think we're missing is the correct typing of the BeforeAll and AfterAll functions so people can access this.testRunContext while using TypeScript without a compilation error.

If you go to test-d/hooks.ts and add:

// should allow test run context via this arg in test run hooks
BeforeAll(async function () {
  this.testRunContext.foo = 'bar'
})
AfterAll(async function () {
  this.testRunContext
})

Then that will currently fail if you run npm run types-test. I think the place to fix it is in src/support_code_library_builder/types.ts. We have explicit types for TestCaseHookFunction etc there and so it would be good to follow that pattern - although we don't need the complexity with the generics for world there, so it should be easier, something like:

export type TestRunHookFunction = (
  this: { testRunContext: TestRunContext }
) => any | Promise<any>

We'll also need to update some docs:

https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/api_reference.md#afteralloptions-fn - basic API doc for the functions
https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/hooks.md#beforeall--afterall - explain how to set test run context
https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/world.md - explain you can access test run context read only

I'm very happy to help out with any of this as required, just let me know. Thanks again!

davidjgoss · 2022-08-19T13:12:09Z

src/support_code_library_builder/world.ts

    this.attach = attach
    this.log = log
    this.parameters = parameters
+    this.testRunContext = cloneDeep(testRunContext)


This is part of the default World class. When users write a custom world they typically extend this but not always. So I think it would be better to do the cloneDeep outside of the world, probably in test_case_runner.ts::resetTestProgressData, and then we know the world will always receive a clone.

davidjgoss · 2022-08-19T13:15:05Z

src/runtime/parallel/worker.ts

@@ -37,6 +37,7 @@ export default class Worker {
  private supportCodeLibrary: ISupportCodeLibrary
  private worldParameters: any
  private runTestRunHooks: RunsTestRunHooks
+  private testRunContext = {}


This is fine to initialize here but we should type it as TestRunContext like in the non-parallel runtime.

davidjgoss · 2022-08-19T13:16:38Z

src/runtime/parallel/worker.ts

@@ -139,6 +141,7 @@ export default class Worker {
      skip,
      supportCodeLibrary: this.supportCodeLibrary,
      worldParameters: this.worldParameters,
+      testRunContext: {}, // TODO: decide what this needs to be


I think the value here should be this.testRunContext.

(At the moment, for parallel each worker runs all the BeforeAll hooks in isolation. In the future, we'll allow doing them at the coordinator level as well and that will make this more complex - but we don't have to worry about that yet!)

davidjgoss · 2022-08-19T13:54:41Z

features/before_after_all_hooks_context.feature

+    When I run cucumber-js
+    Then it passes
+
+Rule: testRunContext can't leak between scenarios


We should test the parallel runtime in another scenario here. We'd lose the guaranteed order so it gets less simple, but we could do something like 5 scenarios of

Scenario: Given the value of foo is unchanged When I change foo to {int}

And run with --parallel 2.

So in other words every scenario checks it has an unmodified context, and then tries to modify its current one. I think that would be enough to give us confidence.

davidjgoss · 2022-08-19T13:57:13Z

features/before_after_all_hooks_context.feature

+      const {AfterAll, BeforeAll, Given} = require('@cucumber/cucumber')
+      const {expect} = require('chai')
+
+      BeforeAll(function() {


Consider adding a second BeforeAll like:

BeforeAll(function() { this.testRunContext.otherVar = {bar: 2} })

And then checking it's all there in the AfterAll hook, to prove that we can incrementally build up the context across multiple BeforeAll hooks. I've no doubt this will work without any changes, but just to document it.

ksathyanm · 2023-06-29T05:20:23Z

Just curious if we have any updates or plan for this PR?

mattwynne · 2023-09-01T23:12:07Z

@BlueMona ping me if you want to pair on finishing this off. I'm here to help if you want it.

BlueMona · 2023-09-02T11:17:11Z

Let’s do it this week then!

…

On Fri, Sep 1, 2023 at 7:12 PM Matt Wynne ***@***.***> wrote: @BlueMona <https://github.com/BlueMona> ping me if you want to pair on finishing this off. I'm here to help if you want it. — Reply to this email directly, view it on GitHub <#1770 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALHLMFA6QMHT7NFI4FZ5RDXYJTVFANCNFSM5CKQKM3Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

mattwynne · 2023-09-02T14:18:07Z

Let’s do it this week then!
…

@BlueMona great!

Can you use https://calendly.com/mattwynne to book a slot?

LANDG-MartinP · 2023-12-20T16:29:18Z

any news on this PR?

davidjgoss · 2023-12-21T10:15:06Z

Just noting here the idea raised on the issue of, rather than adding a new testRunContext concept, just passing in the world parameters and allowing a BeforeAll to return a fresh world parameters if it wants to. So like:

BeforeAll(async function() {
  this.token = await authenticate(this.parameters.url, this.parameters.clientId, this.parameters.clientSecret)
  return {
    ...this.parameters,
    token
  }
})

I think this would solve all the use cases nicely and this PR could be adapted to it without much work - @BlueMona @mattwynne I'm happy to pick this up if you're time-constrained, since I'm working in this area already.

Also somewhat related is #2356 which I'm looking at today.

BlueMona · 2023-12-21T13:04:01Z

David, Yes, please do.

…

On Thu, Dec 21, 2023 at 5:15 AM David Goss ***@***.***> wrote: Just noting here the idea raised on the issue of, rather than adding a new testRunContext concept, just passing in the world parameters and allowing a BeforeAll to return a fresh world parameters if it wants to. So like: BeforeAll(async function() { this.token = await authenticate(this.parameters.url, this.parameters.clientId, this.parameters.clientSecret) return { ...this.parameters, token } }) I think this would solve all the use cases nicely and this PR could be adapted to it without much work - @BlueMona <https://github.com/BlueMona> @mattwynne <https://github.com/mattwynne> I'm happy to pick this up if you're time-constrained, since I'm working in this area already. Also somewhat related is #2356 <#2356> which I'm looking at today. — Reply to this email directly, view it on GitHub <#1770 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALHLMDE5Y34FGZHXLOQ6PTYKQD3NAVCNFSM5CKQKM32U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBWGU4TSNRQGM4Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

…ct-reference-in-beforeall

When we started this change, this new global wasn't available in Node.js yet.

davidjgoss

After some conflict untangling, this fitted together nicely with some other recent changes.

Many thanks @BlueMona for doing most of this!

aslakhellesoy · 2023-12-21T16:24:57Z

Hi @BlueMona,

Thanks for your making your first contribution to Cucumber, and welcome to the Cucumber committers team! You can now push directly to this repo and all other repos under the cucumber organization! 🍾

In return for this generous offer we hope you will:

✅ Continue to use branches and pull requests. When someone on the core team approves a pull request (yours or someone else's), you're welcome to merge it yourself.
💚 Commit to setting a good example by following and upholding our code of conduct in your interactions with other collaborators and users.
💬 Join the community Slack channel to meet the rest of the team and make yourself at home.
ℹ️ Don't feel obliged to help, just do what you can if you have the time and the energy.
🙋 Ask if you need anything. We're looking for feedback about how to make the project more welcoming, so please tell us!

On behalf of the Cucumber core team,
Aslak Hellesøy
Creator of Cucumber

iamkenos · 2023-12-21T22:41:23Z

This one's a great holiday gift. Thanks team 🖤. Happy holidays!

mattwynne · 2023-12-22T05:49:04Z

🎉 nice one @BlueMona! (and @davidjgoss 😄 )

New Feature: shared context in beforeAll and afterAll hooks

956bca7

This is a draft failing scenario to try and describe what @charlierudolph and @davidjgoss are describing in cucumber#1393 Feedback welcome. Co-authored-by: Matt Wynne <matt@cucumber.io>

mattwynne added ✅ accepted The core team has agreed that it is a good idea to fix this ⚡ enhancement Request for new functionality beginners mob candidate labels Aug 17, 2021

mattwynne mentioned this pull request Aug 17, 2021

Feature Request: Give BeforeAll a reference to the Cucumber World object #1393

Closed

mattwynne assigned BlueMona and mattwynne Aug 17, 2021

davidjgoss removed the beginners mob candidate label Oct 10, 2021

mattwynne and others added 7 commits February 4, 2022 09:28

Merge branch 'main' into 1393-add-world-object-reference-in-beforeall

dfa05e1

Merge branch 'main' into 1393-add-world-object-reference-in-beforeall

cccd234

added undefined check to given step

fcde0dc

Co-authored-by: Matt Wynne <matt@mattwynne.net>

pass a testRunContext object to beforeAll hooks

f7a119b

Co-authored-by: Matt Wynne <matt@mattwynne.net>

assign testRunContext as part of the world object

8681d2d

Co-authored-by: Matt Wynne <matt@mattwynne.net>

fix prettier linting

f3e37bf

Co-authored-by: Matt Wynne <matt@mattwynne.net>

add testRunContext to TestCaseRunner properties

a79a7de

Co-authored-by: Matt Wynne <matt@mattwynne.net>

mattwynne mentioned this pull request Feb 24, 2022

Replace contructor parameter object with regular arguments #1932

Closed

7 tasks

Merge branch 'main' into 1393-add-world-object-reference-in-beforeall

c2f1c1f

mattwynne mentioned this pull request Feb 25, 2022

Extract runTestRunHooks #1933

Merged

7 tasks

Mona Ghassemi added 3 commits April 14, 2022 14:41

Merge branch 'main' into 1393-add-world-object-reference-in-beforeall

1b4fc6d

Merge branch '1393-add-world-object-reference-in-beforeall' of github…

3b2814a

….com:BlueMona/cucumber-js into 1393-add-world-object-reference-in-beforeall

WIP: namespace testRunContext in world

913afe8

davidjgoss requested changes Aug 19, 2022

View reviewed changes

davidjgoss added 17 commits December 21, 2023 14:43

resolve conflicts

a7a4ad4

Merge remote-tracking branch 'upstream/main' into 1393-add-world-obje…

d093272

…ct-reference-in-beforeall

lint and formatting

ef4205d

use structuredClone over cloneDeep

e92f69c

When we started this change, this new global wasn't available in Node.js yet.

revert lockfile to match main

8d2078b

update scenario for tweaked approach

16b2ef9

get happy path scenario passing

573a710

prove it accumulates with multiple BeforeAll

1eec65c

make it work for parallel runtime

113e5fc

lint and format fixes

36f2154

trim some artifacts of refactors

30278a4

tweak feature file wording

1e4f89d

add typescript hints

aea8819

documentation

cc08890

update CHANGELOG.md

12ce703

attribution

e70ff21

add example to docs

4c269b1

davidjgoss approved these changes Dec 21, 2023

View reviewed changes

davidjgoss merged commit 739fca6 into cucumber:main Dec 21, 2023
6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Feature: shared context in beforeAll and afterAll hooks #1770

New Feature: shared context in beforeAll and afterAll hooks #1770

BlueMona commented Aug 17, 2021 •

edited by davidjgoss

Loading

mattwynne commented Aug 17, 2021

davidjgoss commented Aug 21, 2021

coveralls commented Feb 24, 2022 •

edited

Loading

BlueMona commented Feb 24, 2022

davidjgoss commented Feb 24, 2022

mattwynne commented Feb 24, 2022

davidjgoss commented Feb 26, 2022

mattwynne commented Feb 28, 2022

kamelbou commented Apr 7, 2022

davidjgoss left a comment •

edited

Loading

davidjgoss Aug 19, 2022

davidjgoss Aug 19, 2022

davidjgoss Aug 19, 2022 •

edited

Loading

davidjgoss Aug 19, 2022

davidjgoss Aug 19, 2022

ksathyanm commented Jun 29, 2023

mattwynne commented Sep 1, 2023

BlueMona commented Sep 2, 2023 via email

mattwynne commented Sep 2, 2023

LANDG-MartinP commented Dec 20, 2023

davidjgoss commented Dec 21, 2023

BlueMona commented Dec 21, 2023 via email

davidjgoss left a comment

aslakhellesoy commented Dec 21, 2023

iamkenos commented Dec 21, 2023 •

edited

Loading

mattwynne commented Dec 22, 2023 •

edited

Loading

New Feature: shared context in beforeAll and afterAll hooks #1770

New Feature: shared context in beforeAll and afterAll hooks #1770

Conversation

BlueMona commented Aug 17, 2021 • edited by davidjgoss Loading

Description

Motivation & context

Type of change

Checklist:

mattwynne commented Aug 17, 2021

davidjgoss commented Aug 21, 2021

coveralls commented Feb 24, 2022 • edited Loading

BlueMona commented Feb 24, 2022

davidjgoss commented Feb 24, 2022

mattwynne commented Feb 24, 2022

davidjgoss commented Feb 26, 2022

mattwynne commented Feb 28, 2022

kamelbou commented Apr 7, 2022

davidjgoss left a comment • edited Loading

Choose a reason for hiding this comment

davidjgoss Aug 19, 2022

Choose a reason for hiding this comment

davidjgoss Aug 19, 2022

Choose a reason for hiding this comment

davidjgoss Aug 19, 2022 • edited Loading

Choose a reason for hiding this comment

davidjgoss Aug 19, 2022

Choose a reason for hiding this comment

davidjgoss Aug 19, 2022

Choose a reason for hiding this comment

ksathyanm commented Jun 29, 2023

mattwynne commented Sep 1, 2023

BlueMona commented Sep 2, 2023 via email

mattwynne commented Sep 2, 2023

LANDG-MartinP commented Dec 20, 2023

davidjgoss commented Dec 21, 2023

BlueMona commented Dec 21, 2023 via email

davidjgoss left a comment

Choose a reason for hiding this comment

aslakhellesoy commented Dec 21, 2023

iamkenos commented Dec 21, 2023 • edited Loading

mattwynne commented Dec 22, 2023 • edited Loading

BlueMona commented Aug 17, 2021 •

edited by davidjgoss

Loading

coveralls commented Feb 24, 2022 •

edited

Loading

davidjgoss left a comment •

edited

Loading

davidjgoss Aug 19, 2022 •

edited

Loading

iamkenos commented Dec 21, 2023 •

edited

Loading

mattwynne commented Dec 22, 2023 •

edited

Loading