Documentation, example projects

[alexvoss]

Hi,

I am trying to use CucumberSwift in a project and am struggling a little with the documentation. I know it does not help that I am new to developing in Swift and SwiftUI but I have to write tests for a project that is already in flight, so I need to catch up rapid. What would help would be to have more examples of step definitions so I know I can read the documentation for Gherkin on the Cucumber pages but have documentation and examples for how to write my tests in Swift.

One thing that would help would be to have a link to the CucumberSwiftSample more prominently displayed. The info there is invaluable for anyone getting started. The Installation page links to it but only at the bottom.

Also the sample contains information not just about installing using various package managers but also about usage. This is duplicated for different package managers. Could we refactor this and make sure that the usage stuff is integrated with the main wiki? The example step definitions use regex syntax but do not provide an actual match group. That is the bit I am wondering about at the moment. Would be good to see a worked example with the syntax for creating a matching group as well as how data from the scenario are used in a test.

Tyler, do you have example OSS projects where CucumberSwift is being used?

I am happy to engage on this over the next few days and even lend a hand if that helps. Happy to discuss longer-term engagement as well. I am currently acting as a test engineer in a Kotlin-Multiplatform+SwiftUI project and am trying to convince people to use BDD.

Cheers,

Alex

[Tyler-Keith-Thompson]

This is a good call out and I would welcome the help. I think that perhaps the best bet would be to use DocC and remove the wiki entirely. This would allow CucumberSwift to have one spot for tutorials with step-by-step for those that want it and straight up code documentation as well.

Perhaps after looking at how useful DocC is it'd be worth considering whether the sample project serves a purpose.

[Tyler-Keith-Thompson]

The docc branch has the first tutorial in place for getting started with CocoaPods. This is mostly a port from CucumberSwiftSample but perhaps a little nicer because of the format with images and code snippets. I'll be porting over Carthage and SPM in the near future.

After all that's done I'll start transitioning Wiki topics.

[alexvoss]

Thanks a lot, this looks great. The biggest problem I had was to convince cocoapods to upgrade from 3.3.6 to 3.3.11 on an M1 machine. Turns out it is a good idea to install it using Homebrew so it does not use the Apple-provided Ruby. Anyway, back to the documentation. Some suggestions:

• add a note that people may want to pod repo updateso they do not get an old version if it has been a while since they did that?
• a newly created project on Xcode 13.3.1 does not have an Info.plist file, see: https://useyourloaf.com/blog/xcode-13-missing-info.plist/ Perhaps add a note on what to do instead?
• In the example step definitions, can you add a case where the matches are used as well as one where the step argument would be used? That will set people on their way to grabbing the results of regular expressions and step arguments. Add a link to the Cucumber documentation to explain those concepts?
• Btw. does CucumberSwift support Cucumber expressions?

Thanks again for the great work. I love how umlaute and stuff are now supported and how the skeletons for step definitions are generated! I had started with 3.3.6 and since then there has been a great improvement. I will try out CucumberSwift in our live project tomorrow.

[Tyler-Keith-Thompson]

Cucumber expressions are not supported just yet. I've always been a little hesitant since regular expressions are better at describing matches. They also become less cryptic and more approachable with swift pitches to add a regex DSL or libraries that do this today

It is on my list though, since I'd like to be compliant with the majority of other Cucumber implementations. It's just been rather low on the list.

Good call on those additions, I'll get right on it. FWIW the generated step stubs do showcase how to match if it finds certain clues in the Gherkin. I believe quoted strings in Gherkin cause it to generate a stub that matches, likewise integers will cause it to generate a stub that matches.

[alexvoss]

One more thing. To get the CucumberSwift tests to show up in the test navigator in Xcode it seems I need to run the all the 'normal' UI tests in my project, which takes rather a long time. Is there a way to get the BDD tests to show without doing this? If not, might it make sense to have a separate target that contains only a trivial UI test in addition to the BDD tests? The reason I mention this here is because all too often tutorials assume a trivial starter-project but I had to start from an existing project, which may not be that unusual?

[Tyler-Keith-Thompson]

TL;DR yeah, just stick them in a different target. That target can be a UI testing target or a unit testing target depending on how you want to prove the Gherkin statements. If it is a UI testing target just delete the template code. I don't know why they chose to have a template that takes eons to run as it loads your app 10 times.

CucumberSwift is just a test runner that ties into XCTest, so you do have to run an XCTest target for it to work. That test target can contain other things, but I feel like that starts to pollute your tests anyways, it's nice to have test targets serve some clear purpose and "BDD tests" fits in my mental model as it's own justified purpose.

Also be aware that until I figure out how and why UI testing targets parallelize differently than UI testing targets test parallelization is not yet supported with CucumberSwift. Unfortunately, Apple doesn't document this or give super easy tie-ins. I've already got a lead based on some other test runners out there, but haven't had the time to actually fix it yet.

[alexvoss]

Another thing: I am not entirely sure I understand how to use the FeaturesPath. I have a larger Project (Kotlin Multiplatform) into which the Xcode project is embedded. The Feature files are written for the overall project and live as a Git Submodule in the outer project in a directory ext/MyFeatures. I have an Info.plist file in my UI Test target. Am I right that the FeaturesPath´ would need to be ../../ext/MyFeatures`? The first dots bring me out of the UI Test target to the wider Xcode project and the second to the outer project. When I run the tests, I only get testGherkin() and no indication that no Features were found. Any ideas?

[Tyler-Keith-Thompson]

TL;DR It should be relative to your folder reference in the bundle, not the file on disk. So if the folder you referenced was ext then FeaturesPath would be ext/MyFeatures. If the folder you referenced was MyFeatures then it should simply be MyFeatures.

Longer version: Xcode is a little deceptive about how it runs tests in general, they're executing inside your derived data folder, not where your source is. A folder reference works a little bit like a symlink in that the project (or workspace) will keep track of updates you make to that folder on disk. However, when tests execute that folder is copied over to the derived data folder in a structure that matches the bundle.

One final note: Version 3.3.11 should give you feedback that no features were found in the form of a crash. It wasn't reasonable to give that feedback with an XCTest Failure, so a regular assertion was necessary.

[alexvoss]

Ok, I managed to set up a Test Plan to specify which non-Cucumber tests to run, restricting it to a trivial one. This has simplified my project configuration and it looks like all is well. I am actually productive writing step definitions at last! Happy camper...
The functionality to filter by tags is also particularly interesting in this regard. I do wonder if it should also be included in the example since it is commonly used, no?

[Tyler-Keith-Thompson]

I've still got a lot too add, but here's a deployed version of DocC: https://tyler-keith-thompson.github.io/CucumberSwift/documentation/cucumberswift