Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing guidelines update #2115

Merged
merged 6 commits into from
Oct 17, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 29 additions & 16 deletions _data/sidebars/scos_dev_sidebar.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4871,30 +4871,43 @@ entries:
- title: Testing guidelines
url: /docs/scos/dev/guidelines/testing-guidelines/testing-guidelines.html
nested:
- title: Testing concepts
url: /docs/scos/dev/guidelines/testing-guidelines/testing-concepts.html
- title: Code coverage
url: /docs/scos/dev/guidelines/testing-guidelines/code-coverage.html
- title: Test framework
url: /docs/scos/dev/guidelines/testing-guidelines/test-framework.html
- title: Setting up tests
url: /docs/scos/dev/guidelines/testing-guidelines/setting-up-tests.html
- title: Publish and Synchronization testing
url: /docs/scos/dev/guidelines/testing-guidelines/publish-and-synchronization-testing.html
- title: Test helpers
url: /docs/scos/dev/guidelines/testing-guidelines/test-helpers.html
url: /docs/scos/dev/guidelines/testing-guidelines/test-helpers/test-helpers.html
nested:
- title: Available test helpers
url: /docs/scos/dev/guidelines/testing-guidelines/test-helpers/available-test-helpers.html
- title: Test helpers best practices
url: /docs/scos/dev/guidelines/testing-guidelines/test-helpers/test-helpers-best-practices.html
- title: Executing tests
url: /docs/scos/dev/guidelines/testing-guidelines/executing-tests/executing-tests.html
nested:
- title: Execute tests
url: /docs/scos/dev/guidelines/testing-guidelines/executing-tests/execute-tests.html
- title: Test console commands
url: /docs/scos/dev/guidelines/testing-guidelines/executing-tests/test-console-commands.html
- title: Test the publish and synchronization process
url: /docs/scos/dev/guidelines/testing-guidelines/executing-tests/test-the-publish-and-synchronization-process.html
- title: Testify
url: /docs/scos/dev/guidelines/testing-guidelines/testify.html
- title: Executing tests
url: /docs/scos/dev/guidelines/testing-guidelines/executing-tests.html
- title: Test framework
url: /docs/scos/dev/guidelines/testing-guidelines/test-framework.html
- title: Code coverage
url: /docs/scos/dev/guidelines/testing-guidelines/code-coverage.html
- title: Testing concepts
url: /docs/scos/dev/guidelines/testing-guidelines/testing-concepts.html
- title: Data builders
url: /docs/scos/dev/guidelines/testing-guidelines/data-builders.html
- title: Testing best practices
url: /docs/scos/dev/guidelines/testing-guidelines/testing-best-practices.html
- title: Available test helpers
url: /docs/scos/dev/guidelines/testing-guidelines/available-test-helpers.html
- title: Testing console commands
url: /docs/scos/dev/guidelines/testing-guidelines/testing-console-commands.html
url: /docs/scos/dev/guidelines/testing-guidelines/testing-best-practices/testing-best-practices.html
nested:
- title: Testing concepts
url: /docs/scos/dev/guidelines/testing-guidelines/testing-best-practices/testing-concepts.html
- title: Best practices for effective testing
url: /docs/scos/dev/guidelines/testing-guidelines/testing-best-practices/best-practices-for-effective-testing.html
- title: Test types best practices
url: /docs/scos/dev/guidelines/testing-guidelines/testing-best-practices/test-types-best-practices.html
- title: Project development guidelines
url: /docs/scos/dev/guidelines/project-development-guidelines.html
- title: Data processing guidelines
Expand Down
7 changes: 7 additions & 0 deletions docs/scos/dev/guidelines/testing-guidelines/code-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,10 @@ Keep in mind, that if you execute the tests with the coverage report, it will ta
Currently, we don't support code coverage for Presentation (Acceptance) tests out of the box.

{% endinfo_block %}

## Next steps

- [Configure your testing environment]
- [Set up an organization of your tests](/docs/scos/dev/guidelines/testing-guidelines/setting-up-tests.html)


Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Executing tests
title: Execute tests
description: Lean what commands you can use to execute your tests.
last_updated: Jun 16, 2021
template: concept-topic-template
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
title: Executing tests
description: Learn how to execute regular tests, how to test console commands and the publish and synchronize process.
last_updated: Aug 28, 2023
template: concept-topic-template
---

This section guides you through yhe execution of tests in your Spryker-based project. Specifically, it focuses on the following topics:

- [How to execute execute tests](/docs/scos/dev/guidelines/testing-guidelines/executing-tests/execute-tests.html).
- [How to test console commands](/docs/scos/dev/guidelines/testing-guidelines/executing-tests/test-console-commands.html).
- [How to test the Publish and Synchronization process](/docs/scos/dev/guidelines/testing-guidelines/executing-tests/test-the-publish-and-synchronization-process.html).
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Testing console commands
title: Test console commands
description: How to test console commands.
last_updated: Jan 12, 2022
template: concept-topic-template
Expand All @@ -26,6 +26,8 @@ related:
link: docs/scos/dev/guidelines/testing-guidelines/testing-best-practices.html
- title: Testing concepts
link: docs/scos/dev/guidelines/testing-guidelines/testing-concepts.html
redirect_from:
- /docs/scos/dev/guidelines/testing-guidelines/testing-console-commands.html
---

Spryker supports several [test helpers](/docs/scos/dev/guidelines/testing-guidelines/test-helpers.html) to assist you in testing your project. This article provides details of how to test console commands with ConsoleHelper.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Publish and Synchronization testing
title: Test the Publish and Synchronization process
description: Learn how you can test Publish and Synchronization
last_updated: Jun 16, 2021
template: concept-topic-template
Expand All @@ -13,6 +13,7 @@ redirect_from:
- /v6/docs/publish-and-synchronization-testing
- /v6/docs/en/publish-and-synchronization-testing
- /docs/scos/dev/guidelines/testing/publish-and-synchronization-testing.html
- /docs/scos/dev/guidelines/testing-guidelines/publish-and-synchronization-testing.html
related:
- title: Available test helpers
link: docs/scos/dev/guidelines/testing-guidelines/available-test-helpers.html
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ Check out the organization within the [tests/PyzTest/](https://github.com/spryke

Together with the [root configuration](/docs/scos/dev/guidelines/testing-guidelines/test-framework.html#configuration), you are now able to organize your tests in a way that each test suite can have its own helper applied and can be executed separately.

## Next Steps
## Next steps

* Learn about the [available test helpers](/docs/scos/dev/guidelines/testing-guidelines/available-test-helpers.html).
* [Create or enable a test helper](/docs/scos/dev/guidelines/testing-guidelines/test-helpers.html).
89 changes: 76 additions & 13 deletions docs/scos/dev/guidelines/testing-guidelines/test-framework.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ related:
link: docs/scos/dev/guidelines/testing-guidelines/testing-console-commands.html
---

To easily test every aspect of Spryker and the code you write, Spryker uses the [Codeception testing framework](https://codeception.com/) and PHPUnit<!-- ](https://phpunit.de/) -->.
To easily test every aspect of Spryker and the code you write, Spryker uses the [Codeception testing framework](https://codeception.com/) and [PHPUnit](https://phpunit.de/).

{% info_block infoBox %}

Expand All @@ -50,13 +50,29 @@ We strongly recommend reading the documentation of both frameworks to get the be

Codeception offers many handy things to write better and cleaner tests. Many solutions this framework has are built on top of PHPUnit. In the next articles, we will only reference Codeception even if these features are available in PHPUnit as well.

On top of Codeception, we have built the [Testify](https://github.com/spryker/testify/) module, which provides many handy helpers. See [Testify Helpers](/docs/scos/dev/guidelines/testing-guidelines/available-test-helpers.html#testify-helpers) for details on the existing helpers.
On top of Codeception, we have built the [Testify](https://github.com/spryker/testify/) module, which provides many handy helpers. See [Testify](/docs/scos/dev/guidelines/testing-guidelines/testify.html) for more details on Testify itself, and [Testify Helpers](/docs/scos/dev/guidelines/testing-guidelines/available-test-helpers.html#testify-helpers) for details on the existing Testify helpers.

## Basic test setup

To start running tests, you require a single `codeception.yml` file. Configuration provided in this file is used by default when executing the [vendor/bin/codecept run](#console-commands) command. This file, along with others, is included in the default Spryker installation and should be regarded as a starting point for your tests.

Here is an example of the `codeception.yml` file:

```
namespace: PyzTest
actor: Tester

include:
# This will include all codeception.yml's that can be found in the defined path
- tests/PyzTest/*/*

...
```
## Configuration

`codeception.yml` in the root of your project is the main entry point for your tests. In this file, the basic configuration for your test suite is defined.
The `codeception.yml` file at the root of your project is the main entry point for your tests. In this file, you define the basic configuration for your test suite.

From this file, you can include other `codeception.yml` for a better organization of your tests.
In this file, you can include other `codeception.yml` files to organize your tests better.

Example:

Expand All @@ -69,10 +85,63 @@ include:
...
```

See [codeception.yml in Spryker Master Suite](https://github.com/spryker-shop/suite/blob/master/codeception.yml) for example.
For an example, refer to [codeception.yml in Spryker Master Suite](https://github.com/spryker-shop/suite/blob/master/codeception.yml).

For more information, see [Codeception configuration documentation](https://codeception.com/docs/reference/Configuration).


{% info_block infoBox "Environment variables" %}

You can specify `.env` files with environment variables in the `codeception.yml` file. The environment variables help configure your system for specific conditions. For example, they can define the store under which tests should be executed:

```
params:
- .env
- .env.store-a.testing
```

{% endinfo_block %}

## Separating tests

In numerous scenarios, you will need to improve your test setup by separating it into logical groups. By default, all test groups are located in `tests/Pyz`. The structure of items within this directory mirrors that of the `src` code:

```
tests/
-- Pyz/
---- Glue/
---- Shared/
---- Yves/
---- Zed/
```
This structure is foundational in nearly all Spryker projects. All of the tests inside those directories are executed based on the configuration in the root `codeception.yml` file.

### Separating tests by namespace

Especially in the development environment, it often makes sense to separate tests by their application namespace. This approach lets you, for example, to run only Zed-related tests. To separate tests by namespaces, place the `codeception.yml` file into the `tests/Pyz/Zed` directory.

Here is an example of the `codeception.yml` file illustrating namespace-based separation:

```
namespace: PyzTest
actor: Tester

include:
# This will include all codeception.yml's that can be found in the defined path for this specific application namespace
- tests/PyzTest/Zed/*
...
```

### Separating tests by stores
Imagine you have a large code base, and you use stores to implement additional or different behavior from the standard module. Suppose you have 10 thousand tests that are testing all your code, but you wish to execute tests for a particular store. In this situation, there's no need to execute all tests twice—once for the default store and once for each individual store. Instead, you can execute your test suite once for the default store and then execute tests for the specific stores you require:

<div class="mxgraph" style="max-width:100%;border:1px solid transparent;" data-mxgraph="{&quot;highlight&quot;:&quot;#0000ff&quot;,&quot;nav&quot;:true,&quot;resize&quot;:true,&quot;toolbar&quot;:&quot;zoom layers tags lightbox&quot;,&quot;edit&quot;:&quot;_blank&quot;,&quot;xml&quot;:&quot;&lt;mxfile host=\&quot;ac.draw.io\&quot; modified=\&quot;2023-08-22T10:37:03.721Z\&quot; agent=\&quot;Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36\&quot; etag=\&quot;S2hL0IvwRyO6C0J7OFjW\&quot; version=\&quot;21.6.1\&quot; type=\&quot;embed\&quot;&gt;\n &lt;diagram id=\&quot;-PcddZtNe2u36sAn28E2\&quot; name=\&quot;Page-1\&quot;&gt;\n &lt;mxGraphModel dx=\&quot;1050\&quot; dy=\&quot;568\&quot; grid=\&quot;1\&quot; gridSize=\&quot;10\&quot; guides=\&quot;1\&quot; tooltips=\&quot;1\&quot; connect=\&quot;1\&quot; arrows=\&quot;1\&quot; fold=\&quot;1\&quot; page=\&quot;1\&quot; pageScale=\&quot;1\&quot; pageWidth=\&quot;850\&quot; pageHeight=\&quot;1100\&quot; math=\&quot;0\&quot; shadow=\&quot;0\&quot;&gt;\n &lt;root&gt;\n &lt;mxCell id=\&quot;0\&quot; /&gt;\n &lt;mxCell id=\&quot;1\&quot; parent=\&quot;0\&quot; /&gt;\n &lt;mxCell id=\&quot;qy1XQQReTDcXhqz7JYdl-10\&quot; value=\&quot;\&quot; style=\&quot;shape=flexArrow;endArrow=classic;html=1;rounded=0;strokeColor=#000000;fillColor=#000000;\&quot; parent=\&quot;1\&quot; edge=\&quot;1\&quot;&gt;\n &lt;mxGeometry width=\&quot;50\&quot; height=\&quot;50\&quot; relative=\&quot;1\&quot; as=\&quot;geometry\&quot;&gt;\n &lt;mxPoint x=\&quot;359.5\&quot; y=\&quot;160\&quot; as=\&quot;sourcePoint\&quot; /&gt;\n &lt;mxPoint x=\&quot;360\&quot; y=\&quot;630\&quot; as=\&quot;targetPoint\&quot; /&gt;\n &lt;/mxGeometry&gt;\n &lt;/mxCell&gt;\n &lt;mxCell id=\&quot;qy1XQQReTDcXhqz7JYdl-6\&quot; value=\&quot;\&quot; style=\&quot;shape=flexArrow;endArrow=classic;html=1;rounded=0;strokeColor=#000000;fillColor=#000000;\&quot; parent=\&quot;1\&quot; edge=\&quot;1\&quot;&gt;\n &lt;mxGeometry width=\&quot;50\&quot; height=\&quot;50\&quot; relative=\&quot;1\&quot; as=\&quot;geometry\&quot;&gt;\n &lt;mxPoint x=\&quot;269.5\&quot; y=\&quot;160\&quot; as=\&quot;sourcePoint\&quot; /&gt;\n &lt;mxPoint x=\&quot;270\&quot; y=\&quot;630\&quot; as=\&quot;targetPoint\&quot; /&gt;\n &lt;/mxGeometry&gt;\n &lt;/mxCell&gt;\n &lt;mxCell id=\&quot;qy1XQQReTDcXhqz7JYdl-5\&quot; value=\&quot;\&quot; style=\&quot;shape=flexArrow;endArrow=classic;html=1;rounded=0;strokeColor=#000000;fillColor=#000000;\&quot; parent=\&quot;1\&quot; edge=\&quot;1\&quot;&gt;\n &lt;mxGeometry width=\&quot;50\&quot; height=\&quot;50\&quot; relative=\&quot;1\&quot; as=\&quot;geometry\&quot;&gt;\n &lt;mxPoint x=\&quot;180\&quot; y=\&quot;160\&quot; as=\&quot;sourcePoint\&quot; /&gt;\n &lt;mxPoint x=\&quot;180\&quot; y=\&quot;630\&quot; as=\&quot;targetPoint\&quot; /&gt;\n &lt;/mxGeometry&gt;\n &lt;/mxCell&gt;\n &lt;mxCell id=\&quot;qy1XQQReTDcXhqz7JYdl-1\&quot; value=\&quot;Module A\&quot; style=\&quot;rounded=1;whiteSpace=wrap;html=1;\&quot; parent=\&quot;1\&quot; vertex=\&quot;1\&quot;&gt;\n &lt;mxGeometry x=\&quot;120\&quot; y=\&quot;220\&quot; width=\&quot;120\&quot; height=\&quot;60\&quot; as=\&quot;geometry\&quot; /&gt;\n &lt;/mxCell&gt;\n &lt;mxCell id=\&quot;qy1XQQReTDcXhqz7JYdl-2\&quot; value=\&quot;Module A Store A\&quot; style=\&quot;rounded=1;whiteSpace=wrap;html=1;\&quot; parent=\&quot;1\&quot; vertex=\&quot;1\&quot;&gt;\n &lt;mxGeometry x=\&quot;210\&quot; y=\&quot;290\&quot; width=\&quot;120\&quot; height=\&quot;60\&quot; as=\&quot;geometry\&quot; /&gt;\n &lt;/mxCell&gt;\n &lt;mxCell id=\&quot;qy1XQQReTDcXhqz7JYdl-3\&quot; value=\&quot;Module B\&quot; style=\&quot;rounded=1;whiteSpace=wrap;html=1;\&quot; parent=\&quot;1\&quot; vertex=\&quot;1\&quot;&gt;\n &lt;mxGeometry x=\&quot;120\&quot; y=\&quot;360\&quot; width=\&quot;120\&quot; height=\&quot;60\&quot; as=\&quot;geometry\&quot; /&gt;\n &lt;/mxCell&gt;\n &lt;mxCell id=\&quot;qy1XQQReTDcXhqz7JYdl-4\&quot; value=\&quot;Module C\&quot; style=\&quot;rounded=1;whiteSpace=wrap;html=1;\&quot; parent=\&quot;1\&quot; vertex=\&quot;1\&quot;&gt;\n &lt;mxGeometry x=\&quot;120\&quot; y=\&quot;430\&quot; width=\&quot;120\&quot; height=\&quot;60\&quot; as=\&quot;geometry\&quot; /&gt;\n &lt;/mxCell&gt;\n &lt;mxCell id=\&quot;qy1XQQReTDcXhqz7JYdl-7\&quot; value=\&quot;Default Store&amp;lt;br&amp;gt;Tests\&quot; style=\&quot;text;html=1;align=center;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;\&quot; parent=\&quot;1\&quot; vertex=\&quot;1\&quot;&gt;\n &lt;mxGeometry x=\&quot;135\&quot; y=\&quot;113\&quot; width=\&quot;90\&quot; height=\&quot;40\&quot; as=\&quot;geometry\&quot; /&gt;\n &lt;/mxCell&gt;\n &lt;mxCell id=\&quot;qy1XQQReTDcXhqz7JYdl-8\&quot; value=\&quot;Store A&amp;lt;br&amp;gt;Tests\&quot; style=\&quot;text;html=1;align=center;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;\&quot; parent=\&quot;1\&quot; vertex=\&quot;1\&quot;&gt;\n &lt;mxGeometry x=\&quot;240\&quot; y=\&quot;113\&quot; width=\&quot;60\&quot; height=\&quot;40\&quot; as=\&quot;geometry\&quot; /&gt;\n &lt;/mxCell&gt;\n &lt;mxCell id=\&quot;qy1XQQReTDcXhqz7JYdl-9\&quot; value=\&quot;Module C Store B\&quot; style=\&quot;rounded=1;whiteSpace=wrap;html=1;\&quot; parent=\&quot;1\&quot; vertex=\&quot;1\&quot;&gt;\n &lt;mxGeometry x=\&quot;300\&quot; y=\&quot;500\&quot; width=\&quot;120\&quot; height=\&quot;60\&quot; as=\&quot;geometry\&quot; /&gt;\n &lt;/mxCell&gt;\n &lt;mxCell id=\&quot;qy1XQQReTDcXhqz7JYdl-11\&quot; value=\&quot;Store B&amp;lt;br&amp;gt;Tests\&quot; style=\&quot;text;html=1;align=center;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;\&quot; parent=\&quot;1\&quot; vertex=\&quot;1\&quot;&gt;\n &lt;mxGeometry x=\&quot;330\&quot; y=\&quot;113\&quot; width=\&quot;60\&quot; height=\&quot;40\&quot; as=\&quot;geometry\&quot; /&gt;\n &lt;/mxCell&gt;\n &lt;/root&gt;\n &lt;/mxGraphModel&gt;\n &lt;/diagram&gt;\n&lt;/mxfile&gt;\n&quot;}"></div>
<script type="text/javascript" src="https://viewer.diagrams.net/js/viewer-static.min.js"></script>

You can achieve this by having separated `codeception.yml` files for the default store and for any other store.

As you can see, there are many ways to leverage Codeception configuration files to achieve fine-grained or coarse-grained groups based on your testing needs.

## Console commands

There are many console commands provided from Codeception, but the most used ones are:
Expand Down Expand Up @@ -104,12 +173,6 @@ Each test suite contains a `codeception.yml`configuration file. This file includ

For example, check the organization in the [Application](https://github.com/spryker-shop/suite/tree/master/tests/PyzTest/Yves/Application) module of Spryker Master Suite.

## Next Steps
## Next step

* [Set up an organization of your tests](/docs/scos/dev/guidelines/testing-guidelines/setting-up-tests.html).
* Learn about the [available test helpers](/docs/scos/dev/guidelines/testing-guidelines/available-test-helpers.html).
* [Create or enable a test helper](/docs/scos/dev/guidelines/testing-guidelines/test-helpers.html).
* Learn about the [console commands you can use to execute your tests](/docs/scos/dev/guidelines/testing-guidelines/executing-tests.html).
* [Configure data builders to create transfers your tests](/docs/scos/dev/guidelines/testing-guidelines/data-builders.html).
* [Generate code coverage report for your tests](/docs/scos/dev/guidelines/testing-guidelines/code-coverage.html).
* Learn about the [testing best practices](/docs/scos/dev/guidelines/testing-guidelines/testing-best-practices.html).
[Set up an organization of your tests](/docs/scos/dev/guidelines/testing-guidelines/setting-up-tests.html)
Loading
Loading