Fix for Kedro Viz Connection Error #1507

ravi-kumar-pilla · 2023-08-29T03:35:44Z

Description

Resolves #1277

Development notes

Start Fast API server using multiprocessing similar to jupyter launcher with/without --autoreload option
Wait for server to start before opening web browser

QA notes

Checkout branch fix/connection-error
Build frontend using make build
Build backend using pip install -e package
cd demo-project, execute command kedro viz or kedro viz --autoreload
Browser window should open after FAST API server is started and the browser should not dispay a connection error

Checklist

Read the contributing guidelines
Opened this PR as a 'Draft Pull Request' if it is work-in-progress
Updated the documentation to reflect the code changes
Added new entries to the RELEASE.md file
Added tests to cover my changes

…x/connection-error

…edro-viz into fix/connection-error

…x/connection-error

…1499) * add stats to data node * lint and format check fix * fix pytests * fix layout issue * fix transcoded data stats Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* initial draft for python 3.11 support * update release doc * add python warnings for e2e tests * modify e2e test * modify e2e test * test by removing lower req scenario * skip e2e tests for lower bound requirement on python 3.11 * skip e2e tests for lower bound requirement on python 3.11 * remove print statements --------- Co-authored-by: Nok Lam Chan <nok_lam_chan@mckinsey.com> Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* initial draft for python 3.11 support * update release doc * add python warnings for e2e tests * modify e2e test * modify e2e test * test by removing lower req scenario * skip e2e tests for lower bound requirement on python 3.11 * skip e2e tests for lower bound requirement on python 3.11 * remove python upperbounds initial draft * fix lint and format errors * test remove upperbound warning * test lowerbound pandas install * revert back pandas requirement * bump lower requirements for pandas * remove upper bound clean up * update release notes * fix PR comments --------- Co-authored-by: Nok Lam Chan <nok_lam_chan@mckinsey.com> Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* Fix: Adding favicon to Kedro Demo * Fix: Change in approach for serving favicon * Lint error fix * Lint error fix * Favicon endpoint test added * Favicon endpoint test added * Lint error fixed * Fix: Adding favicon to Kedro Demo Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> * Fix: Change in approach for serving favicon Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> * Lint error fix Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> * Lint error fix Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> * Favicon endpoint test added Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> * Favicon endpoint test added Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> * Lint error fixed Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> * Fixed favicon endpoint test * Release doc updated * Update RELEASE.md Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com> * Removed pytest.fixture as per review comment --------- Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com> Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* v6.5.0 * release * update-reminder-content * update reminder Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

noklam

Leave a small comment. So to check with my understanding, the main change here is moving the start_browser from run_server to the cli, and the run_server only handle the backend process.

I have a quick question, why multiprocessing is needed?

noklam · 2023-09-01T01:08:42Z

package/kedro_viz/launchers/jupyter.py

@@ -175,7 +116,7 @@ def run_viz(port: int = None, local_ns: Dict[str, Any] = None) -> None:
    viz_process.start()
    _VIZ_PROCESSES[port] = viz_process

-    _wait_for(func=_check_viz_up, host=host, port=port)
+    wait_for(func=check_viz_up, host=host, port=port)


What is the reason to change from private function to public one?

Earlier these functions were inside jupyter.py and used only by jupyter, now it is also used in cli launcher. so moved it to a utils file and these functions are now used in both places

I see they are now shared among two modules, but I think it should be an internal method _wait_for instead of wait_for.

For example, we have similar internal utils here:
https://github.com/kedro-org/kedro/blob/edbbcc7f7a863b7c980f9efcaa444e2fa01f45fa/kedro/config/common.py#L209

Changing from _wait_for -> wait_for singals this is a public function. (though in reality 99% people are using kedro-viz as an appllication but not a module). So this is just to keep things clean internally for ourselves.

https://stackoverflow.com/questions/1301346/what-is-the-meaning-of-single-and-double-underscore-before-an-object-name

noklam · 2023-09-01T01:14:27Z

package/kedro_viz/launchers/cli.py

+        if browser:
+            start_browser(host, port)


ravi-kumar-pilla · 2023-09-01T16:06:55Z

Leave a small comment. So to check with my understanding, the main change here is moving the start_browser from run_server to the cli, and the run_server only handle the backend process.

Yes, I have moved the start_browser logic out of run_server as run_server should just focus on starting the FAST API server.

I have a quick question, why multiprocessing is needed?

So, earlier we used to start the browser without waiting for backend server to start. Now if one process is used to start the server, there should be another process which can wait for the server to start and then start a browser (This can also be done using threading, but since we were using multiprocessing in jupyter already, I thought of using the same in cli launcher as well. )

noklam

I've experienced this issue, definitely an improvement for UX. Great fix, thanks!

rashidakanchwala

LGTM. thanks :)

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

ravi-kumar-pilla and others added 27 commits August 28, 2023 22:15

initial draft for resolving connection error

b0996a7

Merge branch 'main' of https://github.com/kedro-org/kedro-viz into fi…

1dcb2de

…x/connection-error

Merge branch 'main' of https://github.com/kedro-org/kedro-viz into fi…

d4c7f25

…x/connection-error

refactor launchers and test code

d7c6452

modify unit tests

fb9a30d

fix lint errors

d0ca4de

fix run_viz tests

944a24d

Merge branch 'main' of https://github.com/kedro-org/kedro-viz into fi…

d9177a2

…x/connection-error

Merge branch 'fix/connection-error' of https://github.com/kedro-org/k…

13345b3

…edro-viz into fix/connection-error

update unit test for coverage

79fd2c3

Merge branch 'main' of https://github.com/kedro-org/kedro-viz into fi…

bed1b00

…x/connection-error

update unit tests

6ea85d9

Refactor visualize dataset stats from DataNodeMetadata to DataNode (#…

79f05fd

…1499) * add stats to data node * lint and format check fix * fix pytests * fix layout issue * fix transcoded data stats Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

initial draft for resolving connection error

fb24aaf

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

refactor launchers and test code

1092592

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

modify unit tests

26e519d

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

fix lint errors

f990e25

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

fix run_viz tests

84e0aae

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

update unit test for coverage

7909b8d

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

Release v6.5.0 (#1513)

d527151

* v6.5.0 * release * update-reminder-content * update reminder Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

remove branch condition for automate release version check (#1514)

14945ba

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

update unit tests

cac667e

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

add release record

7f00918

add release record

0c32e04

ravi-kumar-pilla marked this pull request as ready for review August 31, 2023 21:41

ravi-kumar-pilla requested review from tynandebold and yetudada as code owners August 31, 2023 21:41

ravi-kumar-pilla requested a review from rashidakanchwala as a code owner August 31, 2023 21:41

ravi-kumar-pilla requested review from noklam, jitu5, merelcht, NeroOkwa, SajidAlamQB and vladimir-mck August 31, 2023 21:41

modify comment

1ee8a5a

noklam reviewed Sep 1, 2023

View reviewed changes

ravi-kumar-pilla added 2 commits September 6, 2023 10:39

fix PR comments

47ce1e7

DCO fix

868af42

noklam self-requested a review September 6, 2023 16:19

noklam approved these changes Sep 6, 2023

View reviewed changes

NeroOkwa approved these changes Sep 6, 2023

View reviewed changes

tynandebold removed the request for review from yetudada September 7, 2023 15:08

rashidakanchwala approved these changes Sep 7, 2023

View reviewed changes

ravi-kumar-pilla added 3 commits September 11, 2023 10:29

Merge branch 'main' into fix/connection-error

ff80b79

fixing dco

822fff1

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

update pytest

f350bbb

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

ravi-kumar-pilla merged commit 335be7d into main Sep 11, 2023
20 checks passed

ravi-kumar-pilla deleted the fix/connection-error branch September 11, 2023 19:17

tynandebold mentioned this pull request Oct 10, 2023

Release v6.6.0 #1566

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix for Kedro Viz Connection Error #1507

Fix for Kedro Viz Connection Error #1507

ravi-kumar-pilla commented Aug 29, 2023 •

edited

Loading

noklam left a comment

noklam Sep 1, 2023

ravi-kumar-pilla Sep 1, 2023

noklam Sep 4, 2023

noklam Sep 1, 2023

ravi-kumar-pilla commented Sep 1, 2023

noklam left a comment •

edited

Loading

rashidakanchwala left a comment

Fix for Kedro Viz Connection Error #1507

Fix for Kedro Viz Connection Error #1507

Conversation

ravi-kumar-pilla commented Aug 29, 2023 • edited Loading

Description

Development notes

QA notes

Checklist

noklam left a comment

Choose a reason for hiding this comment

noklam Sep 1, 2023

Choose a reason for hiding this comment

ravi-kumar-pilla Sep 1, 2023

Choose a reason for hiding this comment

noklam Sep 4, 2023

Choose a reason for hiding this comment

noklam Sep 1, 2023

Choose a reason for hiding this comment

ravi-kumar-pilla commented Sep 1, 2023

noklam left a comment • edited Loading

Choose a reason for hiding this comment

rashidakanchwala left a comment

Choose a reason for hiding this comment

ravi-kumar-pilla commented Aug 29, 2023 •

edited

Loading

noklam left a comment •

edited

Loading