[Maps] support term joins for Elasticsearch document source with vector tile scaling #129771

nreese · 2022-04-07T15:58:33Z

Fixes #124227 and adds "Term join" support for "Vector tiles" scaling.

Why vector tiles? Much faster loading. Map on left uses vector tiles with joins. Map on right uses GeoJSON with joins. Notice the loading time.
https://user-images.githubusercontent.com/373691/162276074-865fb829-baed-4725-b196-bc10c274215f.mov

Testing

Download world_countries_v7.geo.json from https://maps.elastic.co/#file/world_countries
Use geojson upload to index world_countries_v7.geo.json into elasticsearch
Install sample web logs data set
Create new map
Add documents layer with "world_countries" data view
Add term join
Set left field to "iso2"
Set right source to "kibana_sample_data_logs"
Set right field to "geo.dest"

Limitations

There are several limitations to term join support with vector tiles

Multiple joins not supported. To deal with this limitation, term join UI disables "Add join" button, preventing users from adding more then one join. Also when switching to "Use vector tiles" scaling option, user is shown modal with message stating that multiple joins not supported and all joins but the first will be removed
Style properties that use maplibre "layout" can not be styled with fields from term join. The UI has been updated to disable these fields in the field select.

PR updates scaling types help to better define limitations of vector tiles and when to use "Limit results"

…-fix'

nreese · 2022-04-11T17:43:55Z

@elasticmachine merge upstream

…-fix'

elasticmachine · 2022-04-13T12:44:02Z

Pinging @elastic/kibana-gis (Team:Geo)

nreese · 2022-04-14T14:56:43Z

@elasticmachine merge upstream

…-fix'

nickpeihl

lgtm! this is a great feature. With vector tiles even a small dataset like world countries loads much faster.

code review and tested in chrome.

x-pack/plugins/maps/public/actions/layer_actions.ts

nickpeihl · 2022-04-20T13:27:00Z

x-pack/plugins/maps/public/classes/styles/vector/vector_style.tsx

@@ -343,9 +343,21 @@ export class VectorStyle implements IVectorStyle {
    });

    dynamicProperties.forEach((key: VECTOR_STYLES) => {
+      // TODO instead of looking up instance by key, update argument originalProperties to be instances instead of descriptors


should we create an issue for this?

nickpeihl · 2022-04-20T14:18:58Z

Hmm, I just noticed this error in console when switching from GeoJSON to MVT. It seems to only happen when I have multiple joins and click the "Accept" button to switch to a single join.

Uncaught (in promise) Error
    at data_request_actions.ts:182:1
    at index.js:11:1
    at dispatch (redux.js:667:1)
    at stopLoading (data_request_actions.ts:66:1)
    at MvtVectorLayer._syncSupportsFeatureEditing (vector_layer.tsx:540:1)
    at async MvtVectorLayer.syncData (mvt_vector_layer.tsx:203:1)
    at async data_request_actions.ts:126:1

jsanz · 2022-04-20T14:29:39Z

Hmm, I just noticed this error in console when switching from GeoJSON to MVT. It seems to only happen when I have multiple joins and click the "Accept" button to switch to a single join.
Uncaught (in promise) Error
    at data_request_actions.ts:182:1
    at index.js:11:1
    at dispatch (redux.js:667:1)
    at stopLoading (data_request_actions.ts:66:1)
    at MvtVectorLayer._syncSupportsFeatureEditing (vector_layer.tsx:540:1)
    at async MvtVectorLayer.syncData (mvt_vector_layer.tsx:203:1)
    at async data_request_actions.ts:126:1

Same error also when switching between metrics

Co-authored-by: Nick Peihl <nickpeihl@gmail.com>

nreese · 2022-04-20T14:31:48Z

@elasticmachine merge upstream

kibanamachine · 2022-04-20T14:31:51Z

merge conflict between base and head

kibana-ci · 2022-04-20T16:02:53Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`maps`	796	799	+3

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`maps`	222	223	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`maps`	2.6MB	2.6MB	+3.7KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`maps`	27	25	-2

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`maps`	69.8KB	70.1KB	+300.0B

Unknown metric groups

API count

id	before	after	diff
`maps`	223	224	+1

History

💚 Build #38714 succeeded b0ecd46
💚 Build #38242 succeeded a4167f8
💚 Build #37750 succeeded 9922313
💔 Build #37717 failed 50e4a84
💔 Build #37693 failed a923dac
💔 Build #37614 failed 2b30281

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…or tile scaling (elastic#129771) * support join configuration * set feature state * pluckStyleMetaFromFeatures * pluck style meta from joins * refactor * eslint * clear feature state on update * fit to bounds * remove unneeded cast * filter out features without join results * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * remove/add source if promoteId changes * hide all features when there are no results * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * disable layout styles * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * disable add join button * remove multiple term joins when switching to vector tiles * update scaling type in-product help * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * update chropleth wizard to use MVT scaling * clean up * i18n fixes * remove unused i18n tags * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * fix jest tests * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * tslint * fix jest test * add functional test * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * more jest fixes * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * Update x-pack/plugins/maps/public/actions/layer_actions.ts Co-authored-by: Nick Peihl <nickpeihl@gmail.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Nick Peihl <nickpeihl@gmail.com>

nreese and others added 23 commits April 4, 2022 10:43

support join configuration

c30da49

set feature state

f40866d

pluckStyleMetaFromFeatures

ca3a192

pluck style meta from joins

4a3341d

refactor

06a3c5c

eslint

18f602b

clear feature state on update

fb12139

fit to bounds

d5ba7ee

remove unneeded cast

f205cb9

Merge branch 'main' of github.com:elastic/kibana into mvt_joins

6726e7f

filter out features without join results

b8dadf5

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

00077ee

…-fix'

remove/add source if promoteId changes

91299e2

hide all features when there are no results

0a9a02a

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

b4fc6b3

…-fix'

disable layout styles

94653ce

Merge branch 'mvt_joins' of github.com:nreese/kibana into mvt_joins

a015004

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

e19a0b0

…-fix'

disable add join button

0b75dba

Merge branch 'mvt_joins' of github.com:nreese/kibana into mvt_joins

38f4156

remove multiple term joins when switching to vector tiles

cad1e53

update scaling type in-product help

13a6520

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

5b5f864

…-fix'

kibanamachine and others added 6 commits April 11, 2022 12:43

Merge branch 'main' into mvt_joins

6726e43

update chropleth wizard to use MVT scaling

6b0a98f

clean up

91b1830

i18n fixes

249eb64

remove unused i18n tags

8deb218

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

4920d57

…-fix'

nreese and others added 5 commits April 12, 2022 14:53

Merge branch 'mvt_joins' of github.com:nreese/kibana into mvt_joins

5bd4e1e

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

50e4a84

…-fix'

more jest fixes

6459c5a

Merge branch 'mvt_joins' of github.com:nreese/kibana into mvt_joins

629ae1a

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

9922313

…-fix'

nreese marked this pull request as ready for review April 13, 2022 12:40

nreese requested a review from a team as a code owner April 13, 2022 12:40

nreese added release_note:enhancement [Deprecated-Use Team:Presentation]Team:Geo Former Team Label for Geo Team. Now use Team:Presentation v8.3.0 labels Apr 13, 2022

nreese added the backport:skip This commit does not require backporting label Apr 13, 2022

kibanamachine and others added 4 commits April 14, 2022 09:56

Merge branch 'main' into mvt_joins

a4167f8

merge with main

a13e396

Merge branch 'mvt_joins' of github.com:nreese/kibana into mvt_joins

e0c95c1

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

b0ecd46

…-fix'

nickpeihl approved these changes Apr 20, 2022

View reviewed changes

Update x-pack/plugins/maps/public/actions/layer_actions.ts

2b064ea

Co-authored-by: Nick Peihl <nickpeihl@gmail.com>

merge with main

503d702

nreese merged commit 88c8ed9 into elastic:main Apr 20, 2022

This was referenced May 20, 2022

[Maps] create vector tile layer when using choropleth wizard #132648

Merged

[Maps][File upload] Give the option to reduce the complexity of the shape (round decimal places of lat & long) #124234

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Maps] support term joins for Elasticsearch document source with vector tile scaling #129771

[Maps] support term joins for Elasticsearch document source with vector tile scaling #129771

nreese commented Apr 7, 2022 •

edited

Loading

nreese commented Apr 11, 2022

elasticmachine commented Apr 13, 2022

nreese commented Apr 14, 2022

nickpeihl left a comment

nickpeihl Apr 20, 2022

nickpeihl commented Apr 20, 2022

jsanz commented Apr 20, 2022 •

edited

Loading

nreese commented Apr 20, 2022

kibanamachine commented Apr 20, 2022

kibana-ci commented Apr 20, 2022

API count

[Maps] support term joins for Elasticsearch document source with vector tile scaling #129771

[Maps] support term joins for Elasticsearch document source with vector tile scaling #129771

Conversation

nreese commented Apr 7, 2022 • edited Loading

Testing

Limitations

nreese commented Apr 11, 2022

elasticmachine commented Apr 13, 2022

nreese commented Apr 14, 2022

nickpeihl left a comment

Choose a reason for hiding this comment

nickpeihl Apr 20, 2022

Choose a reason for hiding this comment

nickpeihl commented Apr 20, 2022

jsanz commented Apr 20, 2022 • edited Loading

nreese commented Apr 20, 2022

kibanamachine commented Apr 20, 2022

kibana-ci commented Apr 20, 2022

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Public APIs missing exports

Page load bundle

API count

History

nreese commented Apr 7, 2022 •

edited

Loading

jsanz commented Apr 20, 2022 •

edited

Loading