Skip to content
This repository has been archived by the owner on Aug 23, 2023. It is now read-only.

Meta tag enrichment #1433

Merged
merged 12 commits into from
Aug 23, 2019
Merged

Meta tag enrichment #1433

merged 12 commits into from
Aug 23, 2019

Conversation

replay
Copy link
Contributor

@replay replay commented Aug 14, 2019

This implements step 4 of this list: #660 (comment)
Originally the plan was to first implement it without cache, but since it was easy I've already added the enrichment cache as well.

I'll need to create another small PR which adds the meta tags / meta tag values into the autoComplete requests for tags/values. Once this is done then I think the meta tag feature should be ready to start testing it.

@replay replay changed the title [WIP] Meta tag enrichment Meta tag enrichment Aug 14, 2019
@replay
Copy link
Contributor Author

replay commented Aug 16, 2019

Rebased onto the latest master. there were a few conflicts due to the schema changes

Copy link
Contributor

@robert-milan robert-milan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with the new comments. Do we want to continue to base the next few PRs off of this branch and hold off on merging it into master until we can perform regression tests?

@replay
Copy link
Contributor Author

replay commented Aug 21, 2019

I rebased onto the latest master

also update docs accordingly, and while doing so fix some copy-paste
mistakes in the docs
this results in a pretty serious improvement:

benchcmp /tmp/enrichment.before  /tmp/enrichment.after
benchmark                       old ns/op     new ns/op     delta
BenchmarkMetaTagEnricher-12     666           526           -21.02%

benchmark                       old allocs     new allocs     delta
BenchmarkMetaTagEnricher-12     6              1              -83.33%

benchmark                       old bytes     new bytes     delta
BenchmarkMetaTagEnricher-12     66            18            -72.73%
@replay
Copy link
Contributor Author

replay commented Aug 23, 2019

I ran all the tag related benchmarks before and after these modifications, there are no significant changes afaict. In some cases there is a slight slow-down up to 20%, which is kind of expected simply because now more work is being done before returning the query results. So I'll go ahead and merge this.

benchmark                                                                      old ns/op     new ns/op     delta
BenchmarkTagDetailsWithoutFromNorFilter/partitioned-12                         50805         46204         -9.06%
BenchmarkTagDetailsWithoutFromNorFilter/unPartitioned-12                       2803          3209          +14.48%
BenchmarkTagDetailsWithFromAndFilter/partitioned-12                            35107208      33262272      -5.26%
BenchmarkTagDetailsWithFromAndFilter/unPartitioned-12                          228249275     204778690     -10.28%
BenchmarkTagsWithFromAndFilter/partitioned-12                                  19580         20051         +2.41%
BenchmarkTagsWithFromAndFilter/unPartitioned-12                                4026          3990          -0.89%
BenchmarkTagsWithoutFromNorFilter/partitioned-12                               15479         15062         -2.69%
BenchmarkTagsWithoutFromNorFilter/unPartitioned-12                             3760          3785          +0.66%
BenchmarkTagFindSimpleIntersect/partitioned-12                                 848610        820708        -3.29%
BenchmarkTagFindSimpleIntersect/unPartitioned-12                               903341        792565        -12.26%
BenchmarkTagFindRegexIntersect/partitioned-12                                  12210640      11679407      -4.35%
BenchmarkTagFindRegexIntersect/unPartitioned-12                                43254564      42648004      -1.40%
BenchmarkTagFindMatchingAndFiltering/partitioned-12                            1209754       1369904       +13.24%
BenchmarkTagFindMatchingAndFiltering/unPartitioned-12                          1174205       1164590       -0.82%
BenchmarkTagFindMatchingAndFilteringWithRegex/partitioned-12                   5310354       5250881       -1.12%
BenchmarkTagFindMatchingAndFilteringWithRegex/unPartitioned-12                 14390994      17322421      +20.37%
BenchmarkTagQueryFilterAndIntersect/withMetaTagSupport/partitioned-12          29542123      28559320      -3.33%
BenchmarkTagQueryFilterAndIntersect/withMetaTagSupport/unPartitioned-12        128761253     132800803     +3.14%
BenchmarkTagQueryFilterAndIntersect/withoutMetaTagSupport/partitioned-12       20994165      22964626      +9.39%
BenchmarkTagQueryFilterAndIntersect/withoutMetaTagSupport/unPartitioned-12     81875926      83802966      +2.35%
BenchmarkTagQueryFilterAndIntersectOnlyRegex/partitioned-12                    35738448      42909145      +20.06%
BenchmarkTagQueryFilterAndIntersectOnlyRegex/unPartitioned-12                  149519386     153496280     +2.66%
BenchmarkTagQueryKeysByPrefixSimple/partitioned-12                             15958         17159         +7.53%
BenchmarkTagQueryKeysByPrefixSimple/unPartitioned-12                           2297          2421          +5.40%
BenchmarkTagQueryKeysByPrefixExpressions/partitioned-12                        24190740      24306276      +0.48%
BenchmarkTagQueryKeysByPrefixExpressions/unPartitioned-12                      86276614      84616754      -1.92%

benchmark                                                                      old allocs     new allocs     delta
BenchmarkTagDetailsWithoutFromNorFilter/partitioned-12                         49             49             +0.00%
BenchmarkTagDetailsWithoutFromNorFilter/unPartitioned-12                       12             12             +0.00%
BenchmarkTagDetailsWithFromAndFilter/partitioned-12                            49             49             +0.00%
BenchmarkTagDetailsWithFromAndFilter/unPartitioned-12                          18             18             +0.00%
BenchmarkTagsWithFromAndFilter/partitioned-12                                  45             45             +0.00%
BenchmarkTagsWithFromAndFilter/unPartitioned-12                                24             24             +0.00%
BenchmarkTagsWithoutFromNorFilter/partitioned-12                               53             53             +0.00%
BenchmarkTagsWithoutFromNorFilter/unPartitioned-12                             32             32             +0.00%
BenchmarkTagFindSimpleIntersect/partitioned-12                                 181            181            +0.00%
BenchmarkTagFindSimpleIntersect/unPartitioned-12                               34             34             +0.00%
BenchmarkTagFindRegexIntersect/partitioned-12                                  20438          20443          +0.02%
BenchmarkTagFindRegexIntersect/unPartitioned-12                                4057           4058           +0.02%
BenchmarkTagFindMatchingAndFiltering/partitioned-12                            434            434            +0.00%
BenchmarkTagFindMatchingAndFiltering/unPartitioned-12                          260            260            +0.00%
BenchmarkTagFindMatchingAndFilteringWithRegex/partitioned-12                   6944           6943           -0.01%
BenchmarkTagFindMatchingAndFilteringWithRegex/unPartitioned-12                 4589           4587           -0.04%
BenchmarkTagQueryFilterAndIntersect/withMetaTagSupport/partitioned-12          9333           9331           -0.02%
BenchmarkTagQueryFilterAndIntersect/withMetaTagSupport/unPartitioned-12        4965           4954           -0.22%
BenchmarkTagQueryFilterAndIntersect/withoutMetaTagSupport/partitioned-12       5167           5158           -0.17%
BenchmarkTagQueryFilterAndIntersect/withoutMetaTagSupport/unPartitioned-12     869            869            +0.00%
BenchmarkTagQueryFilterAndIntersectOnlyRegex/partitioned-12                    32720          32720          +0.00%
BenchmarkTagQueryFilterAndIntersectOnlyRegex/unPartitioned-12                  4761           4763           +0.04%
BenchmarkTagQueryKeysByPrefixSimple/partitioned-12                             32             32             +0.00%
BenchmarkTagQueryKeysByPrefixSimple/unPartitioned-12                           4              4              +0.00%
BenchmarkTagQueryKeysByPrefixExpressions/partitioned-12                        12595          12748          +1.21%
BenchmarkTagQueryKeysByPrefixExpressions/unPartitioned-12                      1604           1792           +11.72%

benchmark                                                                      old bytes     new bytes     delta
BenchmarkTagDetailsWithoutFromNorFilter/partitioned-12                         7995          7995          +0.00%
BenchmarkTagDetailsWithoutFromNorFilter/unPartitioned-12                       1414          1414          +0.00%
BenchmarkTagDetailsWithFromAndFilter/partitioned-12                            8885          9446          +6.31%
BenchmarkTagDetailsWithFromAndFilter/unPartitioned-12                          3716          3716          +0.00%
BenchmarkTagsWithFromAndFilter/partitioned-12                                  3299          3299          +0.00%
BenchmarkTagsWithFromAndFilter/unPartitioned-12                                1373          1373          +0.00%
BenchmarkTagsWithoutFromNorFilter/partitioned-12                               3432          3432          +0.00%
BenchmarkTagsWithoutFromNorFilter/unPartitioned-12                             1416          1416          +0.00%
BenchmarkTagFindSimpleIntersect/partitioned-12                                 28791         28965         +0.60%
BenchmarkTagFindSimpleIntersect/unPartitioned-12                               4424          4544          +2.71%
BenchmarkTagFindRegexIntersect/partitioned-12                                  1593657       1646638       +3.32%
BenchmarkTagFindRegexIntersect/unPartitioned-12                                463632        493914        +6.53%
BenchmarkTagFindMatchingAndFiltering/partitioned-12                            76393         84543         +10.67%
BenchmarkTagFindMatchingAndFiltering/unPartitioned-12                          50431         55377         +9.81%
BenchmarkTagFindMatchingAndFilteringWithRegex/partitioned-12                   1107738       1239392       +11.88%
BenchmarkTagFindMatchingAndFilteringWithRegex/unPartitioned-12                 849970        932590        +9.72%
BenchmarkTagQueryFilterAndIntersect/withMetaTagSupport/partitioned-12          7300179       7305306       +0.07%
BenchmarkTagQueryFilterAndIntersect/withMetaTagSupport/unPartitioned-12        7095322       7100521       +0.07%
BenchmarkTagQueryFilterAndIntersect/withoutMetaTagSupport/partitioned-12       323172        327697        +1.40%
BenchmarkTagQueryFilterAndIntersect/withoutMetaTagSupport/unPartitioned-12     85028         88769         +4.40%
BenchmarkTagQueryFilterAndIntersectOnlyRegex/partitioned-12                    1882708       1880294       -0.13%
BenchmarkTagQueryFilterAndIntersectOnlyRegex/unPartitioned-12                  359168        360104        +0.26%
BenchmarkTagQueryKeysByPrefixSimple/partitioned-12                             2624          2624          +0.00%
BenchmarkTagQueryKeysByPrefixSimple/unPartitioned-12                           352           352           +0.00%
BenchmarkTagQueryKeysByPrefixExpressions/partitioned-12                        615737        618359        +0.43%
BenchmarkTagQueryKeysByPrefixExpressions/unPartitioned-12                      87087         93386         +7.23%

@replay replay merged commit 7000ea2 into master Aug 23, 2019
@replay replay deleted the meta_tag_enrichment branch August 23, 2019 16:16
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants