-
Notifications
You must be signed in to change notification settings - Fork 17.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cmd/compile: static init maps should never generate code #2559
Comments
Owner changed to builder@golang.org. |
Change https://golang.org/cl/127075 mentions this issue: |
Saves ~105KB of heap for callers who don't use html.UnescapeString. (EscapeString is much more common). Also saves 70KB of binary size, because now the linker can do dead code elimination. (because #2559 is still open and global maps always generate init code) Fixes #26727 Updates #6853 Change-Id: I18fe9a273097e2c7e0cb7f88205cae1bb60fa89b Reviewed-on: https://go-review.googlesource.com/127075 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
This may in fact not be possible because the hash function and even the hash seed can be selected at runtime. However, even if we're generating init code for maps, the linker could still understand that that init code exists solely to populate that map, so if the map is unreferenced, the init code could be eliminated, too. |
In my opinion Kubernetes shouldn't treat their code as a fork of text/template, they should just write their own comparison package. They can use the text/template code as, well, a template for their own code. |
Thanks makes sense, thanks, I will talk to @liggitt to see if that is an option. |
I am still curious why one would fork funcs.go unless there is a bug, in which case one should file an issue to get it fixed. |
I also don't have context on why this was copied, will dig some |
It looks like it wasn't a fork as much as an attributed copy preserving the golang LICENSE. It looks like kubernetes copied this in kubernetes/kubernetes@949704a#diff-c8cc872900781dafd1c58d29c73f4e0b2663cff60398aac88918a266019209ed solely for the purpose of reusing the go implementations of these functions:
Some of those would be trivial to reimplement without reference to the original source with 100% compatibility (neither more tolerant nor more strict), others... less so. In any case, the Kubernetes use doesn't seem related to this issue. |
I agree this is unrelated. The jsonpath package has just borrowed some come from the template library. They can do what they want with it. |
Change https://go.dev/cl/419756 mentions this issue: |
I wrote a work-in-progress CL to help with this: https://go.dev/cl/461315. It implements the weak relocation idea that Austin and I talked about (or something like it). Passes all.bash and trybots, haven't tested it more widely than that. Comments/suggestions welcome. |
Change https://go.dev/cl/461315 mentions this issue: |
Change https://go.dev/cl/463395 mentions this issue: |
Change https://go.dev/cl/463855 mentions this issue: |
Introduce a flag in the object file indicating whether a given function corresponds to a compiler-generated (not user-written) init function, such as "os.init" or "syscall.init". Add code to the compiler to fill in the correct value for the flag, and add support to the loader package in the linker for testing the flag. The new loader API is currently unused, but will be needed in the next CL in this stack. Updates #2559. Updates #36021. Updates #14840. Change-Id: Iea7ad2adda487e4af7a44f062f9817977c53b394 Reviewed-on: https://go-review.googlesource.com/c/go/+/463855 Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Than McIntosh <thanm@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
This patch changes the compiler's pkg init machinery to pick out large initialization assignments to global maps (e.g. var mymap = map[string]int{"foo":1, "bar":2, ... } and extract the map init code into a separate outlined function, which is then called from the main init function with a weak relocation: var mymap map[string]int // KEEP reloc -> map.init.0 func init() { map.init.0() // weak relocation } func map.init.0() { mymap = map[string]int{"foo":1, "bar":2} } The map init outlining is done selectively (only in the case where the RHS code exceeds a size limit of 20 IR nodes). In order to ensure that a given map.init.NNN function is included when its corresponding map is live, we add dummy R_KEEP relocation from the map variable to the map init function. This first patch includes the main compiler compiler changes, and with the weak relocation addition disabled. Subsequent patch includes the requred linker changes along with switching to the call to the outlined routine to a weak relocation. See the later linker change for associated compile time performance numbers. Updates #2559. Updates #36021. Updates #14840. Change-Id: I1fd6fd6397772be1ebd3eb397caf68ae9a3147e9 Reviewed-on: https://go-review.googlesource.com/c/go/+/461315 Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
This patch contains the linker changes needed to enable deadcoding of large unreferenced map variables, in combination with a previous compiler change. We add a new cleanup function that runs just after deadcode that looks for relocations in "init" funcs that are weak, of type R_CALL (and siblings), and are targeting an unreachable function. If we find such a relocation, after checking to make sure it targets a map.init.XXX helper, we redirect the relocation to a point to a no-op routine ("mapinitnoop") in the runtime. Compilebench results for this change: │ out.base.txt │ out.wrap.txt │ │ sec/op │ sec/op vs base │ Template 218.6m ± 2% 221.1m ± 1% ~ (p=0.129 n=39) Unicode 180.5m ± 1% 178.9m ± 1% -0.93% (p=0.006 n=39) GoTypes 1.162 ± 1% 1.156 ± 1% ~ (p=0.850 n=39) Compiler 143.6m ± 1% 142.6m ± 1% ~ (p=0.743 n=39) SSA 8.698 ± 1% 8.719 ± 1% ~ (p=0.145 n=39) Flate 142.6m ± 1% 143.9m ± 3% ~ (p=0.287 n=39) GoParser 247.7m ± 1% 248.8m ± 1% ~ (p=0.265 n=39) Reflect 588.0m ± 1% 590.4m ± 1% ~ (p=0.269 n=39) Tar 198.5m ± 1% 201.3m ± 1% +1.38% (p=0.005 n=39) XML 259.1m ± 1% 260.0m ± 1% ~ (p=0.376 n=39) LinkCompiler 746.8m ± 2% 747.8m ± 1% ~ (p=0.706 n=39) ExternalLinkCompiler 1.906 ± 0% 1.902 ± 1% ~ (p=0.207 n=40) LinkWithoutDebugCompiler 522.4m ± 21% 471.1m ± 1% -9.81% (p=0.000 n=40) StdCmd 21.32 ± 0% 21.39 ± 0% +0.32% (p=0.035 n=40) geomean 609.2m 606.0m -0.53% │ out.base.txt │ out.wrap.txt │ │ user-sec/op │ user-sec/op vs base │ Template 401.9m ± 3% 406.9m ± 2% ~ (p=0.291 n=39) Unicode 191.9m ± 6% 196.1m ± 3% ~ (p=0.052 n=39) GoTypes 3.967 ± 3% 4.056 ± 1% ~ (p=0.099 n=39) Compiler 171.1m ± 3% 173.4m ± 3% ~ (p=0.415 n=39) SSA 30.04 ± 4% 30.25 ± 4% ~ (p=0.106 n=39) Flate 246.3m ± 3% 248.9m ± 4% ~ (p=0.499 n=39) GoParser 518.7m ± 1% 520.6m ± 2% ~ (p=0.531 n=39) Reflect 1.670 ± 1% 1.656 ± 2% ~ (p=0.137 n=39) Tar 352.7m ± 2% 360.3m ± 2% ~ (p=0.117 n=39) XML 528.8m ± 2% 521.1m ± 2% ~ (p=0.296 n=39) LinkCompiler 1.128 ± 2% 1.140 ± 2% ~ (p=0.324 n=39) ExternalLinkCompiler 2.165 ± 2% 2.147 ± 2% ~ (p=0.537 n=40) LinkWithoutDebugCompiler 484.2m ± 4% 490.7m ± 3% ~ (p=0.897 n=40) geomean 818.5m 825.1m +0.80% │ out.base.txt │ out.wrap.txt │ │ text-bytes │ text-bytes vs base │ HelloSize 766.0Ki ± 0% 766.0Ki ± 0% ~ (p=1.000 n=40) ¹ CmdGoSize 10.02Mi ± 0% 10.02Mi ± 0% -0.03% (n=40) geomean 2.738Mi 2.738Mi -0.01% ¹ all samples are equal │ out.base.txt │ out.wrap.txt │ │ data-bytes │ data-bytes vs base │ HelloSize 14.17Ki ± 0% 14.17Ki ± 0% ~ (p=1.000 n=40) ¹ CmdGoSize 308.3Ki ± 0% 298.5Ki ± 0% -3.19% (n=40) geomean 66.10Ki 65.04Ki -1.61% ¹ all samples are equal │ out.base.txt │ out.wrap.txt │ │ bss-bytes │ bss-bytes vs base │ HelloSize 197.3Ki ± 0% 197.3Ki ± 0% ~ (p=1.000 n=40) ¹ CmdGoSize 228.2Ki ± 0% 228.1Ki ± 0% -0.01% (n=40) geomean 212.2Ki 212.1Ki -0.01% ¹ all samples are equal │ out.base.txt │ out.wrap.txt │ │ exe-bytes │ exe-bytes vs base │ HelloSize 1.192Mi ± 0% 1.192Mi ± 0% +0.00% (p=0.000 n=40) CmdGoSize 14.85Mi ± 0% 14.83Mi ± 0% -0.09% (n=40) geomean 4.207Mi 4.205Mi -0.05% Also tested for any linker changes by benchmarking relink of k8s "kubelet"; no changes to speak of there. Updates #2559. Updates #36021. Updates #14840. Change-Id: I4cc5370b3f20679a1065aaaf87bdf2881e257631 Reviewed-on: https://go-review.googlesource.com/c/go/+/463395 Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
For posterity, linker dead code elimination of unused maps is now implemented in this stack; I am going to close out this issue. One thing to note is that in the original code in text/template the code looked like:
Although the new implementation is now capable of dead-coding the |
This comment was marked as resolved.
This comment was marked as resolved.
Austin points out that there is also the possibility of creating an alternate implementation of map literals that are composed only of initialized data (no code required to init). My CLs don't implement this, so leaving this issue open. |
Introduce a flag in the object file indicating whether a given function corresponds to a compiler-generated (not user-written) init function, such as "os.init" or "syscall.init". Add code to the compiler to fill in the correct value for the flag, and add support to the loader package in the linker for testing the flag. The new loader API is currently unused, but will be needed in the next CL in this stack. Updates golang#2559. Updates golang#36021. Updates golang#14840. Change-Id: Iea7ad2adda487e4af7a44f062f9817977c53b394 Reviewed-on: https://go-review.googlesource.com/c/go/+/463855 Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Than McIntosh <thanm@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
This patch changes the compiler's pkg init machinery to pick out large initialization assignments to global maps (e.g. var mymap = map[string]int{"foo":1, "bar":2, ... } and extract the map init code into a separate outlined function, which is then called from the main init function with a weak relocation: var mymap map[string]int // KEEP reloc -> map.init.0 func init() { map.init.0() // weak relocation } func map.init.0() { mymap = map[string]int{"foo":1, "bar":2} } The map init outlining is done selectively (only in the case where the RHS code exceeds a size limit of 20 IR nodes). In order to ensure that a given map.init.NNN function is included when its corresponding map is live, we add dummy R_KEEP relocation from the map variable to the map init function. This first patch includes the main compiler compiler changes, and with the weak relocation addition disabled. Subsequent patch includes the requred linker changes along with switching to the call to the outlined routine to a weak relocation. See the later linker change for associated compile time performance numbers. Updates golang#2559. Updates golang#36021. Updates golang#14840. Change-Id: I1fd6fd6397772be1ebd3eb397caf68ae9a3147e9 Reviewed-on: https://go-review.googlesource.com/c/go/+/461315 Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
This patch contains the linker changes needed to enable deadcoding of large unreferenced map variables, in combination with a previous compiler change. We add a new cleanup function that runs just after deadcode that looks for relocations in "init" funcs that are weak, of type R_CALL (and siblings), and are targeting an unreachable function. If we find such a relocation, after checking to make sure it targets a map.init.XXX helper, we redirect the relocation to a point to a no-op routine ("mapinitnoop") in the runtime. Compilebench results for this change: │ out.base.txt │ out.wrap.txt │ │ sec/op │ sec/op vs base │ Template 218.6m ± 2% 221.1m ± 1% ~ (p=0.129 n=39) Unicode 180.5m ± 1% 178.9m ± 1% -0.93% (p=0.006 n=39) GoTypes 1.162 ± 1% 1.156 ± 1% ~ (p=0.850 n=39) Compiler 143.6m ± 1% 142.6m ± 1% ~ (p=0.743 n=39) SSA 8.698 ± 1% 8.719 ± 1% ~ (p=0.145 n=39) Flate 142.6m ± 1% 143.9m ± 3% ~ (p=0.287 n=39) GoParser 247.7m ± 1% 248.8m ± 1% ~ (p=0.265 n=39) Reflect 588.0m ± 1% 590.4m ± 1% ~ (p=0.269 n=39) Tar 198.5m ± 1% 201.3m ± 1% +1.38% (p=0.005 n=39) XML 259.1m ± 1% 260.0m ± 1% ~ (p=0.376 n=39) LinkCompiler 746.8m ± 2% 747.8m ± 1% ~ (p=0.706 n=39) ExternalLinkCompiler 1.906 ± 0% 1.902 ± 1% ~ (p=0.207 n=40) LinkWithoutDebugCompiler 522.4m ± 21% 471.1m ± 1% -9.81% (p=0.000 n=40) StdCmd 21.32 ± 0% 21.39 ± 0% +0.32% (p=0.035 n=40) geomean 609.2m 606.0m -0.53% │ out.base.txt │ out.wrap.txt │ │ user-sec/op │ user-sec/op vs base │ Template 401.9m ± 3% 406.9m ± 2% ~ (p=0.291 n=39) Unicode 191.9m ± 6% 196.1m ± 3% ~ (p=0.052 n=39) GoTypes 3.967 ± 3% 4.056 ± 1% ~ (p=0.099 n=39) Compiler 171.1m ± 3% 173.4m ± 3% ~ (p=0.415 n=39) SSA 30.04 ± 4% 30.25 ± 4% ~ (p=0.106 n=39) Flate 246.3m ± 3% 248.9m ± 4% ~ (p=0.499 n=39) GoParser 518.7m ± 1% 520.6m ± 2% ~ (p=0.531 n=39) Reflect 1.670 ± 1% 1.656 ± 2% ~ (p=0.137 n=39) Tar 352.7m ± 2% 360.3m ± 2% ~ (p=0.117 n=39) XML 528.8m ± 2% 521.1m ± 2% ~ (p=0.296 n=39) LinkCompiler 1.128 ± 2% 1.140 ± 2% ~ (p=0.324 n=39) ExternalLinkCompiler 2.165 ± 2% 2.147 ± 2% ~ (p=0.537 n=40) LinkWithoutDebugCompiler 484.2m ± 4% 490.7m ± 3% ~ (p=0.897 n=40) geomean 818.5m 825.1m +0.80% │ out.base.txt │ out.wrap.txt │ │ text-bytes │ text-bytes vs base │ HelloSize 766.0Ki ± 0% 766.0Ki ± 0% ~ (p=1.000 n=40) ¹ CmdGoSize 10.02Mi ± 0% 10.02Mi ± 0% -0.03% (n=40) geomean 2.738Mi 2.738Mi -0.01% ¹ all samples are equal │ out.base.txt │ out.wrap.txt │ │ data-bytes │ data-bytes vs base │ HelloSize 14.17Ki ± 0% 14.17Ki ± 0% ~ (p=1.000 n=40) ¹ CmdGoSize 308.3Ki ± 0% 298.5Ki ± 0% -3.19% (n=40) geomean 66.10Ki 65.04Ki -1.61% ¹ all samples are equal │ out.base.txt │ out.wrap.txt │ │ bss-bytes │ bss-bytes vs base │ HelloSize 197.3Ki ± 0% 197.3Ki ± 0% ~ (p=1.000 n=40) ¹ CmdGoSize 228.2Ki ± 0% 228.1Ki ± 0% -0.01% (n=40) geomean 212.2Ki 212.1Ki -0.01% ¹ all samples are equal │ out.base.txt │ out.wrap.txt │ │ exe-bytes │ exe-bytes vs base │ HelloSize 1.192Mi ± 0% 1.192Mi ± 0% +0.00% (p=0.000 n=40) CmdGoSize 14.85Mi ± 0% 14.83Mi ± 0% -0.09% (n=40) geomean 4.207Mi 4.205Mi -0.05% Also tested for any linker changes by benchmarking relink of k8s "kubelet"; no changes to speak of there. Updates golang#2559. Updates golang#36021. Updates golang#14840. Change-Id: I4cc5370b3f20679a1065aaaf87bdf2881e257631 Reviewed-on: https://go-review.googlesource.com/c/go/+/463395 Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
This patch changes the compiler's pkg init machinery to pick out large initialization assignments to global maps (e.g. var mymap = map[string]int{"foo":1, "bar":2, ... } and extract the map init code into a separate outlined function, which is then called from the main init function with a weak relocation: var mymap map[string]int // KEEP reloc -> map.init.0 func init() { map.init.0() // weak relocation } func map.init.0() { mymap = map[string]int{"foo":1, "bar":2} } The map init outlining is done selectively (only in the case where the RHS code exceeds a size limit of 20 IR nodes). In order to ensure that a given map.init.NNN function is included when its corresponding map is live, we add dummy R_KEEP relocation from the map variable to the map init function. This first patch includes the main compiler compiler changes, and with the weak relocation addition disabled. Subsequent patch includes the requred linker changes along with switching to the call to the outlined routine to a weak relocation. See the later linker change for associated compile time performance numbers. Updates golang#2559. Updates golang#36021. Updates golang#14840. Change-Id: I1fd6fd6397772be1ebd3eb397caf68ae9a3147e9 Reviewed-on: https://go-review.googlesource.com/c/go/+/461315 Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
This patch contains the linker changes needed to enable deadcoding of large unreferenced map variables, in combination with a previous compiler change. We add a new cleanup function that runs just after deadcode that looks for relocations in "init" funcs that are weak, of type R_CALL (and siblings), and are targeting an unreachable function. If we find such a relocation, after checking to make sure it targets a map.init.XXX helper, we redirect the relocation to a point to a no-op routine ("mapinitnoop") in the runtime. Compilebench results for this change: │ out.base.txt │ out.wrap.txt │ │ sec/op │ sec/op vs base │ Template 218.6m ± 2% 221.1m ± 1% ~ (p=0.129 n=39) Unicode 180.5m ± 1% 178.9m ± 1% -0.93% (p=0.006 n=39) GoTypes 1.162 ± 1% 1.156 ± 1% ~ (p=0.850 n=39) Compiler 143.6m ± 1% 142.6m ± 1% ~ (p=0.743 n=39) SSA 8.698 ± 1% 8.719 ± 1% ~ (p=0.145 n=39) Flate 142.6m ± 1% 143.9m ± 3% ~ (p=0.287 n=39) GoParser 247.7m ± 1% 248.8m ± 1% ~ (p=0.265 n=39) Reflect 588.0m ± 1% 590.4m ± 1% ~ (p=0.269 n=39) Tar 198.5m ± 1% 201.3m ± 1% +1.38% (p=0.005 n=39) XML 259.1m ± 1% 260.0m ± 1% ~ (p=0.376 n=39) LinkCompiler 746.8m ± 2% 747.8m ± 1% ~ (p=0.706 n=39) ExternalLinkCompiler 1.906 ± 0% 1.902 ± 1% ~ (p=0.207 n=40) LinkWithoutDebugCompiler 522.4m ± 21% 471.1m ± 1% -9.81% (p=0.000 n=40) StdCmd 21.32 ± 0% 21.39 ± 0% +0.32% (p=0.035 n=40) geomean 609.2m 606.0m -0.53% │ out.base.txt │ out.wrap.txt │ │ user-sec/op │ user-sec/op vs base │ Template 401.9m ± 3% 406.9m ± 2% ~ (p=0.291 n=39) Unicode 191.9m ± 6% 196.1m ± 3% ~ (p=0.052 n=39) GoTypes 3.967 ± 3% 4.056 ± 1% ~ (p=0.099 n=39) Compiler 171.1m ± 3% 173.4m ± 3% ~ (p=0.415 n=39) SSA 30.04 ± 4% 30.25 ± 4% ~ (p=0.106 n=39) Flate 246.3m ± 3% 248.9m ± 4% ~ (p=0.499 n=39) GoParser 518.7m ± 1% 520.6m ± 2% ~ (p=0.531 n=39) Reflect 1.670 ± 1% 1.656 ± 2% ~ (p=0.137 n=39) Tar 352.7m ± 2% 360.3m ± 2% ~ (p=0.117 n=39) XML 528.8m ± 2% 521.1m ± 2% ~ (p=0.296 n=39) LinkCompiler 1.128 ± 2% 1.140 ± 2% ~ (p=0.324 n=39) ExternalLinkCompiler 2.165 ± 2% 2.147 ± 2% ~ (p=0.537 n=40) LinkWithoutDebugCompiler 484.2m ± 4% 490.7m ± 3% ~ (p=0.897 n=40) geomean 818.5m 825.1m +0.80% │ out.base.txt │ out.wrap.txt │ │ text-bytes │ text-bytes vs base │ HelloSize 766.0Ki ± 0% 766.0Ki ± 0% ~ (p=1.000 n=40) ¹ CmdGoSize 10.02Mi ± 0% 10.02Mi ± 0% -0.03% (n=40) geomean 2.738Mi 2.738Mi -0.01% ¹ all samples are equal │ out.base.txt │ out.wrap.txt │ │ data-bytes │ data-bytes vs base │ HelloSize 14.17Ki ± 0% 14.17Ki ± 0% ~ (p=1.000 n=40) ¹ CmdGoSize 308.3Ki ± 0% 298.5Ki ± 0% -3.19% (n=40) geomean 66.10Ki 65.04Ki -1.61% ¹ all samples are equal │ out.base.txt │ out.wrap.txt │ │ bss-bytes │ bss-bytes vs base │ HelloSize 197.3Ki ± 0% 197.3Ki ± 0% ~ (p=1.000 n=40) ¹ CmdGoSize 228.2Ki ± 0% 228.1Ki ± 0% -0.01% (n=40) geomean 212.2Ki 212.1Ki -0.01% ¹ all samples are equal │ out.base.txt │ out.wrap.txt │ │ exe-bytes │ exe-bytes vs base │ HelloSize 1.192Mi ± 0% 1.192Mi ± 0% +0.00% (p=0.000 n=40) CmdGoSize 14.85Mi ± 0% 14.83Mi ± 0% -0.09% (n=40) geomean 4.207Mi 4.205Mi -0.05% Also tested for any linker changes by benchmarking relink of k8s "kubelet"; no changes to speak of there. Updates golang#2559. Updates golang#36021. Updates golang#14840. Change-Id: I4cc5370b3f20679a1065aaaf87bdf2881e257631 Reviewed-on: https://go-review.googlesource.com/c/go/+/463395 Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
Introduce a flag in the object file indicating whether a given function corresponds to a compiler-generated (not user-written) init function, such as "os.init" or "syscall.init". Add code to the compiler to fill in the correct value for the flag, and add support to the loader package in the linker for testing the flag. The new loader API is currently unused, but will be needed in the next CL in this stack. Updates golang#2559. Updates golang#36021. Updates golang#14840. Change-Id: Iea7ad2adda487e4af7a44f062f9817977c53b394 Reviewed-on: https://go-review.googlesource.com/c/go/+/463855 Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Than McIntosh <thanm@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
This patch changes the compiler's pkg init machinery to pick out large initialization assignments to global maps (e.g. var mymap = map[string]int{"foo":1, "bar":2, ... } and extract the map init code into a separate outlined function, which is then called from the main init function with a weak relocation: var mymap map[string]int // KEEP reloc -> map.init.0 func init() { map.init.0() // weak relocation } func map.init.0() { mymap = map[string]int{"foo":1, "bar":2} } The map init outlining is done selectively (only in the case where the RHS code exceeds a size limit of 20 IR nodes). In order to ensure that a given map.init.NNN function is included when its corresponding map is live, we add dummy R_KEEP relocation from the map variable to the map init function. This first patch includes the main compiler compiler changes, and with the weak relocation addition disabled. Subsequent patch includes the requred linker changes along with switching to the call to the outlined routine to a weak relocation. See the later linker change for associated compile time performance numbers. Updates golang#2559. Updates golang#36021. Updates golang#14840. Change-Id: I1fd6fd6397772be1ebd3eb397caf68ae9a3147e9 Reviewed-on: https://go-review.googlesource.com/c/go/+/461315 Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
This patch contains the linker changes needed to enable deadcoding of large unreferenced map variables, in combination with a previous compiler change. We add a new cleanup function that runs just after deadcode that looks for relocations in "init" funcs that are weak, of type R_CALL (and siblings), and are targeting an unreachable function. If we find such a relocation, after checking to make sure it targets a map.init.XXX helper, we redirect the relocation to a point to a no-op routine ("mapinitnoop") in the runtime. Compilebench results for this change: │ out.base.txt │ out.wrap.txt │ │ sec/op │ sec/op vs base │ Template 218.6m ± 2% 221.1m ± 1% ~ (p=0.129 n=39) Unicode 180.5m ± 1% 178.9m ± 1% -0.93% (p=0.006 n=39) GoTypes 1.162 ± 1% 1.156 ± 1% ~ (p=0.850 n=39) Compiler 143.6m ± 1% 142.6m ± 1% ~ (p=0.743 n=39) SSA 8.698 ± 1% 8.719 ± 1% ~ (p=0.145 n=39) Flate 142.6m ± 1% 143.9m ± 3% ~ (p=0.287 n=39) GoParser 247.7m ± 1% 248.8m ± 1% ~ (p=0.265 n=39) Reflect 588.0m ± 1% 590.4m ± 1% ~ (p=0.269 n=39) Tar 198.5m ± 1% 201.3m ± 1% +1.38% (p=0.005 n=39) XML 259.1m ± 1% 260.0m ± 1% ~ (p=0.376 n=39) LinkCompiler 746.8m ± 2% 747.8m ± 1% ~ (p=0.706 n=39) ExternalLinkCompiler 1.906 ± 0% 1.902 ± 1% ~ (p=0.207 n=40) LinkWithoutDebugCompiler 522.4m ± 21% 471.1m ± 1% -9.81% (p=0.000 n=40) StdCmd 21.32 ± 0% 21.39 ± 0% +0.32% (p=0.035 n=40) geomean 609.2m 606.0m -0.53% │ out.base.txt │ out.wrap.txt │ │ user-sec/op │ user-sec/op vs base │ Template 401.9m ± 3% 406.9m ± 2% ~ (p=0.291 n=39) Unicode 191.9m ± 6% 196.1m ± 3% ~ (p=0.052 n=39) GoTypes 3.967 ± 3% 4.056 ± 1% ~ (p=0.099 n=39) Compiler 171.1m ± 3% 173.4m ± 3% ~ (p=0.415 n=39) SSA 30.04 ± 4% 30.25 ± 4% ~ (p=0.106 n=39) Flate 246.3m ± 3% 248.9m ± 4% ~ (p=0.499 n=39) GoParser 518.7m ± 1% 520.6m ± 2% ~ (p=0.531 n=39) Reflect 1.670 ± 1% 1.656 ± 2% ~ (p=0.137 n=39) Tar 352.7m ± 2% 360.3m ± 2% ~ (p=0.117 n=39) XML 528.8m ± 2% 521.1m ± 2% ~ (p=0.296 n=39) LinkCompiler 1.128 ± 2% 1.140 ± 2% ~ (p=0.324 n=39) ExternalLinkCompiler 2.165 ± 2% 2.147 ± 2% ~ (p=0.537 n=40) LinkWithoutDebugCompiler 484.2m ± 4% 490.7m ± 3% ~ (p=0.897 n=40) geomean 818.5m 825.1m +0.80% │ out.base.txt │ out.wrap.txt │ │ text-bytes │ text-bytes vs base │ HelloSize 766.0Ki ± 0% 766.0Ki ± 0% ~ (p=1.000 n=40) ¹ CmdGoSize 10.02Mi ± 0% 10.02Mi ± 0% -0.03% (n=40) geomean 2.738Mi 2.738Mi -0.01% ¹ all samples are equal │ out.base.txt │ out.wrap.txt │ │ data-bytes │ data-bytes vs base │ HelloSize 14.17Ki ± 0% 14.17Ki ± 0% ~ (p=1.000 n=40) ¹ CmdGoSize 308.3Ki ± 0% 298.5Ki ± 0% -3.19% (n=40) geomean 66.10Ki 65.04Ki -1.61% ¹ all samples are equal │ out.base.txt │ out.wrap.txt │ │ bss-bytes │ bss-bytes vs base │ HelloSize 197.3Ki ± 0% 197.3Ki ± 0% ~ (p=1.000 n=40) ¹ CmdGoSize 228.2Ki ± 0% 228.1Ki ± 0% -0.01% (n=40) geomean 212.2Ki 212.1Ki -0.01% ¹ all samples are equal │ out.base.txt │ out.wrap.txt │ │ exe-bytes │ exe-bytes vs base │ HelloSize 1.192Mi ± 0% 1.192Mi ± 0% +0.00% (p=0.000 n=40) CmdGoSize 14.85Mi ± 0% 14.83Mi ± 0% -0.09% (n=40) geomean 4.207Mi 4.205Mi -0.05% Also tested for any linker changes by benchmarking relink of k8s "kubelet"; no changes to speak of there. Updates golang#2559. Updates golang#36021. Updates golang#14840. Change-Id: I4cc5370b3f20679a1065aaaf87bdf2881e257631 Reviewed-on: https://go-review.googlesource.com/c/go/+/463395 Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
The text was updated successfully, but these errors were encountered: