When loading packages from source, many files are being parsed
repeatedly, for example due to test variants. While the median number of
times a file gets parsed is 2, it is significantly higher (up to 28
times) when parsing the standard library, because of test variant
shenanigans.
By caching file contents and their parsed representations we can cut
down on processing time and garbage produced. When loading individual
packages or 3rd party projects, the effect is rather small. However when
loading the entire standard library, the effect is substantial.
name old time/op new time/op delta
Jaeger-8 2.95s ± 7% 2.84s ± 8% ~ (p=0.089 n=10+10)
Std-8 4.96s ± 7% 4.23s ± 3% -14.62% (p=0.000 n=9+9)
Strconv-8 892ms ±34% 877ms ±21% ~ (p=0.853 n=10+10)
name old alloc/op new alloc/op delta
Jaeger-8 1.22GB ± 0% 1.21GB ± 0% -0.84% (p=0.000 n=10+10)
Std-8 2.57GB ± 0% 2.20GB ± 0% -14.61% (p=0.000 n=10+8)
Strconv-8 201MB ± 1% 200MB ± 1% ~ (p=0.105 n=10+10)
name old allocs/op new allocs/op delta
Jaeger-8 12.7M ± 0% 12.4M ± 0% -1.82% (p=0.000 n=9+10)
Std-8 26.4M ± 0% 17.3M ± 0% -34.62% (p=0.000 n=9+9)
Strconv-8 1.94M ± 0% 1.91M ± 0% -1.50% (p=0.000 n=10+10)
When loading std, peak RSS decreases from 1.96 GB to 1.57 GB.
While we're here, we simplify our ParseFile implementation. The contract
of ParseFile specifies that implementations must use src for parsing,
and use filename only for display purposes. As such, we mustn't ever
call it with a nil src, making the check for a nil src in our own
implementation superfluous.
Change-Id: I33daac20fc52ccdb3187a336633f712d01b71d86
Reviewed-on: https://go-review.googlesource.com/c/tools/+/171377
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
cb2dda6eab