https://golang.org/issue/33157 explains the issues with overlays. The
gopls tests caught this bug, but the go/packages tests didn't, so modify
the go/packages tests correspondingly.
Change-Id: I8ea8e06e145aa2420655cbe4884e60f36acfad7b
Reviewed-on: https://go-review.googlesource.com/c/tools/+/186299
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This change adds a Logf field to the packages.Config.
Change-Id: I144a9a1e1181585bbe621898c4a3e6a007a38322
Reviewed-on: https://go-review.googlesource.com/c/tools/+/185993
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
This adds a comment in go/packages/external.go that specifies what the
driver protocol is.
Change-Id: Ie0c272a84cd34ba80f80f68b328463d8ddd07189
Reviewed-on: https://go-review.googlesource.com/c/tools/+/184943
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
In some cases, it's safe to avoid loading additional packages
when computing overlays. However it's not always safe to do so.
Avoid some unnecessary loads when it's completely safe to do so.
Updates golang/go#32538
Change-Id: Ie12204735940a540c9b3f29742f8479bcab5f077
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181917
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
The test was wrong and the code was wrong.
Change-Id: I4ffe4bb4788912210b307a06e168629c6800d0fb
Reviewed-on: https://go-review.googlesource.com/c/tools/+/184043
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
go list sometimes sticks a leading newline in its error messages
have go/packages trim them until go list does the right thing
(of course, no one should depend on this behavior)
Fixesgolang/go#32363
Change-Id: I6e145fb85bfc9d710c5f06146a64ec6919f59e36
Reviewed-on: https://go-review.googlesource.com/c/tools/+/183258
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dominik Honnef <dominik@honnef.co>
This change retries the query even if go/packages returns an error, not
just an empty response.
This also fixes support for ad-hoc packages in gopls.
Change-Id: I7bc07c225568efd18f4e5759f7eb3c23cd12bfa4
Reviewed-on: https://go-review.googlesource.com/c/tools/+/182580
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
This will do a go list using a filename directly if listing using
the directory failed.
Fixesgolang/go#32587
Change-Id: Id9993968f0ebc18a455132e0f1468356416a66dd
Reviewed-on: https://go-review.googlesource.com/c/tools/+/182465
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
so that they don't refer to the deprecated LoadX LoadModes
Also break apart the NeedX from the LoadX fields so that the needX fields
are rendered in the godoc next to the definition of LoadMode
Fixesgolang/go#32364
Change-Id: Ic8837950a5e25937c556b62fbedbd8dc5356cfdb
Reviewed-on: https://go-review.googlesource.com/c/tools/+/179741
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dominik Honnef <dominik@honnef.co>
Refactor the overlay code to create package structs for
new packages that don't already exist. This requires
calling out to the go command to determine module
roots to figure out which module a package belongs to.
The extra go list call is done in sequence in this CL
but can easily be done in parallel with other go list
calls in the future.
Change-Id: Ia0f7812fba250d154033038cb1e2afa7dedf0e16
Reviewed-on: https://go-review.googlesource.com/c/tools/+/179600
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Update doc to specify that users should request the fields needed in the
Package struct by using the NeedX bits directly.
Change-Id: I9213827fa7e5d01800d79173fe5161f39ffda85e
Reviewed-on: https://go-review.googlesource.com/c/tools/+/173959
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
If go/packages.Load tries to load a package with a missing dependency, it
should not return an error, but a package with an error set on it. This is a
workaround for go list -e -compiled (or even just go list -e) returning a
non-zero exit status for packages with missing dependencies.
Change-Id: I2d7d848ae5133235f595baf7b30296077e891ee3
Reviewed-on: https://go-review.googlesource.com/c/tools/+/170891
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Users of packagestest will create modules that don't exist on the
Internet and can change. There's no point in checking sum.golang.org for
them under any circumstances.
Similarly for the various goimports tests.
Fixesgolang/go#32216.
Change-Id: Id9a6b660564cb744530bf9d209fca19008fb9c4f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/178722
Reviewed-by: Ian Cottrell <iancottrell@google.com>
For various reasons we need an internal-facing imports API. Move imports
to internal/imports, leaving behind a small wrapper package. The wrapper
package captures the globals at time of call into the options struct.
Also converts the last goimports tests to use the test helpers, and
fixes go/packages in module mode to work with empty modules, which was
necessary to get those last tests converted.
Change-Id: Ib1212c67908741a1800b992ef1935d563c6ade32
Reviewed-on: https://go-review.googlesource.com/c/tools/+/175437
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Amends prematurely submitted CL 173918. We now use file:/// URLs for
go1.13 and later and file:// URLs for go1.12 and earlier.
Fixesgolang/go#31675
Change-Id: I009c63a900bdfd091bf46def5cea5a0843639b47
Reviewed-on: https://go-review.googlesource.com/c/tools/+/173919
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
On Windows, file URLs should look like file:///C:/foo/bar instead of
file://C:/foo/bar. In the latter case, the "C:" is parsed as the host.
Updates golang/go#31675
Change-Id: I7f75be44dd5d289de3ffdbd20a78130ed03cd233
Reviewed-on: https://go-review.googlesource.com/c/tools/+/173918
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
When loading packages from source, many files are being parsed
repeatedly, for example due to test variants. While the median number of
times a file gets parsed is 2, it is significantly higher (up to 28
times) when parsing the standard library, because of test variant
shenanigans.
By caching file contents and their parsed representations we can cut
down on processing time and garbage produced. When loading individual
packages or 3rd party projects, the effect is rather small. However when
loading the entire standard library, the effect is substantial.
name old time/op new time/op delta
Jaeger-8 2.95s ± 7% 2.84s ± 8% ~ (p=0.089 n=10+10)
Std-8 4.96s ± 7% 4.23s ± 3% -14.62% (p=0.000 n=9+9)
Strconv-8 892ms ±34% 877ms ±21% ~ (p=0.853 n=10+10)
name old alloc/op new alloc/op delta
Jaeger-8 1.22GB ± 0% 1.21GB ± 0% -0.84% (p=0.000 n=10+10)
Std-8 2.57GB ± 0% 2.20GB ± 0% -14.61% (p=0.000 n=10+8)
Strconv-8 201MB ± 1% 200MB ± 1% ~ (p=0.105 n=10+10)
name old allocs/op new allocs/op delta
Jaeger-8 12.7M ± 0% 12.4M ± 0% -1.82% (p=0.000 n=9+10)
Std-8 26.4M ± 0% 17.3M ± 0% -34.62% (p=0.000 n=9+9)
Strconv-8 1.94M ± 0% 1.91M ± 0% -1.50% (p=0.000 n=10+10)
When loading std, peak RSS decreases from 1.96 GB to 1.57 GB.
While we're here, we simplify our ParseFile implementation. The contract
of ParseFile specifies that implementations must use src for parsing,
and use filename only for display purposes. As such, we mustn't ever
call it with a nil src, making the check for a nil src in our own
implementation superfluous.
Change-Id: I33daac20fc52ccdb3187a336633f712d01b71d86
Reviewed-on: https://go-review.googlesource.com/c/tools/+/171377
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
This change adds support in go/packages for defining an entire package
in an overlay. We also add corresponding tests for this in gopls, to
confirm that it works as expected.
Fixesgolang/go#31467
Change-Id: Iead203ab2964a7ac4f571be97624b725ac5de7e0
Reviewed-on: https://go-review.googlesource.com/c/tools/+/172409
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
If a file in an ad-hoc package doesn't exist, go list should exit 0 and
return an dummy package with an error set on it. Since it doesn't do that
yet, add a work-around.
Updates golang/go#29280
Change-Id: I6019f28ce4770582f274919d1aa35d85a634687e
Reviewed-on: https://go-review.googlesource.com/c/tools/+/171018
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This code fixes a bug when a user specifies NeedTypes, which
implicitly requires TypesSizes, but TypesSizes isn't fetched,
which causes typechecking to explode.
Also fix a similar issue where NeedImports isn't implicitly
fetched for NeedDeps.
I added a TODO for a better fix, which is to have an "implicitMode"
in the loader type containing all the data that's needed as a prerequisite
for other fields. Then we can use implicitMode when fetching data,
and cfg.Mode to clear out the fields the user didn't request.
Fixesgolang/go#31163
Change-Id: If3506765470af43dfb24d06fcbd31b66a623f2e0
Reviewed-on: https://go-review.googlesource.com/c/tools/+/170342
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This fixes a naming preference missed in CL 166537.
Since func golistDriverCurrent is no longer polymorphic, it was
suggested we simply call it golistDriver.
Change-Id: Ibf517365c20953628ede3457b9247efd8b79897c
Reviewed-on: https://go-review.googlesource.com/c/tools/+/168837
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
This change moves the NeedTypesSizes bit to LoadTypes instead of LoadSyntax.
Fixesgolang/go#31163
Change-Id: Icaf16639202533fbb2190756a325b36d8ac9251c
Reviewed-on: https://go-review.googlesource.com/c/tools/+/170016
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change also fixes the corresponding code in go/packages, which was
actually not filling in the TypesSizes if the bit was set.
Change-Id: I2d5a849045768a81c94218eb41da2faec26189a3
Reviewed-on: https://go-review.googlesource.com/c/tools/+/170010
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
The options are all unexported, and this CL is (almost) a no-op:
the one difference is that since needImports and needSyntax are now
independently specified, LoadSyntax and LoadAllSyntax are equivalent,
because LoadSyntax needs both the needImports and needSyntax bits.
I want to pin down the options that we want to split into, and
future CLs can allow the options to be used individually...
Updates golang/go#29429
Updates golang/go#29427
Change-Id: I5b2913e2c53e7ade56905e46912b076ccc339827
Reviewed-on: https://go-review.googlesource.com/c/tools/+/162140
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Stop fighting the behavior of the go tool when run in these directories.
Updates golang/go#30790
Change-Id: I32dfeb0bafa3ed3664500f1768b2293e5257d09b
Reviewed-on: https://go-review.googlesource.com/c/tools/+/168757
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
exec.Command already runs exec.LookPath when given a name that contains
no path separators. There's no need to call exec.LookPath a second time
to detect that cmd.Run failed because of a missing executable file.
It can be detected from the returned error. Do so because it's cleaner.
Also improve the error text to say that the problem was that the go
executable file was not found in $PATH (or %PATH%, etc., depending on
the underlying operating system). In the general case, we can't know if
Go is or isn't installed.
Example error text on macOS:
gopackages: 'go list' driver requires 'go', but executable file not found in $PATH
Updates golang/go#29552
Change-Id: I769553f125240dccd02098c22641f6a1ed10063c
Reviewed-on: https://go-review.googlesource.com/c/tools/+/168897
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
If we can't execute the go command, check to see if it exists. If it doesn't,
say so in the error.
Fixesgolang/go#29552
Change-Id: Iffd2db5fc8f1daa2e458eaa326b27d9a0d971b6b
Reviewed-on: https://go-review.googlesource.com/c/tools/+/168777
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Make it slightly more clear that this is not a user error, but an
internal error.
Updates golang/go#30519
Change-Id: I7adb3b5bb1548eab8e46db48946d55f9d59a4311
Reviewed-on: https://go-review.googlesource.com/c/tools/+/168657
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
go list functionality changed in 1.11 and compatibility logic was added
to support 1.10 and before. Now that Go 1.12 has been released, support
for those version has ended and we can remove the legacy code.
Change-Id: Ifdd5c566dbbfe4fade5be27ad9ae20052d604c15
Reviewed-on: https://go-review.googlesource.com/c/tools/+/166537
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
This CL fixes packages ignoring errors regarding files that have
non .go extensions. Packages can be called with just file names
or path which includes files. These aren't checked at all by
packages if they are go files or not, but it fails silently because
of it.
In more detail, go list fails with named files error in STDERR.
However, that is ignored, because go list notoriously abused STDERR
for non-error messages.
Fixesgolang/go#29899
Change-Id: Ie4dc39da0b87200ebd23e6c607396557685e2807
Reviewed-on: https://go-review.googlesource.com/c/tools/+/164663
Reviewed-by: Michael Matloob <matloob@golang.org>
Some of the tests on the name query operate on test modules or module cache
trees that are checked in as testdata. When the tests run, they can modify
the go.mod files and the cache tree directories. Copy the directories
to a temporary directory to avoid getting spurious git diffs showing up.
Change-Id: I991a4510201988d596833faea88425a335d3228b
Reviewed-on: https://go-review.googlesource.com/c/tools/+/167859
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
This is mostly to allow access to exported file contents for tests
that need the source.
Change-Id: I0ef946d7bdd971b931e509d2cb54e2c59649fe47
Reviewed-on: https://go-review.googlesource.com/c/tools/+/166883
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Our tests depend on the dependency graphs of the errors which
is not guaranteed to have a stable dependency graph across go versions.
Especially because we're planning on making changes to the errors package
in Go 1.13.
Instead, use the container/list package, which is completely useless
and won't be changed (unless/until Go gets generics).
Fixesgolang/go#30448
Change-Id: Ia5f4853d1da336dde2f025b1dd5e1d6223571dd6
Reviewed-on: https://go-review.googlesource.com/c/164298
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Address a few irritating glitches in the go list debug logging.
- Print a fully runnable command line, with args like "a" "b" "c"
instead of [a b c].
- Include stderr in the debug logs for cases where the command fails.
- Print the correct PWD environment var from cmd instead of cfg.
Change-Id: I58e77b370baf8378a21377b81ee2ba5d21a557ab
Reviewed-on: https://go-review.googlesource.com/c/163497
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Rebecca Stambler
Michael Matloob
Heschi Kreinick
Ian Cottrell
Jay Conrod
Dominik Honnef
Brad Fitzpatrick
Colin Arnott
Zac Bergquist
Dmitri Shuralyov
Gergely Brautigam
Elias Naur
Josh Bleecher Snyder