If go/packages.Load tries to load a package with a missing dependency, it
should not return an error, but a package with an error set on it. This is a
workaround for go list -e -compiled (or even just go list -e) returning a
non-zero exit status for packages with missing dependencies.
Change-Id: I2d7d848ae5133235f595baf7b30296077e891ee3
Reviewed-on: https://go-review.googlesource.com/c/tools/+/170891
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This change adds support in go/packages for defining an entire package
in an overlay. We also add corresponding tests for this in gopls, to
confirm that it works as expected.
Fixesgolang/go#31467
Change-Id: Iead203ab2964a7ac4f571be97624b725ac5de7e0
Reviewed-on: https://go-review.googlesource.com/c/tools/+/172409
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
If a file in an ad-hoc package doesn't exist, go list should exit 0 and
return an dummy package with an error set on it. Since it doesn't do that
yet, add a work-around.
Updates golang/go#29280
Change-Id: I6019f28ce4770582f274919d1aa35d85a634687e
Reviewed-on: https://go-review.googlesource.com/c/tools/+/171018
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This code fixes a bug when a user specifies NeedTypes, which
implicitly requires TypesSizes, but TypesSizes isn't fetched,
which causes typechecking to explode.
Also fix a similar issue where NeedImports isn't implicitly
fetched for NeedDeps.
I added a TODO for a better fix, which is to have an "implicitMode"
in the loader type containing all the data that's needed as a prerequisite
for other fields. Then we can use implicitMode when fetching data,
and cfg.Mode to clear out the fields the user didn't request.
Fixesgolang/go#31163
Change-Id: If3506765470af43dfb24d06fcbd31b66a623f2e0
Reviewed-on: https://go-review.googlesource.com/c/tools/+/170342
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This fixes a naming preference missed in CL 166537.
Since func golistDriverCurrent is no longer polymorphic, it was
suggested we simply call it golistDriver.
Change-Id: Ibf517365c20953628ede3457b9247efd8b79897c
Reviewed-on: https://go-review.googlesource.com/c/tools/+/168837
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
The options are all unexported, and this CL is (almost) a no-op:
the one difference is that since needImports and needSyntax are now
independently specified, LoadSyntax and LoadAllSyntax are equivalent,
because LoadSyntax needs both the needImports and needSyntax bits.
I want to pin down the options that we want to split into, and
future CLs can allow the options to be used individually...
Updates golang/go#29429
Updates golang/go#29427
Change-Id: I5b2913e2c53e7ade56905e46912b076ccc339827
Reviewed-on: https://go-review.googlesource.com/c/tools/+/162140
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
exec.Command already runs exec.LookPath when given a name that contains
no path separators. There's no need to call exec.LookPath a second time
to detect that cmd.Run failed because of a missing executable file.
It can be detected from the returned error. Do so because it's cleaner.
Also improve the error text to say that the problem was that the go
executable file was not found in $PATH (or %PATH%, etc., depending on
the underlying operating system). In the general case, we can't know if
Go is or isn't installed.
Example error text on macOS:
gopackages: 'go list' driver requires 'go', but executable file not found in $PATH
Updates golang/go#29552
Change-Id: I769553f125240dccd02098c22641f6a1ed10063c
Reviewed-on: https://go-review.googlesource.com/c/tools/+/168897
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
If we can't execute the go command, check to see if it exists. If it doesn't,
say so in the error.
Fixesgolang/go#29552
Change-Id: Iffd2db5fc8f1daa2e458eaa326b27d9a0d971b6b
Reviewed-on: https://go-review.googlesource.com/c/tools/+/168777
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Make it slightly more clear that this is not a user error, but an
internal error.
Updates golang/go#30519
Change-Id: I7adb3b5bb1548eab8e46db48946d55f9d59a4311
Reviewed-on: https://go-review.googlesource.com/c/tools/+/168657
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
go list functionality changed in 1.11 and compatibility logic was added
to support 1.10 and before. Now that Go 1.12 has been released, support
for those version has ended and we can remove the legacy code.
Change-Id: Ifdd5c566dbbfe4fade5be27ad9ae20052d604c15
Reviewed-on: https://go-review.googlesource.com/c/tools/+/166537
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
This CL fixes packages ignoring errors regarding files that have
non .go extensions. Packages can be called with just file names
or path which includes files. These aren't checked at all by
packages if they are go files or not, but it fails silently because
of it.
In more detail, go list fails with named files error in STDERR.
However, that is ignored, because go list notoriously abused STDERR
for non-error messages.
Fixesgolang/go#29899
Change-Id: Ie4dc39da0b87200ebd23e6c607396557685e2807
Reviewed-on: https://go-review.googlesource.com/c/tools/+/164663
Reviewed-by: Michael Matloob <matloob@golang.org>
Address a few irritating glitches in the go list debug logging.
- Print a fully runnable command line, with args like "a" "b" "c"
instead of [a b c].
- Include stderr in the debug logs for cases where the command fails.
- Print the correct PWD environment var from cmd instead of cfg.
Change-Id: I58e77b370baf8378a21377b81ee2ba5d21a557ab
Reviewed-on: https://go-review.googlesource.com/c/163497
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
It was originally added to support goimports, but goimports
is going with another solution. We're going to disable it for
now, but not delete it, so that the goimports code path that
uses it can continue to be tested if and when we want to
use it.
We don't think there are any other users of name= but if
there are, please let us know and we'll work with you to
fix you, or we'll stop or revert this change. Thanks!
Change-Id: I73b7b6c0a5788148af5f3380189055b450f7b45e
Reviewed-on: https://go-review.googlesource.com/c/159702
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
go/packages needs to call `go list` multiple times, which causes
redundant work and slows down goimports. If we reimplement `go list` in
memory, we can reuse state, saving time. `go list` also does work we
don't really need, like adding stuff to go.mod, and skipping that saves
more time.
We start with `go list -m`, which does MVS and such. The remaining work
is mostly mapping import paths and directories through the in-scope
modules to make sure we're giving the right answers. Unfortunately this
is quite subtle, and I don't know where all the traps are. I did my
best.
cmd/go already has tests for `go list`, of course, and packagestest is
not well suited to tests of this complexity. So I ripped off the script
tests in cmd/go that seemed relevant and made sure that our logic
returns the right stuff in each case. I'm sure that there are more cases
to cover, but this hit all the stuff I knew about and quite a bit I
didn't.
Since we may want to use the go/packages code path in the future, e.g.
for Bazel, I left that in place. It won't be used unless the magic env
var is set.
Files in internal and imports/testdata/mod were copied verbatim from
cmd/go.
Change-Id: I1248d99c400c1a0c7ef180d4460b9b8a3db0246b
Reviewed-on: https://go-review.googlesource.com/c/158097
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
At least on the go list driver, which is the default fallback. Note that
we need to call 'go list' on Load("foo") and Load(), but not
Load("name=bar"). This is what the current logic was trying to
accomplish.
But it didn't take into account the case where there are zero initial
patterns, in which case we should still call 'go list'. Fix that, and
add a test.
Fixes#28767.
Change-Id: I40af9eb7f2407449c5683df1403928e2c57c86a4
Reviewed-on: https://go-review.googlesource.com/c/155898
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Pass -find whenever possible to speed up go list calls. Add an
environment variable, GOPACKAGESDEBUG, that controls debug logging
so that we don't have to tell users to recompile goimports to debug it.
Change-Id: If39ff7829279dafa4e066e74a024c27a8235154b
Reviewed-on: https://go-review.googlesource.com/c/155477
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
refine needs to match packages back to roots, and it gets confused if
the same root appears twice. Deduplicate roots the same way we
deduplicate packages.
Introduce a new wrapper around driverResponse to make this easier, and
pass that around instead of the addPackage callback.
Fixesgolang/go#29297
Change-Id: I49ea37155c507af136391b9eb55a83b6dedfcc14
Reviewed-on: https://go-review.googlesource.com/c/155020
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Also fix packagesdriver to use "go list" if the gopackagesdriver
program is not available, rather than returning nil with no error.
Also fix some comments.
Change-Id: I0457cfa4eab4294760af71c780c62c05ad841c85
Reviewed-on: https://go-review.googlesource.com/c/154757
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
files specified in file= queries will now also be checked for in overlay
packages.
Fixesgolang/go#29048
Change-Id: I096d02c3777e6c7decedfcb97c0d011ce9b47b28
Reviewed-on: https://go-review.googlesource.com/c/152797
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This support is not perfect, but should produce more accurate results than
the previous behavior, which is to silently drop those files.
Change-Id: Ia00c042bf303e9ec0fb1cbd579c0fccb29073de0
Reviewed-on: https://go-review.googlesource.com/c/151999
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
When running go list for a file= query, the go/packages code previously ran "go list ." command in the directory of the provided file. This was necessary for compatibility with the fallback, but caused problems when running file= queries for files in the module cache. We now use absolute directory paths (supported in Go >=1.11) in the non-fallback case and retain the previous behavior in the fallback case.
Change-Id: I3a9feaaeda12e068ec3d510b5f78437482c65b4c
Reviewed-on: https://go-review.googlesource.com/c/148199
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This is a workaround for a go list regression that broke
go/packages but went unnoticed by because of a missing
call to packages.PrintErrors, added here.
Updates golang/go#28749
Change-Id: I1819a6143134a422791106ac037d3458ef864322
Reviewed-on: https://go-review.googlesource.com/c/149237
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This will allow Sizes to be marshalled and unmarshalled. All the Sizes
we care about ane StdSizes anyways.
Change-Id: I79d1dcaebba32f7730de4375945e372eeefa78fe
Reviewed-on: https://go-review.googlesource.com/c/147978
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
This is more reliable than looking in the environment.
Change-Id: I96c093b89faaece6b6256eefb4a4aac4d66b9cc9
Reviewed-on: https://go-review.googlesource.com/c/146757
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
This is based on 147340 and has the same test, but with a different fix.
We now produce a nice error if there is an empty root, instead of
crashing, and the cause of empty roots in go list has been fixed.
The underlying call to go list is returning the same package more than
once, and we only fix the first entry in the root list, so the second
one got left blank.
The fix was to not add the second duplicate copy to the output of the
driver at all.
Change-Id: I9f1b2f0fd63635ba101cdd3c8a5108530e968ba9
Reviewed-on: https://go-review.googlesource.com/c/147440
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
For performance, bail out of runNamedQueries before running go env if
there's nothing to do.
Add t.Helper() calls to packagestest.TestAll.
Escape # in packagestest.Export. The testing package adds #NN suffixes
to subtests that have redundant names.
Log how long gopathwalk.Walk took for name= queries when debug is on.
Change-Id: I37cb0ed11cf58e1693e29dea04697e5885ecc62b
Reviewed-on: https://go-review.googlesource.com/c/147203
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
When the name= query constructs the temporary module, it may find things
that don't resolve. In at least some cases (#28518), allowing go list
to access the network results in not just bad performance but actual
failures. Default GOPROXY to "off" when doing queries on the temporary
module to try to address that.
Also, add some more debug logging, including various environment
variables, so that it's easier to reproduce failing commands.
Change-Id: I1a6d3ffa5c845271ce48e9fe802a2491ccadcd7c
Reviewed-on: https://go-review.googlesource.com/c/146477
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Treat the top-level package of a module specially so that the VCS
working dir name doesn't have to match the package name. Generally we
take the position that directory names should match packages, but for
the top-level package that's a pretty strict requirement. Now we always
check the top-level directory, just in case.
Change-Id: I2e96751bf46736fbaef434fa9788fdc1339c37b9
Reviewed-on: https://go-review.googlesource.com/c/144817
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The builders are currently flaky failing because of a root order being
wrong.
This causes all root lists to be sorted before being returned, so the
order is always stable no matter which underlying driver is running.
Fixesgolang/go#27594
Change-Id: I09db45c67ad00f23dfaec8e271acbd13fc338888
Reviewed-on: https://go-review.googlesource.com/c/143737
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Darwin convertes the working directory to a real path which affects the
path a command is started with when you exec.
This causes a problem where the path the go list command is run with
does not match the path the outside program is trying to use, which
causes both undesirably changed paths in it's outputs, and sometimes
failures to cope with modules when source files are "outside" the module
root.
The go standard library has a special feature in os.Getwd where it
checks if the PWD environment variable is the same inode as the system
cwd, and if it is it returns the PWD instead. This change deliberately
sets the PWD before running go list so that behaviour is triggered,
which fixes all the paths.
This was breaking the mac build bots in a goimports test, it should be
fixed now.
Change-Id: I0f5d3c7d020b55749738036ba51c19884bb26598
Reviewed-on: https://go-review.googlesource.com/c/143517
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Some editors combine stdout and stderr when running tools, so
printing to stderr without failing confuses them. Stop printing go
list's warnings.
They're still useful for debugging, so users can set
GOPACKAGESPRINTGOLISTERRORS to see them. This isn't part of the API and
has no compatibility guarantees.
Change-Id: I9cf091cf59d082123149888468fa0d126dc972c7
Reviewed-on: https://go-review.googlesource.com/c/142997
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add an implementation of name= for go list. It will be used to
implement goimports and godoc-like lookups by package name.
Imported a copy of the semver package from the stdlib to do version
comparison, and tweaked the gopathwalk API to include a hint about what
kind of source directory is being traversed.
Note that the tests, despite my best efforts, are not hermetic: go list
insists on doing version lookups in situations where it seems to me like
it shouldn't need to.
I think this implementation is ready for serious use. The one thing I'm
nervous about is that it currently does a substring match when looking
for a package name, so if you look up a package named "a" you will get
a huge number of results. This matches goimports' behavior but I don't
know if it's suitable for general use.
Change-Id: I2b7f823b74571fe30d3bd9c7dfafb4e6a40df5d3
Reviewed-on: https://go-review.googlesource.com/c/138878
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
This change updates the language accepted through the arguments
to packages.Load to make it more consistent. There are now two layers.
A pattern containing an "=" is considered to be a special query, and the
part of the pattern up to the first "=" is considered the query type. All
other patterns are to be interpreted as the build system interprets it.
For now two special queries exist. file= has the behavior that contains:
did: finding packages containing the given file. The query type pattern=
is used to pass through a pattern to be interpreted by the build system.
pattern= is a type of escaping to allow passing through patterns that
contain an "=" to be interpreted by the underlying buildsystem. To allow
for new query types to be introduced, packages.Load will report an
error if the qury type is not understood. We expect name= to be added in
an upcoming change.
"contains:" changes to "file=". A new
Change-Id: I1b208d1c998c67d5556cdc872d7694273cedb7e4
Reviewed-on: https://go-review.googlesource.com/c/141681
Reviewed-by: Alan Donovan <adonovan@google.com>
The documentation for Load says:
“Load returns an error if any of the patterns was invalid as defined
by the underlying build system. It may return an empty list of
packages without an error, for instance for an empty expansion of a
valid wildcard. Errors associated with a particular package are
recorded in the corresponding Package's Errors list, and do not cause
Load to return an error.”
Therefore, it should not be an error for a pattern to match no
packages. If the pattern is a literal package path that does not
exist, we should prefer to return a *Package for it with an error in
the Errors field.
Change-Id: Iaecfb920097e3b520e763bd52c0e326d2e7a4861
Reviewed-on: https://go-review.googlesource.com/137075
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
This change adds the -buildflag argument to go/packages/gopackages,
which may be passed repeatedly to set config.BuildFlags. I felt -flag
was too vague.
config.Flags is renamed to config.BuildFlags for consistency, and
packages.findExternalDriver now passes -buildflag instead of -flag to
drivers. This will break existing drivers.
Change-Id: Iaed58026373a46e137a236ee9a652eb3a9433ee3
Reviewed-on: https://go-review.googlesource.com/130136
Reviewed-by: Alan Donovan <adonovan@google.com>
For any identity imports to be added to the import map of a package,
make sure these are not already in there. If the package imports "C",
be sure not to overwrite the import map entries for "runtime/cgo", "unsafe",
and "syscall".
Test the import map when test variants are loaded. This tests the fix
in 'go list' made by https://golang.org/cl/128836Fixesgolang/go#26847
Change-Id: I05dff4d3a75fab03f333f6d88128de6a6bf169e6
Reviewed-on: https://go-review.googlesource.com/128876
Run-TryBot: Suzy Mueller <suzmue@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
This deletes the raw and golist packatges, instead using the main
Package structure as the serialzied form between the build tool and the
loader.
This requires a few new fields on Package for information that we used
to hide, and some extra json tags on Package.
It also required Package to have custom JSON marshalling methods.
Change-Id: Ib72171bc93e2b494b97f4b0266504acaa9b3f647
Reviewed-on: https://go-review.googlesource.com/128120
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
We want to make it public to enable external sources of information
(other than go list), but we don't want to clutter the main
go/packages API with things that are intended for implementers only.
We also want to move the two API's from experimental to stable at
different times.
We also need to split the API because the golist implementation has to depend on
the raw structs, and then the main packages.Load has to depend on the golist
implementation.
The golist functionality is now in it's own package, and provides a Load method
to return the raw structures. This can be re-used by more complex raw providers.
Change-Id: I767af216b0d1ad9391e613cbf24eb2db5f503369
Reviewed-on: https://go-review.googlesource.com/126718
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
All the other members of raw package are stable for any given package,
DepOnly relates to the query patterns, not thepackages.
Instead the raw functions now return the set of roots matched
Other minor changes included:
rawConfig.ExtraFlags -> rawConfig.Flags
delete rawConfig.Context
Change-Id: I3a729fa4557043f65da386eed94ab46fdfacb8ad
Reviewed-on: https://go-review.googlesource.com/127035
Reviewed-by: Michael Matloob <matloob@golang.org>
The go/loader based implementation was too different. Instead
Just call go list twice to get all the dependencies. This immediately
enables more functionality.
Change-Id: I22883492e996133bc61664314318c5e0a9cc10dc
Reviewed-on: https://go-review.googlesource.com/125939
Reviewed-by: Suzy Mueller <suzmue@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This separates the go list specific behavior from the generalised go/packages
loading behaviour, to enable alternate build system back ends.
Change-Id: I07e7649f8f2afc7470a3ee3d0d743cbbcc6f713e
Reviewed-on: https://go-review.googlesource.com/125715
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
After a discussion in cl/125535 this is a proposal to clean up some
internal function signatures that were becomming unwieldy.
Change-Id: I37a9e525ab18dface9ea9fb0b0ec68ac942ee632
Reviewed-on: https://go-review.googlesource.com/125659
Reviewed-by: Michael Matloob <matloob@golang.org>