Pass -find whenever possible to speed up go list calls. Add an
environment variable, GOPACKAGESDEBUG, that controls debug logging
so that we don't have to tell users to recompile goimports to debug it.
Change-Id: If39ff7829279dafa4e066e74a024c27a8235154b
Reviewed-on: https://go-review.googlesource.com/c/155477
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Added a View interface to the source package, which allows for reading
of other files (in the same package or in other packages). We were
already reading files in jump to definition (to handle the lack of
column information in export data), but now we can also read files in
diagnostics, which allows us to determine the end of an identifier so
that we can report ranges in diagnostic messages.
Updates golang/go#29150
Change-Id: I7958d860dea8f41f2df88a467b5e2946bba4d1c5
Reviewed-on: https://go-review.googlesource.com/c/154742
Reviewed-by: Ian Cottrell <iancottrell@google.com>
refine needs to match packages back to roots, and it gets confused if
the same root appears twice. Deduplicate roots the same way we
deduplicate packages.
Introduce a new wrapper around driverResponse to make this easier, and
pass that around instead of the addPackage callback.
Fixesgolang/go#29297
Change-Id: I49ea37155c507af136391b9eb55a83b6dedfcc14
Reviewed-on: https://go-review.googlesource.com/c/155020
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Also fix packagesdriver to use "go list" if the gopackagesdriver
program is not available, rather than returning nil with no error.
Also fix some comments.
Change-Id: I0457cfa4eab4294760af71c780c62c05ad841c85
Reviewed-on: https://go-review.googlesource.com/c/154757
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
This is needed to write the more advanced test cases for editor integrated
tools.
The expectation system also uses the overlay rather than the file if it is
supplied.
Change-Id: I8fd21f4efe5ac5869fa6e25d3cd0d5096051e5e5
Reviewed-on: https://go-review.googlesource.com/c/153240
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
we check length of the map, and a non empty map forces source mode.
Change-Id: Id314ad87b96d9154198122586fd8672a70c5ca95
Reviewed-on: https://go-review.googlesource.com/c/153198
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
files specified in file= queries will now also be checked for in overlay
packages.
Fixesgolang/go#29048
Change-Id: I096d02c3777e6c7decedfcb97c0d011ce9b47b28
Reviewed-on: https://go-review.googlesource.com/c/152797
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This support is not perfect, but should produce more accurate results than
the previous behavior, which is to silently drop those files.
Change-Id: Ia00c042bf303e9ec0fb1cbd579c0fccb29073de0
Reviewed-on: https://go-review.googlesource.com/c/151999
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Our tests compare import graphs from go/packages to expected graphs,
and one of the test cases imports math/bits. But in tip math/bits
picked up a dependency on unsafe, which means the expected graph
is different when run against a Go version >= go1.11. Remove that edge
before comparing against the expected graph to work around the breakage.
Change-Id: Ic586a75ba530741d251df9f87d0817a8e37d92ea
Reviewed-on: https://go-review.googlesource.com/c/151657
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
When running go list for a file= query, the go/packages code previously ran "go list ." command in the directory of the provided file. This was necessary for compatibility with the fallback, but caused problems when running file= queries for files in the module cache. We now use absolute directory paths (supported in Go >=1.11) in the non-fallback case and retain the previous behavior in the fallback case.
Change-Id: I3a9feaaeda12e068ec3d510b5f78437482c65b4c
Reviewed-on: https://go-review.googlesource.com/c/148199
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This is needed because testing.TB does not have a "Run" method for sub
tests, and the signature of the function it takes needs to match the
testing parameter.
Change-Id: I9201dd75891ed1bda300a4247fe7eac1e25bd30d
Reviewed-on: https://go-review.googlesource.com/c/150857
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Pass in parameters to the driver in stdin rather than through argv.
This allows us to more easily route parameters to the driver and add
more parameters without breaking users. The driver doesn't need to
have the same interface as go list, because it's unlikely the driver interface
will converge with the go list interface.
We still pass in the query "words" to the driver in the argv, because
that's as good a place as any, but there's no "list" command.
Question: should we add a "command" option to the driverRequest struct?
Change-Id: Ifdbb3f84b6bfd04259f5ab63e756341d7f69de9b
Reviewed-on: https://go-review.googlesource.com/c/150337
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This makes our internal version of go to definition be tested with the
same test data that godef now uses
Change-Id: I04e488b6b9b2d891181f202ea1125b823a079c50
Reviewed-on: https://go-review.googlesource.com/c/150045
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
This allows it to be used in benchmarks as well as tests.
Change-Id: I1eb7307a0a7d27c541b14dd8b84c4bc2c770f3c9
Reviewed-on: https://go-review.googlesource.com/c/150257
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
This exposes the ability to add markers to the public interface, and
changes the way markers are collected to make it so a standard call to
Expect can replicate the internal behaviour.
This allows custom rules to also add marks.
Also add a special EOF identifier that acts like a mark at the end of
the file in which it occurs.
Change-Id: Ic5e41cbc5b7ae3c4d1c5b8baba980147c1d22ef1
Reviewed-on: https://go-review.googlesource.com/c/149610
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is a workaround for a go list regression that broke
go/packages but went unnoticed by because of a missing
call to packages.PrintErrors, added here.
Updates golang/go#28749
Change-Id: I1819a6143134a422791106ac037d3458ef864322
Reviewed-on: https://go-review.googlesource.com/c/149237
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Use the packagestest framework to test completion. Add support for a
slice of token.Position to packagestest to support this.
Change-Id: Ie5ddece4446a3c74419727461a77faa3788cb040
Reviewed-on: https://go-review.googlesource.com/c/148197
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
The fallback driver wasn't returning the roots in a
deterministic order because it was using sticking them in
a map and then iterating over that map to get each element.
Put them into a slice instead (and make a few small
associated changes) to preserve behavior.
Fixesgolang/go#28040Fixesgolang/go#28609
Change-Id: Ib8f8c88d65b7a48b2b04ca91e2d3c316d5bb5803
Reviewed-on: https://go-review.googlesource.com/c/148880
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This will allow Sizes to be marshalled and unmarshalled. All the Sizes
we care about ane StdSizes anyways.
Change-Id: I79d1dcaebba32f7730de4375945e372eeefa78fe
Reviewed-on: https://go-review.googlesource.com/c/147978
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
This is more reliable than looking in the environment.
Change-Id: I96c093b89faaece6b6256eefb4a4aac4d66b9cc9
Reviewed-on: https://go-review.googlesource.com/c/146757
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
This is based on 147340 and has the same test, but with a different fix.
We now produce a nice error if there is an empty root, instead of
crashing, and the cause of empty roots in go list has been fixed.
The underlying call to go list is returning the same package more than
once, and we only fix the first entry in the root list, so the second
one got left blank.
The fix was to not add the second duplicate copy to the output of the
driver at all.
Change-Id: I9f1b2f0fd63635ba101cdd3c8a5108530e968ba9
Reviewed-on: https://go-review.googlesource.com/c/147440
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
For performance, bail out of runNamedQueries before running go env if
there's nothing to do.
Add t.Helper() calls to packagestest.TestAll.
Escape # in packagestest.Export. The testing package adds #NN suffixes
to subtests that have redundant names.
Log how long gopathwalk.Walk took for name= queries when debug is on.
Change-Id: I37cb0ed11cf58e1693e29dea04697e5885ecc62b
Reviewed-on: https://go-review.googlesource.com/c/147203
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Add some basic tests for diagnostics using the new
go/packages/packagestest framework.
Change-Id: I6a7bfba6c392928a9eb123ab71ceb73785c12600
Reviewed-on: https://go-review.googlesource.com/c/145697
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
When the name= query constructs the temporary module, it may find things
that don't resolve. In at least some cases (#28518), allowing go list
to access the network results in not just bad performance but actual
failures. Default GOPROXY to "off" when doing queries on the temporary
module to try to address that.
Also, add some more debug logging, including various environment
variables, so that it's easier to reproduce failing commands.
Change-Id: I1a6d3ffa5c845271ce48e9fe802a2491ccadcd7c
Reviewed-on: https://go-review.googlesource.com/c/146477
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This adds a library that allows markers and actions inside comments in go source
files.
It then also adds an easy way to use that library for tests using packagestest.Expect
This is used to easily write code inspection and manipulation tests in a
language that is common to all tests.
Change-Id: I755caaad1557c9b8779ad9ecda2b3309550d6976
Reviewed-on: https://go-review.googlesource.com/c/142998
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
We do insist that drivers are stable (not sorted, just stable) and that
refine is also stable, which allows us to promise a stable output.
I also changed refine so it returns the root set in the same order that
the original root id list was supplied, as this seems to be a strictly
better experience.
Change-Id: I8eb0bffd7547865d14a6c6f18646018b9af140bd
Reviewed-on: https://go-review.googlesource.com/c/145877
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
If a Module's Name ends with a version suffix (v2), create its module
with an appropriate version number (v2.0.0).
Change-Id: I1a16f054fac5717e79871ba1bf9a6343c0ce2f5b
Reviewed-on: https://go-review.googlesource.com/c/146097
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Treat the top-level package of a module specially so that the VCS
working dir name doesn't have to match the package name. Generally we
take the position that directory names should match packages, but for
the top-level package that's a pretty strict requirement. Now we always
check the top-level directory, just in case.
Change-Id: I2e96751bf46736fbaef434fa9788fdc1339c37b9
Reviewed-on: https://go-review.googlesource.com/c/144817
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The builders are currently flaky failing because of a root order being
wrong.
This causes all root lists to be sorted before being returned, so the
order is always stable no matter which underlying driver is running.
Fixesgolang/go#27594
Change-Id: I09db45c67ad00f23dfaec8e271acbd13fc338888
Reviewed-on: https://go-review.googlesource.com/c/143737
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Instead of writing replace directives to link the test's modules
together, fill a GOPROXY dir and use `go mod download` to fill a real
GOCACHE with them. This makes the tests more realistic, since replace
directives will be somewhat unusual. It also gives the name= query
something to search.
Actually doing this in a way that's compatible with packagestest's
abstraction is a little tricky, since it wants to know where the files
will be. The actual files will be created by go mod download, so we have
to get Export to put them there to begin with, then move them out of the
way.
Since the GOPROXY zip format doesn't support symlinks, those will only
work in the primary module.
Change-Id: I6bc1d368f1c950d789e409213107d60bb1389802
Reviewed-on: https://go-review.googlesource.com/c/144498
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
I tried to introduce new modules where it made sense. I deleted the
weird goroot test because I removed the optimization it was testing in
an earlier CL.
Change-Id: I219ddaa4f462a4aeb640f62215d16f246511a5fe
Reviewed-on: https://go-review.googlesource.com/c/144497
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Darwin convertes the working directory to a real path which affects the
path a command is started with when you exec.
This causes a problem where the path the go list command is run with
does not match the path the outside program is trying to use, which
causes both undesirably changed paths in it's outputs, and sometimes
failures to cope with modules when source files are "outside" the module
root.
The go standard library has a special feature in os.Getwd where it
checks if the PWD environment variable is the same inode as the system
cwd, and if it is it returns the PWD instead. This change deliberately
sets the PWD before running go list so that behaviour is triggered,
which fixes all the paths.
This was breaking the mac build bots in a goimports test, it should be
fixed now.
Change-Id: I0f5d3c7d020b55749738036ba51c19884bb26598
Reviewed-on: https://go-review.googlesource.com/c/143517
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
packagestest.Modules is now available pre-go1.11 but doesn't add
itself to packagestest.All.
strings.ReplaceAll is not available in pre-go1.11
Change-Id: Ia8bf0e82bb853c6f29d31ca5c54651097342b19c
Reviewed-on: https://go-review.googlesource.com/c/143419
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This extracts some of the test code from packages, and adds the ability to run
the same test with multiple drivers.
This should generally be useful for all tools that run on top of go/packages as
well when writing tests for them.
Change-Id: I88c596ad07c0782270c5798d92ae29f7549943cf
Reviewed-on: https://go-review.googlesource.com/c/140118
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Some editors combine stdout and stderr when running tools, so
printing to stderr without failing confuses them. Stop printing go
list's warnings.
They're still useful for debugging, so users can set
GOPACKAGESPRINTGOLISTERRORS to see them. This isn't part of the API and
has no compatibility guarantees.
Change-Id: I9cf091cf59d082123149888468fa0d126dc972c7
Reviewed-on: https://go-review.googlesource.com/c/142997
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Can't do module tests on a pre-modules version of Go.
Change-Id: I8e3e22d472f37ebc9f930a68c9a6c0f4b7ba7ea0
Reviewed-on: https://go-review.googlesource.com/c/142700
Run-TryBot: Heschi Kreinick <heschi@google.com>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Add an implementation of name= for go list. It will be used to
implement goimports and godoc-like lookups by package name.
Imported a copy of the semver package from the stdlib to do version
comparison, and tweaked the gopathwalk API to include a hint about what
kind of source directory is being traversed.
Note that the tests, despite my best efforts, are not hermetic: go list
insists on doing version lookups in situations where it seems to me like
it shouldn't need to.
I think this implementation is ready for serious use. The one thing I'm
nervous about is that it currently does a substring match when looking
for a package name, so if you look up a package named "a" you will get
a huge number of results. This matches goimports' behavior but I don't
know if it's suitable for general use.
Change-Id: I2b7f823b74571fe30d3bd9c7dfafb4e6a40df5d3
Reviewed-on: https://go-review.googlesource.com/c/138878
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Heschi Kreinick
Rebecca Stambler
Ian Lance Taylor
Michael Matloob
Ian Cottrell
Alan Donovan
Daniel Martí
Fazlul Shahriar
Brad Fitzpatrick