buildtag checker:
- This checker has been modified from the version in vet to handle Go
and non-Go files differently, to avoid having to re-read-and-parse
Go files in the common case.
- The old cmd/vet driver would run this check on all the files in
a directory whereas new drivers will run it only on the files
selected for a particular configuration, so some of the checks
(those in checkArguments) will never fire. But this is not a regression
relative to 'go vet', because it too presents cmd/vet with only the
files selected as part of the package.
analysistest:
- fix bug that processed a block of //-comments as one.
- treat "...// want..." within a //-comment as a want comment.
This is required for adding expectations on lines that are already comments.
Change-Id: Iacf3684864e07532f77176481afbf059a9638f3b
Reviewed-on: https://go-review.googlesource.com/c/139797
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
The ctrlflow Analyzer builds a control-flow graph (see
golang.org/x/tools/go/cfg) for each named and unnamed function in the
package.
It computes for each function whether it can never return, either
because the function is an intrinsic that stops the thread (e.g.
os.Exit), or because control never reaches a return statement, or
because the function inevitably calls another function that never
returns. For each such function it exports a noReturn fact.
This change also:
- adds 'inspect', another Analyzer that builds an optimized AST
traversal table for use by nearly every other Analyzer.
- changes analysistest.Run to return the analysis result to enable
further testing.
(This required changing it to analyze one package at a time,
which is no less efficient, and is the typical case.)
Change-Id: I877e2b2363a365a9976aa9c2719ad3fba4df2634
Reviewed-on: https://go-review.googlesource.com/c/139478
Reviewed-by: Michael Matloob <matloob@golang.org>
This test is unsurprisingly failing on the nocgo builder because
the cgo packages don't exist on those builders.
Updates golang/go#28040
Change-Id: I633b73bb48e76824645e4e8dd141fb42c9adc19f
Reviewed-on: https://go-review.googlesource.com/c/140121
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
internal/checker
- don't display "[name.category]" in diagnostic
output. Most users don't care which analyzer reported the message.
The -json output retains this information.
- print better log messages for analyze -debug=f.
- print (non-JSON) text output to standard error, like a compiler would.
passes/pkgfact
- fix a nil deref panic when encountering non-renaming imports.
- require names to have underscores before and after (_x_)
as this avoids a huge number of spurious matches in (e.g.) the
syscall package.
- don't export empty facts.
Change-Id: I86c003b96521334e371f9d5fcea1323cd779d7f0
Reviewed-on: https://go-review.googlesource.com/c/139657
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
CL 139320 unset these env vars for the whole process in two tests,
but the correct fix is to unset them for subprocesses forked by go/packages.
Change-Id: I35e3ab9e424b00326e9e813e4daf0ae92ec36e26
Reviewed-on: https://go-review.googlesource.com/c/139477
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This fixes 'go test ./...' in the root of the tools repo with GO111MODULE=on.
Updates golang/go#27858
Change-Id: I7492d2a2406997a399fe2badd24882fcb19d37c5
Reviewed-on: https://go-review.googlesource.com/c/139320
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
This change adds support for testing the facts produced by an Analyzer
using a similar mechanism to the way it checks for diagnostics.
A "// want ..." comment may now contain a mixture of expectations for
diagnostics and facts. Diagnostics are indicated by a string literal,
as before. Facts are indicated by name:"regexp" where name identifies
the object (declared on the same line) with which the fact is
associated.
func neverReturns() { // want neverReturns:"noReturn"
for {}
}
Also:
- analysistest: report errors during package loading.
(We don't yet have a way to test RunDespiteErrors Analyzers in the
face of errors.)
- tests for Facts produced by findcall and pkgfacts.
(Findcall now produces facts just for testing.)
- Add String method to various Fact types.
Should the Fact interface have this method?
Change-Id: Ifa15fbd49d6ec3042b5fe9d3ebf22f4bdfdc8769
Reviewed-on: https://go-review.googlesource.com/139157
Reviewed-by: Michael Matloob <matloob@golang.org>
Some users may set GO111MODULE=on, and we will eventually want to be able to
build x/tools itself in module mode.
Updates golang/go#27858
Updates golang/go#27852
Change-Id: Iaf488b2a89e6526471530245cb580f1f0391a770
Reviewed-on: https://go-review.googlesource.com/137815
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Some applications (for example, diagnostics shown to a user in an
editor) may want to distinguish between errors generated by
the driver, parser, and type-checker. The Error struct did not have any
mechanism for doing this, so add an ErrorKind field and set it in
appendError.
Change-Id: If347163225d1e3a567e98610e9ba8a0930e4659c
Reviewed-on: https://go-review.googlesource.com/139317
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
TestLoadImportsC just works on Go 1.10. And there's no good reason
that TestLoadAllSyntaxImportErrors shouldn't work, even though
it's currently always skipped.
Change-Id: Icd8d311f12c5731cc635937a00251eab0a3077ec
Reviewed-on: https://go-review.googlesource.com/139117
Reviewed-by: Alan Donovan <adonovan@google.com>
A previous change (golang.org/cl/137096) that made absolute package paths relative to
GOROOT or GOPATH entries also accidentally applied to .go filesnames.
Filter those out of the list of paths considered to make relative to
search path.
(package paths that don't start with './' or '/' are relative to GOROOT or GOPATH,
but filenames are not.)
Change-Id: I67fbd0e5caa7e53f3ab5b77f55d6841fe2132578
Reviewed-on: https://go-review.googlesource.com/c/138880
Reviewed-by: Alan Donovan <adonovan@google.com>
This applies a part of golang.org/cl/137075 to the fallback. If go list
doesn't find a package, it returns an almost empty package structure
with an error set on it. That change passed through those packages
from the 1.11+ go list, so users could determine there wasn't a match.
This change does the same from the go 1.10 fallback go list code.
Change-Id: I98acc186c0a9eeef0416e9fec0e1fe0e29ddc51c
Reviewed-on: https://go-review.googlesource.com/c/139158
Reviewed-by: Alan Donovan <adonovan@google.com>
The behavior of go/types.ObjectString (or perhaps the underlying
object) changed at some point.
Change-Id: I77f1d13c180e1f78ddc08e80e5d38aa01f425111
Reviewed-on: https://go-review.googlesource.com/138777
Reviewed-by: Robert Griesemer <gri@golang.org>
This new package provides helper functions for traversal over the
syntax trees of a package, including node filtering by type, and
materialization of the traversal stack.
During construction, the inspector does a complete traversal and
builds a list of push/pop events and their node type. Subsequent
method calls that request a traversal scan this list, rather than walk
the AST, and perform type filtering using efficient bit sets.
Experiments suggest the inspector's traversals are about 2.5x faster
than ast.Inspect, but it may take around 5 traversals for this benefit
to amortize the inspector's construction cost.
This design is well-suited to the ongoing reworking of cmd/vet (see
docs.google.com/document/d/1-azPLXaLgTCKeKDNg0HVMq2ovMlD-e7n1ZHzZVzOlJk),
which historically made a single pass over the ASTs but is being
replaced by a design that requires a separate pass for each analysis.
Change-Id: I9a67aed6a3bf948076641d96447860d97ede67b4
Reviewed-on: https://go-review.googlesource.com/135655
Reviewed-by: Robert Griesemer <gri@golang.org>
Another change to bring the go1.10 fallback's functionality in line with
the go1.11 go list implementation. The fallback now uses go env to
determine GOPATH and GOROOT and searches them to see if any match an
absolute path, and if so trims the GOPATH/GOROOT entry off the start
of the path.
Fixesgolang/go#27734
Change-Id: Ibd2313fc4301d42fd8c0cd98f1f3e7a313d65eb7
Reviewed-on: https://go-review.googlesource.com/137096
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
The format of Analyzer.Doc is now specified as a short title followed
by a longer description. This allows us to build a nice
self-documenting command-line interface. Much of the documentation in
vet's doc.go and file-level comments can now be displayed to the user.
Change-Id: I462343e97ac9b743284aaa3e06e7a81d11e9593f
Reviewed-on: https://go-review.googlesource.com/138396
Reviewed-by: Michael Matloob <matloob@golang.org>
It's easy to forget to pass the last argument,
in which case the test would silently pass.
Change-Id: I95249e1fe8bee75cfaa535fcf723d04f102214fc
Reviewed-on: https://go-review.googlesource.com/138395
Reviewed-by: Michael Matloob <matloob@golang.org>
Type-checker objects are canonical, so they are usually identified by
their address in memory (a pointer), but a pointer has meaning only
within one address space. By contrast, objectpath names allow the
identity of a logical object to be sent from one program to another,
establishing a correspondence between types.Object variables that are
distinct but logically equivalent.
This package was developed for Google's internal fork of guru.
It is needed for lemma support in the analysis API; see
docs.google.com/document/d/1-azPLXaLgTCKeKDNg0HVMq2ovMlD-e7n1ZHzZVzOlJk
Change-Id: I9899ce14d57909858a68f84e90d58a039f2bb7a0
Reviewed-on: https://go-review.googlesource.com/135675
Reviewed-by: Robert Griesemer <gri@golang.org>
The documentation for Load says:
“Load returns an error if any of the patterns was invalid as defined
by the underlying build system. It may return an empty list of
packages without an error, for instance for an empty expansion of a
valid wildcard. Errors associated with a particular package are
recorded in the corresponding Package's Errors list, and do not cause
Load to return an error.”
Therefore, it should not be an error for a pattern to match no
packages. If the pattern is a literal package path that does not
exist, we should prefer to return a *Package for it with an error in
the Errors field.
Change-Id: Iaecfb920097e3b520e763bd52c0e326d2e7a4861
Reviewed-on: https://go-review.googlesource.com/137075
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
This change creates a fork of vet from the standard distribution.
It was created by this script:
$ mkdir go/analysis/passes/vet/
$ cd go/analysis/passes/vet/
$ (cd $GOROOT/src/cmd/vet >/dev/null && git co 31d19c0 && tar cf - .) | tar xf -
$ rm -fr all # We'll deal with cmd/vet/all later.
$ rm -fr internal/cfg # Published as golang.org/x/tools/go/cfg.
$ sed -i -e '1s?^?// +build ignore\n\n?' *.go
All the Go files have been tagged "ignore" for now.
A series of follow-up changes will convert each vet check
into an instance of the new go/analysis API's Analyzer.
At some point soon, cmd/vet in the standard distribution will use a
vendored copy of this code. Until then we will periodically integrate
any changes made to cmd/vet to this fork. The current version of
cmd/vet will be recorded in the REVISION file.
Change-Id: I0c63eeb17cc612b3f013679595dcbc71a90950f7
Reviewed-on: https://go-review.googlesource.com/138137
Reviewed-by: Michael Matloob <matloob@golang.org>
This field lists the names of non-Go files that are part of the
package under analysis.
Change-Id: Ic967dc18b98e018c691442f7378cb29db30a1454
Reviewed-on: https://go-review.googlesource.com/138136
Reviewed-by: Michael Matloob <matloob@golang.org>
Forcing clients to register their Fact types with gob made for an
unfriendly API. Now the driver is again responsible for doing it.
The FactTypes field is now a slice of Fact values (not reflect.Types)
used only for their dynamic type, which is slightly more convenient.
Change-Id: I01219edb24bd2371ba642bb56508aa80c19a9b61
Reviewed-on: https://go-review.googlesource.com/137836
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
The supported x/tools versions are now all at least Go1.9.
Change-Id: I9476329f2be8f3c560efb280f06d65669a3e9f85
Reviewed-on: https://go-review.googlesource.com/137996
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
CL 137735 only fixed Darwin, and was submitted prematurely.
Change-Id: Idf9706ab2dc6ef716471cd6a2089bb0be63a54a2
Reviewed-on: https://go-review.googlesource.com/137835
Reviewed-by: Michael Matloob <matloob@golang.org>
The test's pathname sanitization heuristically assumed that $TMPDIR
contained /tmp, which is not the case on Darwin or Windows. Now we
pass it the precise directory prefix to strip off.
Fixesgolang/go#27877
Change-Id: I85167d721ebb9c4f6d74016a00025fd726939e47
Reviewed-on: https://go-review.googlesource.com/137735
Run-TryBot: Andrew Bonventre <andybons@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
...by removing references to trace API for now.
Change-Id: Ide6bbbfd98e15a3773b4a10232bcbf2dc2153341
Reviewed-on: https://go-review.googlesource.com/137615
Reviewed-by: Alan Donovan <adonovan@google.com>
Package checker is a driver for the analysis API.
It is an internal package, but is exposed by three different APIs:
analysistest, singlechecker, and multichecker.
Checker uses go/packages to load the specified packages (plus their
dependencies, if any analysis uses facts) from source code.
It constructs a graph of actions (analysis passes), whose dependency
edges may be "horizontal", when one analyzer depends on the output of
another applied to the same package, or "vertical", when an an
analyzer consumes facts produced by the same analyzer applied to a
dependency package.
The graph is executed in parallel, unless -debug=p.
Facts are passed from one pass to another in memory.
If -debug=s, facts are serialized, to exercise that logic.
Findings are printed at the end.
The -json flag selects JSON output.
Use -debug=t to print timing information.
Always use -debug=tp, for sequential mode, when timing.
Also:
- analysistest: a wrapper for testing checkers
- multichecker: a wrapper for writing multi-checker tools
Analysis flags are prefixed by name: -findcall.name=foo.
- cmd/analyze: a command-line tool based on multichecker
- singlechecker: a wrapper for writing single-checker tools
Analysis flags are unprefixed: -name=foo.
- passes/findcall/cmd/findcall: a standalone tool for the findcall analysis
- tests for findcall
- tests for pkgfact
Change-Id: Icfd4a49cee17e7de1ddb6ec15a62dc667fb2db04
Reviewed-on: https://go-review.googlesource.com/135679
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This package defines a control-flow graph of Go statements and
expressions. It was originally built as part of cmd/vet (where
it was in turn derived from x/tools/go/ssa) and is being
published to make it available to other analyses.
Change-Id: Ib6077c5a856bb74d6a8411e9c3f9e2f79beb5658
Reviewed-on: https://go-review.googlesource.com/135636
Reviewed-by: Ian Cottrell <iancottrell@google.com>
For all packages p, q where there's an dependency path of
p.test -> ... -> q -> ... -> p, Go creates test variants
q [p.test] of each q and replaces the dependency on q with
a dependency on q [p.test], and replaces p with the expanded
test variant p[p.test]. Fix the fallback logic to add
these missing test variants. (Before this change, it was
only producing the variant of p: p [p.test].)
Fixesgolang/go#27670
Change-Id: Ic56ba35fadcdf8c5928ec76f5a7b0ebe650c9f02
Reviewed-on: https://go-review.googlesource.com/136176
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Analysis -> Analyzer
Unit -> Pass
Output -> Result
Inputs -> ResultOf
Lemma -> Fact
Set{Object,Package}Lemma -> Export{Object,Package}Fact
{Object,Package}Lemma -> Import{Object,Package}Fact
LemmaTypes -> UsesFacts bool
plugins/ -> passes/
Notes:
- Unit.Output is no longer a field; it's the result of calling Analyzer.Run.
- Because analyzers no longer declare their LemmaTypes, they, not the
driver, are now responsible for registering Fact types with Gob.
A follow-up change will additionally rename:
Finding -> Report
Pass.Syntax -> Pass.Files
Pass.Info -> Pass.TypesInfo
Change-Id: Iccbdadbea5a0aafe732e23a344dd57fd93681931
Reviewed-on: https://go-review.googlesource.com/137095
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This test introduces noise when using 'go test all' or 'go test ./...'
to test go/packages and the tools that depend on it.
Since it has been broken for around a month, skip it indefinitely.
Updates golang/go#27292
Change-Id: I796292310332712e14bc8a0b73e36a8ed6f8a73f
Reviewed-on: https://go-review.googlesource.com/137315
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Part of the testmain generation logic (sans coverage) has been taken
from go build and put into golist_fallback_testmain.go.
golist_fallback invokes this logic and builds up the package metadata
for the testmain.
The tests checking for testmain are now no longer skipped from
packages_test.go.
Change-Id: I487a947f087f3ad4161ea6c2bed06ebb2f833422
Reviewed-on: https://go-review.googlesource.com/134119
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
This CL contains just the API, the validate function,
and two example analyses, findcall and pkglemma.
Change-Id: Ia1f2652647050b1e0e15dad8b9ae10cf1a5fbdbc
Synopsis: go-review.googlesource.com/c/tools/+/134935
Design: docs.google.com/document/d/1-azPLXaLgTCKeKDNg0HVMq2ovMlD-e7n1ZHzZVzOlJk
Reviewed-on: https://go-review.googlesource.com/135635
Reviewed-by: Ian Cottrell <iancottrell@google.com>
StaticCallee reports the destination of a CallExpr, if it is a static
call to a Go function or method.
StaticCallee is not a complicated function, but I wrote it four times
during the course of prototyping the analysis API in
go-review.googlesource.com/c/tools/+/134935.
Change-Id: Icd26fc1e5f6ed9edebd4d0a00fdf18aa0acb074c
Reviewed-on: https://go-review.googlesource.com/135676
Reviewed-by: Robert Griesemer <gri@golang.org>
For some reason, the type of the last Param in a wrapper function
for a variadic method (T) f(...U) had type [][]U instead of []U.
(Possibly a workaround for a long-since fixed bug in go/types?)
Added a sanity check to ensure that the common suffix of
fn.Params and fn.Signature.Params match in type.
Fixesgolang/go#27453
Change-Id: I9506f4f67a7ff3a283e9ec0142f638aad00287a9
Reviewed-on: https://go-review.googlesource.com/134515
Reviewed-by: Robert Griesemer <gri@golang.org>
Errors relating to a particular package (as opposed to the Load
operation as a whole) are now only recorded in the Errors
field of the corresponding Package.
Clients are responsible for printing or otherwise handling errors.
This is a breaking API change.
The PrintErrors function prints all accumulated errors,
dependencies first, and is provided for convenience.
PrintErrors is based on Visit, another helper function, which
visits each Package in an import graph.
Also:
- add an Example documenting typical use.
- update clients to handle errors explicitly.
Depends on https://go-review.googlesource.com/c/tools/+/130576
Change-Id: I39407ab7f46dae2f0dd0fdde21080e172e1258aa
Reviewed-on: https://go-review.googlesource.com/131015
Reviewed-by: Michael Matloob <matloob@golang.org>
This change fixes a bug that occured in the golist fallback logic when
an x_test package imported its own package under test. For a package
"a", if "a_test" imported "a", we'd populate "a_test"'s import map
with an entry "a [a.test]" pointing to the test variant of the package
with id "a [a.test]". This change fixes the key to be "a", the correct
import path of the package, not "a [a.test], which is the ID".
Change-Id: If798f2675b01aa537c6ccc129dc35d042d967337
Reviewed-on: https://go-review.googlesource.com/133356
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Rather than document the range of possible error types, requiring
clumsy client code to extract position information, we now expose a
single concrete type for all errors. Position information (in
standardized string form) is technically optional, but we should
strive for 100%, fixing gaps as they arise.
This change enables us to unify the Package and JSON structs in a
follow-up.
Question: should we eliminate the Config.Error hook and be silent by default?
Pro:
+ most clients suppress it.
Con:
- clients that want to print errors (e.g. vet-like tools) would have
to traverse the entire import graph to find them.
- silence is not the most fail-safe behavior.
Change-Id: Ie92b9fb7641ceda429f00928474b650d1dfadedd
Reviewed-on: https://go-review.googlesource.com/130576
Reviewed-by: Michael Matloob <matloob@golang.org>
This makes Load use the value of GOPACKAGESDRIVER in the current
process's environment if the loader's environment wasn't explicitly
set.
Note that if GOPACKAGESDRIVER is not set, we'll still search PATH in
the current process, regardless of whether PATH is set in the loader's
environment. This has not changed.
Change-Id: I3081a6cb0876a21ccd4d87b4b8d082aa47911b75
Reviewed-on: https://go-review.googlesource.com/131016
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
The interpreter has no intrinsic for reflect.Value.MapRange,
so it no longer supports fmt.Sprint on a map, which uses it.
Suppressing part of complit.go test for now.
Change-Id: Id6ade19bdbb92593d6da57c82e75f311fb65b4fe
Reviewed-on: https://go-review.googlesource.com/131075
Reviewed-by: Robert Griesemer <gri@golang.org>
Visit packages in deterministic order for predictable JSON.
(I don't know why the test only appears to fail on darwin;
I would expect it to randomly fail in linux 50% of the time.)
Change-Id: I5270804077fc9ca8f529a1f4657e1c35f0586579
Reviewed-on: https://go-review.googlesource.com/130755
Reviewed-by: Michael Matloob <matloob@golang.org>
This change adds the -buildflag argument to go/packages/gopackages,
which may be passed repeatedly to set config.BuildFlags. I felt -flag
was too vague.
config.Flags is renamed to config.BuildFlags for consistency, and
packages.findExternalDriver now passes -buildflag instead of -flag to
drivers. This will break existing drivers.
Change-Id: Iaed58026373a46e137a236ee9a652eb3a9433ee3
Reviewed-on: https://go-review.googlesource.com/130136
Reviewed-by: Alan Donovan <adonovan@google.com>
The legacy implementation currently always loads from source code, so
packages are more complete than expected.
Change-Id: Ib8c9f7ac590038108dba05c1f47d22e70734945c
Reviewed-on: https://go-review.googlesource.com/130095
Reviewed-by: Michael Matloob <matloob@golang.org>
In ModeTypes, we only need the roots, in ModeSyntax we need the direct
dependancies.
We now do exactly what we need rather than the entire set of transitive
dependancies.
Without this change tools operating on source of a deep dependancy tree are far
to slow.
Change-Id: I3ddbe660709184758ad33ab7cdea860244433ce6
Reviewed-on: https://go-review.googlesource.com/129355
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
This saves 35% of wall time in a command such as
$ gopackages -mode=syntax golang.org/x/tools/go/packages/gopackages fmt
that would otherwise typecheck every function in each package that
depends on fmt. It also applies in -mode=types when source was loaded
as a fall back for missing export data due to an error in a lower package.
Also, remove Config.TypeChecker field now that not a single of its
subfields is passed through go/packages to the type checker.
(The Sizes function is logically a result, not an input, of loading,
though we have yet to implement it properly.)
Change-Id: I472d21b34fc5e2832f7353e82992a67a06e4e4cc
Reviewed-on: https://go-review.googlesource.com/129497
Reviewed-by: Ian Cottrell <iancottrell@google.com>
We were calling types.Type.Underlying() too early, which caused CanPoint
to see an ordinary struct, not reflect.Value.
Change-Id: I23ba3b9451bc1abc1ad8a45c790d2e22c1481f26
Reviewed-on: https://go-review.googlesource.com/129196
Run-TryBot: Dominik Honnef <dominik@honnef.co>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
This is fixes for review comments made on cl/128120
Change-Id: Ie6382d6ec91db2bbdece564a83ae028be4e2e269
Reviewed-on: https://go-review.googlesource.com/129000
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
For any identity imports to be added to the import map of a package,
make sure these are not already in there. If the package imports "C",
be sure not to overwrite the import map entries for "runtime/cgo", "unsafe",
and "syscall".
Test the import map when test variants are loaded. This tests the fix
in 'go list' made by https://golang.org/cl/128836Fixesgolang/go#26847
Change-Id: I05dff4d3a75fab03f333f6d88128de6a6bf169e6
Reviewed-on: https://go-review.googlesource.com/128876
Run-TryBot: Suzy Mueller <suzmue@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
Copy imports.VendorlessPath into go/packages to avoid an unnecessary
dependency on imports.
Change-Id: Ie77c2fb87f2199f139ece9f3d1b707f065fc1a79
Reviewed-on: https://go-review.googlesource.com/128996
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This deletes the raw and golist packatges, instead using the main
Package structure as the serialzied form between the build tool and the
loader.
This requires a few new fields on Package for information that we used
to hide, and some extra json tags on Package.
It also required Package to have custom JSON marshalling methods.
Change-Id: Ib72171bc93e2b494b97f4b0266504acaa9b3f647
Reviewed-on: https://go-review.googlesource.com/128120
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change addresses some style issues pointed out by rsc in
golang.org/cl/126177. It doesn't fix the comment on listfunc
or the comment on the processing of the containing directories.
Those will be handled in a separate cl.
Change-Id: I2094ad1e5e2ee181a646466675f4d993bd69ff9b
Reviewed-on: https://go-review.googlesource.com/128897
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
This change addresses adonovan's comments in golang.org/cl/125939.
Make sure to include CXXFiles, MFiles, HFiles, FFiles, SwigFiles,
SwigCXXFiles, and SysoFiles from the go list output into the Package
struct's OtherFiles field.
When computing packages from the output of fallback golist,
make sure to create the test variant of a package whenever creating
an x_test for a package because the x_test always depends on
the test variant even when there are no non-x_test files.
Sort dependency packages from the fallback golist to ensure
output is deterministic.
Change-Id: I3a942898c7edbe0ad62dbdf3d521775ffd9b9594
Reviewed-on: https://go-review.googlesource.com/128838
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
Also:
- make a nil *Config equivalent to new(Config).
- add a test for nil *Config.
- document that Load may return an empty list of packages.
Change-Id: I642133abe18553ca8c7f46b7bd2709a03eda0b28
Reviewed-on: https://go-review.googlesource.com/128875
Reviewed-by: Michael Matloob <matloob@golang.org>
Use export data in LoadTypes mode, when it is available to get the type information.
The lock can be moved back to loadFromExportData, as each package is
loaded in topological order, and all calls to loadFromExportData for a
particular package occur when that package is being loaded.
Fixesgolang/go#26834
Change-Id: Ib6c28eb8642a473cc100d54d0aac7b90644d5d22
Reviewed-on: https://go-review.googlesource.com/128365
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
Adds ssautil.Packages function, which creates ssa.Packages
from packages.Packages.
Deprecates testmain synthesis logic now that go/packages
reports the test packages synthesized by the build system
as first-class packages.
Updates docs, examples, and tests.
Flags potential confusion around legacy concept of "importable" packages.
Change-Id: I6d9cd7c6436c715d1ef39e3e280f4af4d48ccc5a
Reviewed-on: https://go-review.googlesource.com/128675
Reviewed-by: Dominik Honnef <dominik@honnef.co>
Also collect errors so we can set them on the Package (once
golang.org/cl/128120 is in).
Change-Id: I2950405404f060312813e4aa27393496078a3b7e
Reviewed-on: https://go-review.googlesource.com/128357
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
...as it is required not just for Syntax processing but also
when loading export data.
Clarify comments surrounding Types, Fset.
Remove bogus comment re: Dir.
Change-Id: I8d234bc311d73cbed4fd271ef883ebba22ba2708
Reviewed-on: https://go-review.googlesource.com/128595
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Check for the go-build subdirectory to identify a filepath in the go
build cache. The go command uses a subdirectory of the default os cache
called "go-build" for the build cache.
Additionally, this responds to the comments about function names and tests in the following
CLs:
https://golang.org/cl/125536https://golang.org/cl/125302
Fixes https://github.com/golang/go/issues/26387
Change-Id: I4ec65822e1e178d3de215a773e42f09fcc5bdb0d
Reviewed-on: https://go-review.googlesource.com/128360
Run-TryBot: Suzy Mueller <suzmue@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
The fallback was being reset to the incorrect value
after it was set. This change fixes it.
Also add a test that triggers the fallback (a query with both
a contains: line and a regular package path).
Change-Id: I49a9aeb3a0c7d7cc308ac56f4985545315a5bfd2
Reviewed-on: https://go-review.googlesource.com/128356
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
This allows an external binary (not go list) to be the source of package
information.
It uses a binary called gopackagesraw if present in the PATH.
The binary can be overriden by specifying the GOPACKAGESRAW
environment variable.
The command must accept the -test -deps -export and -flags, and take a
list of package patterns to match. It then returns a raw.Results followed by
the matching raw.Package structs in json format on stdout.
Change-Id: I836014d837a284999ded0c1157b8e6dac058ba69
Reviewed-on: https://go-review.googlesource.com/125938
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
This change moves the cgo support for go loader to a common
internal package that can be reused by go/packages. That
support is used to invoke cgo to process the files returned
by the fallback pre-go1.11 Go list.
The cgo processing does not propagate all of the config correctly.
We're building a go/build.Package to get the cgo command line
invocation and it might be incorrectly configured if the user's
configuration is different from go/build.Default.
Change-Id: I0de3c65cb723587e1551edeb77219614ba9650bf
Reviewed-on: https://go-review.googlesource.com/127816
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Diamonds in the dependency graph could cause the same package
to be processed by two separate gorotines, causing a potential
race between checking if the package's Types field is set, and
setting the field.
adonovan's comment suggested this might be unnecessary. We'll
ask him about this once he returns from his vacation.
Change-Id: I27511b6e44bb1b6a04ea294206fd8ed4ea60f56f
Reviewed-on: https://go-review.googlesource.com/127636
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Suzy Mueller <suzmue@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is mostly straightforward, except that go list -e -compiled
will now return errors if a package can't build. This is a bug.
We need to skip the errors test until that's fixed.
For now, don't try to run go list with no arguments because it will
fail. So when all arguments are contains, we will check for empty
patterns and skip running go list.
Change-Id: I7c92b1bf9448e5dbea22f9ade9fb9429945e3719
Reviewed-on: https://go-review.googlesource.com/127339
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
In Go 1.10 and earlier, the Deps field returned by go list does not
include the dependencies of any test files. To get the dependencies
of the test packages, we need to run go list another time (for a total
of 3 calls to go list), to get the dependencies of the packages the
tests import.
Fixesgolang/go#26753
Change-Id: Id3b424ac935e5ed459b4d4a55630715171bb9710
Reviewed-on: https://go-review.googlesource.com/127457
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It got out of sync after the round of API design changes, this brings it
back in line.
Also fix the behavior of passing nil as the config to match the
expected/documented behavior.
Change-Id: I8b51f5bd29ef9338ddb920fb872d857f180f9c84
Reviewed-on: https://go-review.googlesource.com/127255
Reviewed-by: Michael Matloob <matloob@golang.org>
It's been removed from go list.
Change-Id: I45b496bb6b693462484b1089cdb89c4e19e35b2e
Reviewed-on: https://go-review.googlesource.com/127341
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
We want to make it public to enable external sources of information
(other than go list), but we don't want to clutter the main
go/packages API with things that are intended for implementers only.
We also want to move the two API's from experimental to stable at
different times.
We also need to split the API because the golist implementation has to depend on
the raw structs, and then the main packages.Load has to depend on the golist
implementation.
The golist functionality is now in it's own package, and provides a Load method
to return the raw structures. This can be re-used by more complex raw providers.
Change-Id: I767af216b0d1ad9391e613cbf24eb2db5f503369
Reviewed-on: https://go-review.googlesource.com/126718
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
All the other members of raw package are stable for any given package,
DepOnly relates to the query patterns, not thepackages.
Instead the raw functions now return the set of roots matched
Other minor changes included:
rawConfig.ExtraFlags -> rawConfig.Flags
delete rawConfig.Context
Change-Id: I3a729fa4557043f65da386eed94ab46fdfacb8ad
Reviewed-on: https://go-review.googlesource.com/127035
Reviewed-by: Michael Matloob <matloob@golang.org>
It is a bounds check on an array before an access, the test cannot
continue safely if it fails, so it should be fatal.
Change-Id: Icd081df9cb81d549519d69bd34739d950946ccf5
Reviewed-on: https://go-review.googlesource.com/126975
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
We ask the underlying tool to produce export data if needed, but if it
cant we should fall back to source rather than just failing.
Change-Id: I376d41b4c8ad4d3c1aef3ca4ab4e3d0466b83b56
Reviewed-on: https://go-review.googlesource.com/126875
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
go list has been changed so that when -tests is added it reports the
test packages as well as the test mains in the initial set.
This fixes all the tests that assumed the old behaviour.
I changed the test that checked the initial set to check the entire
graph because the comparison became unreadable with the expanded set,
and this seemd like a nicer standardised way to check the behaviour.
Change-Id: I69404f8ef77cd092b0f02011a7789db71bee9b47
Reviewed-on: https://go-review.googlesource.com/126796
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Add the Flags field to the Config struct in packages to provide a way
for users to pass along additional information to the underlying query
tool.
Since users that need Flags will already know something about the build
system they are using (and flags will vary depending on the underlying
build system), they can pass through the flags that they need for that
build system.
For example, build tags should be passed through using the Flags field in go build,
using "-tags=".
Change-Id: Ia65bf0d003db2f6d9aaad6cd09c602f4bc5bf3e3
Reviewed-on: https://go-review.googlesource.com/125302
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
If a 'word' provided to go/packages' Load function starts with contains,
go/packages will interpret that word as the package containing the given
file.
For example:
packages.Load(config, "contains:/usr/local/go/src/fmt/format.go")
would load the fmt package from the Go installation at /usr/local/go.
This implementation uses "go list ." in the directory the file is
contained in to find the package, but this won't work in the module
cache. We plan to add support to go list directly to help find the
containing package. Then, because we won't need to change directory,
go list will have knowledge of the correct vgo root module, and will
be able to surface correct results.
Change-Id: I6bff62447c12f13dae5e4c0c65f729d9f271c388
Reviewed-on: https://go-review.googlesource.com/126177
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Test that the import graph for packages in the GOPATH that use vendoring
are correct and are keyed by the import path as it appears in the source
file.
Change-Id: Ibf9a516b4bfcc9f2d45f349d2ae49a0dbf958e30
Reviewed-on: https://go-review.googlesource.com/125298
Run-TryBot: Suzy Mueller <suzmue@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Paths to files in a Package should exist and be absolute.
Check that both GoFiles and OtherFiles are absolute paths.
Change-Id: I381830425ef313a91fa49be4d1751631d0a3a0c8
Reviewed-on: https://go-review.googlesource.com/125536
Run-TryBot: Suzy Mueller <suzmue@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
The go/loader based implementation was too different. Instead
Just call go list twice to get all the dependencies. This immediately
enables more functionality.
Change-Id: I22883492e996133bc61664314318c5e0a9cc10dc
Reviewed-on: https://go-review.googlesource.com/125939
Reviewed-by: Suzy Mueller <suzmue@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
It got broken when we renamed all the modes, and because it is just an
example binary it is not testsed.
Change-Id: Ia6608af574d0abebb4fe0d967e1bed88af6563bd
Reviewed-on: https://go-review.googlesource.com/125937
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
This separates the go list specific behavior from the generalised go/packages
loading behaviour, to enable alternate build system back ends.
Change-Id: I07e7649f8f2afc7470a3ee3d0d743cbbcc6f713e
Reviewed-on: https://go-review.googlesource.com/125715
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
After a discussion in cl/125535 this is a proposal to clean up some
internal function signatures that were becomming unwieldy.
Change-Id: I37a9e525ab18dface9ea9fb0b0ec68ac942ee632
Reviewed-on: https://go-review.googlesource.com/125659
Reviewed-by: Michael Matloob <matloob@golang.org>
If go list fails because it doesn't support the new flags added
for Go 1.11 (which will be also released in Go 1.10.4) try again
using the Loader to approximate the packages requested.
This implementation is incomplete. It will never support test packages
because of the two phase test loading process of the Loader. It
also doesn't reliably have access to export data so it will
always do an upgraded whole-program query. But we'll try to get
the best level of support we can for the go/packages interface
given the limitations of the loader.
Once Go 1.12 is released, we'll delete this support. By then, most
Go users should have at least switched to Go 1.10.4.
Change-Id: I5248e20980032695a86b052caa9ff368ecf7b142
Reviewed-on: https://go-review.googlesource.com/125616
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Collapse the main entry points to the single Load function.
Make public the Mode enumeration in the Config.
Add a lowest level mode that does not do the import graph.
Remove PkgPath.
Remove the DisableCgo option.
Rename some fields of Package:
Srcs -> GoFiles
OtherSrcs -> OtherFiles
Type -> Types
Files -> Syntax
Info -> TypesInfo
delete the All function
Change-Id: I54e5dc3ca5cb0b7e33fe6aa6f70cc66e18ea8cac
Reviewed-on: https://go-review.googlesource.com/125535
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
This makes Package a struct we can re-use in other places/algorithms.
Change-Id: I2e095a624bf44223319ba1ea3d60deb727b38f81
Reviewed-on: https://go-review.googlesource.com/124855
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
I made a pass through the documentation trying to simplify
and make it more accessible to readers who are unfamiliar
with all the ins and outs of the implementation.
I also added TODOs about things I don't understand
and names that we should think about changing.
Change-Id: I633316bd407f3360eb8a683bc8d85fba93ca381e
Reviewed-on: https://go-review.googlesource.com/125305
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
We have not yet settled on the meaning of Package.IsTest, and more
deeply which packages are test and which are non test packages.
This change will remove Package.IsTest in the meantime to avoid
confusion. We may later add a boolean or some other way to distinguish
test and non-test packages.
Change-Id: If6b128f7914009fdd42b8bc3de3bff73c8f006cd
Reviewed-on: https://go-review.googlesource.com/124388
Reviewed-by: Russ Cox <rsc@golang.org>
This change adds optional Dir string and Env []string options
that are passed through to the build system's metadata query.
As with exec.Cmd, the defaults are inherited from the parent
process.
Options.GOPATH is gone. If the client needs to override
GOPATH, they must use Env, but typically the inherited environment
is correct.
The tests not longer use os.Chdir.
We now guarantee that Package.Srcs are absolute file names.
Added test for Options.Dir and relative patterns.
This is a copy of golang.org/cl/123777, which had a merge conflict.
Change-Id: I301724581f3d8c712ea78fa5ab5cabd909940dca
Reviewed-on: https://go-review.googlesource.com/123777
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Alan Donovan
Michael Matloob
Bryan C. Mills
Rebecca Stambler
Robert Griesemer
Koichi Shiraishi
Todd Neal
Brad Fitzpatrick
Jay Conrod
Ian Cottrell
Dominik Honnef
Suzy Mueller
Kazuhiro Sera
Russ Cox