Also collect errors so we can set them on the Package (once
golang.org/cl/128120 is in).
Change-Id: I2950405404f060312813e4aa27393496078a3b7e
Reviewed-on: https://go-review.googlesource.com/128357
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
This change moves the cgo support for go loader to a common
internal package that can be reused by go/packages. That
support is used to invoke cgo to process the files returned
by the fallback pre-go1.11 Go list.
The cgo processing does not propagate all of the config correctly.
We're building a go/build.Package to get the cgo command line
invocation and it might be incorrectly configured if the user's
configuration is different from go/build.Default.
Change-Id: I0de3c65cb723587e1551edeb77219614ba9650bf
Reviewed-on: https://go-review.googlesource.com/127816
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
(*build.Context).Import has signature of
(ctx, importPath, fromDir, mode)
not
(ctx, fromDir, importPath, mode)
and the loader actually uses the first version.
The thinko was introduced in d6e83e53 (go/loader: changes for vendor
support; CL 18053) and was possible because importPath and fromDir both
have the same type string.
Fix it.
Change-Id: I52deaec6d141846b8a495835b121c1bdc2864215
Reviewed-on: https://go-review.googlesource.com/47343
Reviewed-by: Alan Donovan <adonovan@google.com>
The loader was calling (*types.Checker).Files on the "unsafe" package,
a global variable. Even with zero files, this operation is not a no-op
because it sets the package's "complete" flag, leading to a data race.
(Because Unsafe.complete is already set at construction, the
race is benign, but is reported by -race nonetheless.)
Fixesgolang/go#20718
Change-Id: I5a4f95be5ab4c60ea3b6c2a7fb6f1b67acbf42bc
Reviewed-on: https://go-review.googlesource.com/46071
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
https://golang.org/cl/37694 is a backwards-incompatible change to the
go/types API that causes "unsafe" to be included among the results of
(*types.Package).Imports(). Client packages such as go/loader and
go/ssa now need to recognize the special unsafe package and its
unusual *Bultin members and not create a types.Package for it.
Change-Id: I902725d3053cb887246978efdb5ec5dcdc0e4818
Reviewed-on: https://go-review.googlesource.com/37710
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Details:
- remove go1.5 "default version" labels on most files
- remove go1.6 labels on a few files
- go/loader: use conf.Cwd not "." in a couple places; update faulty
test expectations. (Not sure why this wasn't caught sooner.)
- go/ssa/interp: add 'mono' result to time.now intrinsic
- go/gcimporter15/bimport.go: make consistent with the version in gc
- go/ssa/interp: update test error message
- go/ssa: update a comment
The go/gcimporter15/bexport.go logic is stale and needs to be brought
up to date. Needs a separate CL since it's tricky.
Tested on go1.6, go1.7, go1.8.
Change-Id: I841189d30e131b7c49a4e8690ea7c40b55041bae
Reviewed-on: https://go-review.googlesource.com/36540
Reviewed-by: Robert Griesemer <gri@golang.org>
This fixes some print calls with wrong format directives. Additionally,
struct initialisers were changed to use keyed fields, purely to reduce
the amount of noise generated by go vet.
Change-Id: Ib9f6fd8f2dff7ce84826478de0ba83dda9746270
Reviewed-on: https://go-review.googlesource.com/21180
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This CL makes little observable difference to the behavior but paves the
way for streaming 'referrers' and (later) 'implements' queries which
scan the entire workspace, but print each result as soon as it is found.
The go/loader package now exposes a hook, AfterTypeCheck, that lets
clients inspect each package as soon as it is type-checked, and also
modify it, for instance to release unneeded data structures.
A 'referrers' query applied to an exported object must scan the entire
workspace. It uses this hook so to gather uses of the query object in
streaming fashion. However, for now, it still accumulates the results
and prints them all at the end, though I propose to change that in a
follow-up.
Code details:
- The referrers logic had a 2-iteration loop to load first the query
package and then if necessary the enlarged program. The second
iteration has now been unrolled and split into globalReferrers.
- Queries for package names (whether in a package declaration or
a qualified identifier) have been split off into packageReferrers.
It now loads all direct importers of the query package,
which catches some references that were missing before.
(It used to inspect only the forward dependencies of the query
package.)
Also:
- Referrers.Pos (the position of query identifier) was removed from the
JSON output. It's a nuisance to compute now, and it's already
absent from the plain output.
(In a follow-up, I plan to simplify the information content of the
JSON output exactly what is currently printed in the plain output.)
Change-Id: Ia5677636dc7b0fe4461a5d393107665757fb9a97
Reviewed-on: https://go-review.googlesource.com/19794
Reviewed-by: Daniel Morsing <daniel.morsing@gmail.com>
For the default build.Context, the Package.Goroot flag indicates when a
package was loaded from the standard library. Until now, the loader
used it to enable the typechecker's intrinsics for the "unsafe" package.
This seemed like a good check, but it is troublesome for clients that
use a nonstandard build.Context. For example, if a client defines
nonstandard Context hooks that load all packages, whether standard or
user-defined, from a flat sstable, there is no way for those hooks to
indicate which packages should have this flag set and which not. As a
result the contents of the "unsafe" package directory are treated as Go
source code when they are merely documentation.
Change-Id: Iea0a7cc9877507d73606391293971a28279c4e4b
Reviewed-on: https://go-review.googlesource.com/19188
Reviewed-by: Robert Griesemer <gri@golang.org>
...by making the cache non-blocking and duplicate-suppressing.
(The bottleneck was introduced while adding vendoring support.)
This reduces the 95th percentile map shard time for type-checking 120K
packages to 8min from 20min.
Also: move the I/O-limiting counting semaphore and the NoGoError check
from FindPackage to findPackage so that all implementations benefit.
Change-Id: I43527122262cf80475dd3212d78c340e1c71e36c
Reviewed-on: https://go-review.googlesource.com/18580
Reviewed-by: Robert Griesemer <gri@golang.org>
The loader treats GOROOT's "unsafe" package specially,
with no source files and a Package of types.Unsafe.
Tested on Go 1.4.1, 1.5, and ~1.6 (tip).
Fixes issue #13882
Change-Id: I86c4e394665d86a50ec3852d6d702f0e9c5d2276
Reviewed-on: https://go-review.googlesource.com/18457
Reviewed-by: Robert Griesemer <gri@golang.org>
(See https://go-review.googlesource.com/#/c/18450)
Details:
- delete buildutil.AllowVendor flag.
Instead use unexported 'go16' var in each package that needs it.
- delete buildutil.StripVendor; was never needed.
- remove inapplicable vendoring comments from loader14.go
- importgraph: go1.5: don't bother checking for 1.6 before calling
absolutize (minor simplifiication and deoptimization).
Tested on 1.4.1, 1.5, and ~1.6 (tip).
Change-Id: If692b0df2eb6c120a9c09d7b1ed99f1b4c6b0826
Reviewed-on: https://go-review.googlesource.com/18452
Reviewed-by: Russ Cox <rsc@golang.org>
A few files have been forked and tagged "go1.5,!go1.6" to work around
minor API changes between the two types packages:
- constant.Value.String() in oracle/describe.go and its tests;
- constant.ToInt must now be called before constant.Int64Val.
- types.Config{Importer: importer.Default()} in a number of places
- go/types/typeutil/import_test.go uses lowercase names to avoid 'import "C"'.
Files in go/types/typesutil, missing from my previous CL, have been
tagged !go1.5; these files will be deleted in February.
All affected packages were tested using 1.4.1, 1.5, and ~1.6 (tip).
Change-Id: Iec7fd370e1434508149b378438fb37f65b8d2ba8
Reviewed-on: https://go-review.googlesource.com/18207
Reviewed-by: Robert Griesemer <gri@golang.org>
This change will ensure that the tree continues to work with go1.4.1.
All files continue to depend on golang.org/x/tools/go/types, but in a
follow-up change, I will switch the primary files to depend on the
standard go/types package. Another (smaller) set of files will be
forked and tagged, this time !1.6, due to API differences between the
two packages.
All tests pass using 1.4.1, 1.5, and ~1.6 (tip).
Change-Id: Ifd75a6330e120957d646be91693daaba1ce0e8c9
Reviewed-on: https://go-review.googlesource.com/18333
Reviewed-by: Robert Griesemer <gri@golang.org>
Packages specified on the command line should be interpreted relative to
cwd iff they are local (e.g. ./http within $GOROOT/src/net), otherwise a
request for, say, "golang.org/x/net/http2/hpack" might return the vendored
package depending on the working directory.
The FindPackage hook function now takes a build.ImportMode parameter, so
it matches the signature of (*build.Context).Import. The AllowVendor
flag is enabled only for imports within source files, but not for the
initial packages.
+ test.
Change-Id: I756dc46b70928d2fd9f824e6670092d8169e0d64
Reviewed-on: https://go-review.googlesource.com/18318
Reviewed-by: Robert Griesemer <gri@golang.org>
Prior to this change, the loader made the simplifying assumption
that an import path is equal to a package path, that is, a
subdirectory of src/. (This assumption was already false because
relative imports "../foo" are possible, though discouraged.)
Now, an import "foo" may denote "a/vendor/foo" or "b/vendor/foo"
depending on whether it appears beneath a/ or b/. Thus import
paths and package paths are no longer the same, and the directory
containing an import declaration is a necessary input to the
import resolution function.
This change makes the loader hygienic w.r.t. the directory of
each import declaration, and cleans up the terminology.
The only API change is to the FindPackage hook, which most
clients never use.
Details:
- Add a 'fromDir string' parameter to the FindPackage hook function.
- Add a dir field to each PackageInfo.
- doImport (called by go/types) now consists of two steps:
use FindPackage(dir, importPath) to locate the package,
then consult the import map using the canonical package path.
Only the first step can fail.
- Memoize FindPackage.
- Simplify importInfo now that it no longer has to deal with errors.
Replace a condition variable with a channel.
- Use a separate type to report importErrors.
- Rename loadAll to importAll
- Delete the importMode constant.
- Test.
Change-Id: I3defab51bfa12b48b1511a2172fb48dc8e9150e6
Reviewed-on: https://go-review.googlesource.com/18053
Reviewed-by: Robert Griesemer <gri@golang.org>
Makes programs like ssadump work on packages using vendored code,
for example net/http.
For golang/go#12278.
Depends on CL 17726 in main repository.
Change-Id: Ibabf564e397044a0f449087124dd96161081baaf
Reviewed-on: https://go-review.googlesource.com/17727
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
This makes 'ulimit -n 128; go test -short -v' pass.
It did not before, and that was breaking the openbsd/386 builder.
For golang/go#11811.
Change-Id: Idfdb2f4007ed06c6084486c0e58a561add552d2c
Reviewed-on: https://go-review.googlesource.com/13695
Reviewed-by: Robert Griesemer <gri@golang.org>
This change sets ParserMode=AllErrors so that the parser is never
allowed to discard the AST and use a dummy one just because it saw too
many errors.
Also, change (*loader.Program).PathEnclosingInterval so that other
clients that forget to set this flag don't panic while calling
fset.File(f.Pos()).Base() on an ast.File f with no position info.
Change-Id: Ie544f169d367d2aa85426212b27063dc72e36fb1
Reviewed-on: https://go-review.googlesource.com/10290
Reviewed-by: Robert Griesemer <gri@golang.org>
The loader package now loads, parses, and type-checks a whole program
from source, and that is all.
Also:
- simplified loader logic
- ssa.Create is gone; use ssautil.CreateProgram.
- ssautil.LoadPackage renamed to BuildPackage.
It is now independent of go/types' Import hook and the Packages map.
- ssadump: -importbin flag removed.
The value of this flag was that it caused the tool to print IR
for only a single package; this is now the normal behaviour.
Fixes#9955
Change-Id: I4571118258ab1a46dccece3241b7dc51401a3acc
Reviewed-on: https://go-review.googlesource.com/8953
Reviewed-by: Robert Griesemer <gri@golang.org>
Also: make it easier to find packages. Hide the importMap field
(never used) and expose a (*Program).Package method which searches
importMap and then Created.
Also: move huge comments into doc.go.
Change-Id: Iad96a12524b7c41ad9acd1e806af23171e71fa7c
Reviewed-on: https://go-review.googlesource.com/9030
Reviewed-by: Robert Griesemer <gri@golang.org>
Remove all dependencies from non-test code in go/ssa to go/loader,
except the deprecated Create function which will be eliminated in
favor of ssautil.CreateProgram in a mechnanical followup.
Add Examples of two main use cases of SSA construction:
loading a complete program from source; and
building a single package, loading its dependencies from import data.
Add tests to ssautil of the two load functions.
Suggestions welcome for better names.
Planned follow-ups:
- replace all references to ssa.Create with ssautil.CreateProgram and eliminate it.
- eliminate support in go/loader for the ImportBinary flag, and the
PackageCreated hook which is no longer needed since clients can
create the package themselves (see Example).
Step 1 to fixing issue 9955.
Change-Id: I4e64df67fcd5b7f0c0388047e06cea247fddfec5
Reviewed-on: https://go-review.googlesource.com/8669
Reviewed-by: Robert Griesemer <gri@golang.org>
Config.Cwd sets the base directory; os.Getwd is its default.
+Test.
Change-Id: I213abfb30085cd1306719ed6f94aeae6a3170bc0
Reviewed-on: https://go-review.googlesource.com/7502
Reviewed-by: Robert Griesemer <gri@golang.org>
Add tests for:
- this naming
- missing file in created package
Change-Id: I07c66b66e845c52d4685509c362b34f1f0c92648
Reviewed-on: https://go-review.googlesource.com/6310
Reviewed-by: Robert Griesemer <gri@golang.org>
...since the zero value is more useful by far.
This is a breaking API change, obviously. (One or two tests in this
CL have intentional been left using the zero value, i.e., they now
load source.)
Change-Id: I42287bfcdb1afef8ee84e5eac12534dd0a1fd5d2
Reviewed-on: https://go-review.googlesource.com/5653
Reviewed-by: Robert Griesemer <gri@golang.org>
Google's proprietary build system, for example, does not use the
_test.go suffix to distinguish test from non-test files; this
information is stated explicitly in another form.
Change-Id: I3a8e919dbc556b6d5cfea1d2123da2616bd934d4
Reviewed-on: https://go-review.googlesource.com/5450
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Before this change, many kinds of error would cause the loader to stop.
making it brittle when analyzing large codebases, as in "godoc -analysis".
This change moves operations that used to occur during
configuration---(*build.Context).Import, loading, and parsing of
initial packages---into the Load call, and ensures that all failures
during Loading are reported at the end so that the maximum amount of
progress is made.
Also: redesign the tests and add many new cases.
Change-Id: Ia8cd99416af7c5d4a5fe133908adfa83676d401f
Reviewed-on: https://go-review.googlesource.com/3626
Reviewed-by: Robert Griesemer <gri@golang.org>
See loader.go comments for orientation.
This is what happens when you use start using Dmitry's new trace tool. :)
Tests:
- added test of cycles
- load/parse/typecheck of 'godoc' is ~2.5x faster
- race detector finds no errors.
Change-Id: Icb5712c7825002342baf471b216252cecd9149d6
Reviewed-on: https://go-review.googlesource.com/1582
Reviewed-by: Robert Griesemer <gri@golang.org>
PackageCreated is a hook called when a types.Package
is created but before it has been populated.
The package's import Path() and Scope() are defined,
but not its Name() since no package declaration has
been seen yet.
Clients may use this to insert synthetic items into
the package scope, for example.
Change-Id: I210a0c4c766f03f715f03f26d5cd765f15f56e04
Reviewed-on: https://go-review.googlesource.com/2138
Reviewed-by: Alan Donovan <adonovan@google.com>
Rewrite performed with this command:
sed -i '' 's_code.google.com/p/go\._golang.org/x/_g' \
$(grep -lr 'code.google.com/p/go.' *)
LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/170920043
(godoc is excluded from this CL since it will continue to use
/src/pkg in its URL namespace, making the necessary cleanup
more subtle.)
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/141770043
Previously these were recorded in the PackageInfo, but not
reported to the user's error handler (types.Config.Error),
which is typically what prints them.
Minor subtlety: that function must now be able to handle error
values that are not of type types.Error.
+ Test (and renamed it).
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/121290043
This transforms the virtualized directory (build.Context).Dir to a physical one,
for proprietary build systems that distinguish them.
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/116230043
This method was deemed unfit for the API. See the original CL for discussion.
««« original CL description
go.tools/go/loader: Add Program.FilePath convenience method for getting the full path of a source file.
LGTM=gri
R=gri, adonovan
CC=golang-codereviews
https://golang.org/cl/107160049
»»»
LGTM=gri
R=gri, adonovan
CC=golang-codereviews
https://golang.org/cl/107570043
Before, Load() would just fail. Now, it gathers all frontend
errors (not just the first go/types error) in PackageInfo.Errors.
There are still cases where Load() can fail hard, e.g. errors in x_test.go
files. That case is trickier to fix and remains a TODO item.
Also, make godoc display all scanner/parser/type errors in the source view.
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/108940043
Files that import "C" are not valid Go source files and
require preprocessing. Until now, the loader has simply
hard-coded CGO_ENABLED=0 (in effect) which causes go/build to
use build tags select pure Go implementations where they exist
(e.g. in $GOROOT). Where they don't (e.g. arbitrary user
code) this leads to masses of spurious type errors.
(Reported by Guillaume Charmes, private correspondence.)
This change causes the loader to invoke the cgo preprocessor
on such files and to load the preprocessed files instead,
using the original names. This means that the syntax offset
position information is garbage, although thanks to //line
directives, the line numbers at least should be good.
See comment in cgo.go for details.
This CL changes the loader's default behaviour and may make it slower.
CGO_ENABLED=0 enables the old behaviour.
Tested via stdlib_test, which now loads all standard packages
using cgo, and also exercises CGO_ENABLED=0 for "net" and "os/user".
LGTM=gri
R=gri, rsc
CC=golang-codereviews, guillaume.charmes
https://golang.org/cl/86140043
Existing tools use the default value of zero; their behaviour is unchanged.
(*Config).ParseFile is used only from tests.
LGTM=crawshaw, rsc, gri
R=crawshaw, gri, rsc
CC=golang-codereviews
https://golang.org/cl/79290044
Also, add loader.Config.DisplayPath hook, which allows the
filename returned by build.Context.Import() to be transformed
prior to attaching to the AST. This allows a virtual file
system to be used without leaking into the user interface.
Eliminate parsePackageFiles hook; I don't think we need it any
more. The test that was using it has been rewritten to use
the build.Context hooks.
LGTM=gri
R=gri, crawshaw
CC=daniel.morsing, golang-codereviews, rsc
https://golang.org/cl/75520046
Alan Donovan
Michael Matloob
Kirill Smelkov
Dominik Honnef
Alan Donovan
Russ Cox
Peter Collingbourne
David Symonds
Andrew Gerrand
Gordon Klaus