Commit Graph

621 Commits

Author SHA1 Message Date
Alan Donovan 3a85b8da38 refactor/importgraph: reduce I/O concurrency to avoid EMFILE
Use an concurrency-limiting semaphore to reduce I/O parallelism in Import.

Also, start the producer in a new goroutine so that it runs in parallel
with the consumer.  Paradoxically, this reduces the peak number of
goroutines.

Also, in buildutil.ForEachPackage, make the concurrency limiting
semaphore global, since I/O parallelism is a process-wide resource.

Change-Id: I282b717c50603361826e5675077c9f464c874132
Reviewed-on: https://go-review.googlesource.com/18215
Reviewed-by: Michael Matloob <matloob@golang.org>
2015-12-30 22:01:03 +00:00
Alan Donovan b2a7698651 go/loader: in stdlib_test, remove workaround for lack of vendoring
Change-Id: I058081ef596e8b5231321ef05d5256ec44cc58c6
Reviewed-on: https://go-review.googlesource.com/18214
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-12-30 16:37:28 +00:00
Alan Donovan a09dffbb6c go/vcs: workaround EOF bug in token-based XML decoder
...using same logic as in $GOROOT/go/src/cmd/go/discovery.go

See issue #13757

Change-Id: I2ace0abed0743f4a8675fa8780e190d640b57e6c
Reviewed-on: https://go-review.googlesource.com/18196
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-12-29 20:27:04 +00:00
Dmitri Shuralyov 157f8abbbd go/vcs: accept domain.root in RepoRootForImportDynamic
This is the same change as in https://golang.org/cl/12193.

Fixes golang/go#13506.

Related to golang/go#9357.

Change-Id: I9c7d956008641b1907e14bcb08198235f5f9552f
Reviewed-on: https://go-review.googlesource.com/18152
Reviewed-by: Russ Cox <rsc@golang.org>
2015-12-29 15:00:21 +00:00
Alan Donovan 2b78567943 go/loader: fix tests to work at go1.5 and tip
Change-Id: Icc80f112a7ec4b64519220c57b6ccb2612166656
Reviewed-on: https://go-review.googlesource.com/18197
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-12-28 22:42:32 +00:00
Alan Donovan b14dd1a5b7 go/importer: in TestImportStdLib, skip 2 packages that use vendoring
(See issue #13756)

Change-Id: Icda14c34f3460e941ae092f147432c795465a077
Reviewed-on: https://go-review.googlesource.com/18195
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-12-28 21:28:10 +00:00
Alan Donovan 9b7a39c6be go/loader: fix brittle example test
We need a better approach for tests such as these.

Change-Id: I6fb608acc096088fe377f6f2c40ff6124cf3d09c
Reviewed-on: https://go-review.googlesource.com/18194
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-12-28 21:24:47 +00:00
Alan Donovan 8463610142 go/ssa/interp: enable support for Example tests
1. Added intrinsic for os.Pipe to avoid use of "unsafe".  This makes
   Example tests work in the interpreter, fixing the broken "unicode"
   package (which recently sprouted some Examples) and allowing us to
   enable tests for:

      encoding/csv
      flag
      path
      text/scanner
      unicode/utf8

   The implementation of os.Pipe may require tweaking for non-Linux platforms.

2. Added intrinsic for strings.Index to avoid assembly dependency.

3. Added dumb intrinsic for testing۰runExample to work around a bug in
   go/ssa/testmain.go that requires an invasive fix
   (Until then, the output of Example functions will not be checked.)

Change-Id: I6374c9c47aa802275b7cdc98525e057f5db0615a
Reviewed-on: https://go-review.googlesource.com/18191
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-12-28 21:19:28 +00:00
Alan Donovan e3e5aae018 go/types: in TestStdlib, skip 2 packages that use vendoring
(gcimporter does not support vendoring---see issue #13756)

Change-Id: Ief4101e4f33007d497ed1519845febfcffdded38
Reviewed-on: https://go-review.googlesource.com/18193
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-12-28 21:04:49 +00:00
Alan Donovan f75338cbc6 go/types: skip two failing $GOROOT/test/fixedbugs tests
Change-Id: I7dfb95f34ae23c0b7905714b32068dc336a05656
Reviewed-on: https://go-review.googlesource.com/18192
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-12-28 19:40:50 +00:00
Alan Donovan d6e83e534d go/loader: changes for vendor support
Prior to this change, the loader made the simplifying assumption
that an import path is equal to a package path, that is, a
subdirectory of src/.  (This assumption was already false because
relative imports "../foo" are possible, though discouraged.)
Now, an import "foo" may denote "a/vendor/foo" or "b/vendor/foo"
depending on whether it appears beneath a/ or b/.  Thus import
paths and package paths are no longer the same, and the directory
containing an import declaration is a necessary input to the
import resolution function.

This change makes the loader hygienic w.r.t. the directory of
each import declaration, and cleans up the terminology.
The only API change is to the FindPackage hook, which most
clients never use.

Details:
- Add a 'fromDir string' parameter to the FindPackage hook function.
- Add a dir field to each PackageInfo.
- doImport (called by go/types) now consists of two steps:
  use FindPackage(dir, importPath) to locate the package,
  then consult the import map using the canonical package path.
  Only the first step can fail.
- Memoize FindPackage.
- Simplify importInfo now that it no longer has to deal with errors.
  Replace a condition variable with a channel.
- Use a separate type to report importErrors.
- Rename loadAll to importAll
- Delete the importMode constant.
- Test.

Change-Id: I3defab51bfa12b48b1511a2172fb48dc8e9150e6
Reviewed-on: https://go-review.googlesource.com/18053
Reviewed-by: Robert Griesemer <gri@golang.org>
2015-12-22 18:31:24 +00:00
Alan Donovan 27534aa951 go/buildutil: changes for vendor support
Add AllowVendor constant for convenience across x/tools.
 (Will delete go/loader's importMode later)
Add StripVendor utility function + test.

Change-Id: I885076cb4ea67588996d85b749b85f49cd619f0d
Reviewed-on: https://go-review.googlesource.com/18049
Reviewed-by: Robert Griesemer <gri@golang.org>
2015-12-18 20:00:43 +00:00
Alan Donovan 42bc1282f3 go/loader: update example_test for Go1.6
Change-Id: Ib3bf913e3c8f3cf06ba855f8dda855453e0a1aa2
Reviewed-on: https://go-review.googlesource.com/18048
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-12-18 18:15:14 +00:00
Russ Cox 34ae141ecc go/loader: use build.AllowVendor during Import on Go 1.6 and later
Makes programs like ssadump work on packages using vendored code,
for example net/http.

For golang/go#12278.
Depends on CL 17726 in main repository.

Change-Id: Ibabf564e397044a0f449087124dd96161081baaf
Reviewed-on: https://go-review.googlesource.com/17727
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-12-17 20:34:07 +00:00
Brad Fitzpatrick 1cdaff4a02 go/types: fix build
Fixes golang/go#13603

Change-Id: Ie35094e9fd9976317340fae19425bffa307d7c8a
Reviewed-on: https://go-review.googlesource.com/17753
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-12-14 01:42:23 +00:00
David Symonds e842f6af3a go/types: Remove ancient Go 1.1 compatibility indirections.
Change-Id: Ie16cc01ea2bd4806eaa7bd77cfbd2fa90f11ea36
Reviewed-on: https://go-review.googlesource.com/17792
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-12-14 00:47:24 +00:00
Alex Brainman fff8fd7b19 go/buildutil: ignore path letters case in ContainingPackage on windows
Fixes golang/go#13368

Change-Id: I4cee2078bc64b6f175f206daa9609246cc1b1f85
Reviewed-on: https://go-review.googlesource.com/17211
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-11-25 03:21:30 +00:00
Alan Donovan 977844c7af go/ssa: fix crash when building map[*struct{}]bool{{}: true}
Since Go 1.5, if a key expression in a map literal is itself a composite
literal, the type may be omitted.  An & operation may be implied,
but was not generated, resulting in a type mismatch crash.

Added regression test.
Also, added missing intrinsic math.hasSSE4 to interpreter.

Fixes issue 13341

Change-Id: I6a5e560460cdcac295a500a3fc3738cbe8aa5853
Reviewed-on: https://go-review.googlesource.com/17123
Reviewed-by: Robert Griesemer <gri@golang.org>
2015-11-23 19:08:17 +00:00
Dmitri Shuralyov 747ae2594e go/buildutil: Fix example code for TagsFlag.
The example incorrectly used buildutil.TagsDoc identifier which does not
exist. It should be buildutil.TagsFlagDoc instead.

The issue was introduced in https://golang.org/cl/9172 when this code was
first written.

Also use more idiomatic style for Go comments, '//' followed by a space,
then tab (instead of skipping the space). Although both render correctly
in godoc, this form is more correct and consistent.

Change-Id: I3b3c9767fe313106a8ff81e7887f3241c5806b59
Reviewed-on: https://go-review.googlesource.com/16743
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-11-09 14:36:24 +00:00
Alan Donovan 0f9d71c428 go/pointer: fix bug in constraint generation of "for _, v := range map"
+ regression test

Change-Id: I9ec28f222e14af0cb737494e3c48e5423cdcf236
Reviewed-on: https://go-review.googlesource.com/16294
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2015-10-26 16:28:05 +00:00
Daniel Morsing b6b32a4fbf go/ssa: emit DebugRefs for x, y = z() assignments
This would cause oracle callee lookup failures when the RHS is an
invoke-style call.

Fixes golang/go#12999

Change-Id: Ifd561c4e7bf26f57ace5f62afac746b926c70993
Reviewed-on: https://go-review.googlesource.com/16210
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-10-23 10:25:05 +00:00
Brad Fitzpatrick 4caddfa68c go/vcs: fix doc typos
Change-Id: Ife1f9534d08d3bd9beeb4e6e798733d71e6f72b5
Reviewed-on: https://go-review.googlesource.com/15750
Reviewed-by: Dmitri Shuralyov <shurcool@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-10-10 23:46:02 +00:00
Alan Donovan 9d2ff756b7 go/loader: document cgo limitation
Change-Id: I8ce744d204fe2850d322b3095d04e8a399984983
Reviewed-on: https://go-review.googlesource.com/15391
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-10-05 16:43:58 +00:00
Alan Donovan 8049553ca8 go/ssa: use sync.Once instead of sync.atomic
This adds a missing memory barrier at the end of Package.Build.

Change-Id: Ife35d5ad5a48ba121f35656fef682863d4f2aef6
Reviewed-on: https://go-review.googlesource.com/14761
Reviewed-by: Robert Griesemer <gri@golang.org>
2015-09-18 15:11:51 +00:00
Alan Donovan afcda55b39 go/ssa: rename some API features (incompatible change)
Rename the following exported names:
	(*Program).BuildAll → Build
	(*Program).Method   → MethodValue
	Package.Object      → Pkg
Also:
	(*Function).pkgobj  → pkg

Change-Id: Iff7e6c240ebe6786ba759278ac0daa3d66698013
Reviewed-on: https://go-review.googlesource.com/14134
Reviewed-by: Robert Griesemer <gri@golang.org>
2015-08-31 22:20:12 +00:00
Alan Donovan b9f1f6a3c1 go/loader: in stdlib_test, skip packages that depend on vendoring
until vendoring support is added (soon).

Change-Id: I954514b5c6b82622c0c831a289ccdcc16dbd37f8
Reviewed-on: https://go-review.googlesource.com/14130
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-08-31 20:55:27 +00:00
Russ Cox 93604a3dc2 go/loader: fix fd leak and limit calls to ioutil.ReadDir
This makes 'ulimit -n 128; go test -short -v' pass.
It did not before, and that was breaking the openbsd/386 builder.

For golang/go#11811.

Change-Id: Idfdb2f4007ed06c6084486c0e58a561add552d2c
Reviewed-on: https://go-review.googlesource.com/13695
Reviewed-by: Robert Griesemer <gri@golang.org>
2015-08-18 23:39:26 +00:00
Robert Griesemer 958ab4f550 x/tools: disable tests not supported on Android
For golang/go#11811.

Change-Id: Icf16a2d47fcf2fe0d79dd825ccb851a3d63a660f
Reviewed-on: https://go-review.googlesource.com/13268
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2015-08-06 00:36:03 +00:00
Robert Griesemer 874e71548b go/ssa/interp: exclude another long-running test if -short is set
For #11811.

Change-Id: Ic3bf3c035ce402f144b5b6670d5ef062874b167e
Reviewed-on: https://go-review.googlesource.com/13260
Reviewed-by: Chris Manghane <cmang@golang.org>
2015-08-05 21:28:02 +00:00
Robert Griesemer 50fdc6b946 go/loader: exclude example tests from Windows tests
The output of ExampleConfig_CreateFromFiles and ExampleConfig_Import
are different for Windows that for other platforms: They contain
internal/syscall/windows packages and unicode/utf16 not present in
the output for other platforms.

For golang/go#11811.

Change-Id: Id391fbeec8123616da86cb68fc3cefcd513b2493
Reviewed-on: https://go-review.googlesource.com/13032
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-07-31 22:13:26 +00:00
Robert Griesemer 784fe34e06 go/ssa/interp: abort long-running test when testing.Short() is set
This should help on slower machines.

For golang/go#11811.

Change-Id: Ibb5d5bf0f6cedcda6437ef0ee3fc1f4ba89dab90
Reviewed-on: https://go-review.googlesource.com/13009
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-07-31 22:12:02 +00:00
Alex Brainman d89a972634 go/loader: remove blank line
Blank line was inserted by mistake in CL 12830.

Change-Id: I9e4b3562a4efcab43c5a85c2960a0d22c0aab752
Reviewed-on: https://go-review.googlesource.com/12883
Reviewed-by: Dmitri Shuralyov <shurcool@gmail.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
2015-07-31 21:22:20 +00:00
Robert Griesemer fe546a3233 go/types: exclude some tests when running against Go 1.4
For golang/go#11811.

Change-Id: I130f9608840177cfb7fb9fae30765fcb5aa77411
Reviewed-on: https://go-review.googlesource.com/13008
Reviewed-by: Russ Cox <rsc@golang.org>
2015-07-31 20:09:52 +00:00
Alan Donovan 14b8253455 go/loader: fork example_test.go for go1.4 since the expectations differ
Change-Id: I91138351525ce49b9afccdc66c59e7a4c1c0e096
Reviewed-on: https://go-review.googlesource.com/12870
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2015-07-30 00:13:23 +00:00
Alex Brainman f25d0131d4 go/loader: make TestLoad_ParseError_AllowErrors pass on windows
Update golang/go#11811

Change-Id: Ic5f1ea87c88d563b6e0ef2d44042e28a9ea03a03
Reviewed-on: https://go-review.googlesource.com/12830
Reviewed-by: Robert Griesemer <gri@golang.org>
2015-07-30 00:12:49 +00:00
Alan Donovan 739a26ae62 go/loader: add paranoid assertions to help diagnose issue 11012
Change-Id: I5e24fe0fb605bdb39de11309e0e3c8ffd7a1eb8b
Reviewed-on: https://go-review.googlesource.com/12842
Reviewed-by: Robert Griesemer <gri@golang.org>
2015-07-29 18:39:32 +00:00
Alex Brainman 166a1118e9 go/loader: update examples Output
Some standard library dependencies have changed (packages and files).
Both ExampleConfig_CreateFromFiles and ExampleConfig_Import Output
needs to be adjusted. Do that.

Update golang/go#11811

Change-Id: I523f2adc1aa46f0932a71ccb23dd7c5a6b07fb27
Reviewed-on: https://go-review.googlesource.com/12832
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-07-29 16:23:00 +00:00
Michal Cierniak 7fe0389f47 go/ssa: update package doc to mention ssautil.CreateProgram
Change-Id: I92fb6cf5683c335826d2e84394cc594639e34fe6
Reviewed-on: https://go-review.googlesource.com/12542
Reviewed-by: Dave Cheney <dave@cheney.net>
2015-07-27 18:48:45 +00:00
Robert Griesemer 1c460ffc92 go/types: run tests that were excluded by mistake
Backport of https://go-review.googlesource.com/#/c/12517/.

TBR=adonovan

Change-Id: I6b010c977c9a770a27c3fa1271c07c38d9b84185
Reviewed-on: https://go-review.googlesource.com/12601
Reviewed-by: Robert Griesemer <gri@golang.org>
2015-07-23 22:55:45 +00:00
Robert Griesemer 88421f4cb2 go/types: unexport UniverseByte/Rune; make Typ a slice
Also: backport https://go-review.googlesource.com/#/c/12443/ .

Change-Id: Ia8a664c627a70a17701b2d48946704c15f4f49c0
Reviewed-on: https://go-review.googlesource.com/12482
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-07-21 18:42:36 +00:00
Robert Griesemer 436a79e5dd go/types: fix internal inInteger operand predicate
Backport of https://go-review.googlesource.com/#/c/12045/

Fixes golang/go#11594.

Change-Id: I3c81e2b1bbdc084d4a323fd963c1c780054ce223
Reviewed-on: https://go-review.googlesource.com/12046
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-07-21 17:32:59 +00:00
Alan Donovan c5ca59aab8 go/loader: issue informative error (not panic) if cgo used in ad hoc package
See https://github.com/golang/go/issues/11627.

Change-Id: I458bc4ea54d0db34f3ba96060d284eda4bad7111
Reviewed-on: https://go-review.googlesource.com/12190
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2015-07-14 18:01:18 +00:00
Alan Donovan 543085f0ec go/callgraph: add "concurrent" or "deferred" prefix to (*Edge).Description
Fixes golang/go#11518

Change-Id: If6c86c1995c0b6a1226deedaa1713947baa6593a
Reviewed-on: https://go-review.googlesource.com/12085
Reviewed-by: Daniel Morsing <daniel.morsing@gmail.com>
2015-07-13 18:17:08 +00:00
Robert Griesemer 4cd43f3077 go/types: exclude tests that run too long
Backport of the go/types-specific changes of:

https://go-review.googlesource.com/#/c/11673/

(from std repo go/types). This will allow running
the go/types tests again in the x/tools repo.

Change-Id: I97d4664d5aed6309e74b571f86f36f8bb4df4fca
Reviewed-on: https://go-review.googlesource.com/12042
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-07-10 21:57:21 +00:00
Alan Donovan 997b3545fd go/types: change {Type,Object,Selection}String to accept a Qualifier function
The optional Qualifier function determines what prefix to attach to
package-level names, enabling clients to qualify packages in different
ways, for example, using only the package name instead of its complete
path, or using the locally appropriate name for package given a set of
(possibly renaming) imports.

Prior to this change, clients wanting this behavior had to copy
hundreds of lines of complex printing logic.

Fun fact: (*types.Package).Path and (*types.Package).Name are valid
Qualifier functions.

We provide the RelativeTo helper function to create Qualifiers so that
the old behavior remains a one-liner.

Fixes golang/go#11133

Change-Id: Ibd63f639c7b3aa1738826d6165f2d810efeb8293
Reviewed-on: https://go-review.googlesource.com/11692
Reviewed-by: Robert Griesemer <gri@golang.org>
2015-06-30 19:00:00 +00:00
Alan Donovan 749901c676 go/types: add an API test of the Scope type
Also: make (*Scope).Innermost work for Package scopes.

Change-Id: I9836676e94f95df897101606bed6f29ba46e0f9d
Reviewed-on: https://go-review.googlesource.com/11691
Reviewed-by: Robert Griesemer <gri@golang.org>
2015-06-29 20:05:11 +00:00
Alan Donovan 27bc91e0ba go/ssa/interp: disable interpretation of packages that have sprouted Examples
since we can't interpret os.Pipe yet.

Change-Id: If27dadc18532274ce97ad7e7557e8614dd15279e
Reviewed-on: https://go-review.googlesource.com/11712
Reviewed-by: Robert Griesemer <gri@golang.org>
2015-06-29 19:24:15 +00:00
Robert Griesemer e841ad2e00 go/types: fix minor typo in comment
Change-Id: I2d696655c409f94a5ab6a3837b4b83bc7eb0b15d
Reviewed-on: https://go-review.googlesource.com/11366
Reviewed-by: Robert Griesemer <gri@golang.org>
2015-06-23 21:03:00 +00:00
Robert Griesemer 8c8cd541fa go/types: in string(x) conversions, x must be of integer type
Fixes golang/go#11357.

Change-Id: Id6994a0fe3830cf56d3dbdd60a4dff89404e5a41
Reviewed-on: https://go-review.googlesource.com/11365
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-23 20:21:05 +00:00
Robert Griesemer 6369699504 go/types: better error message for certain unaary/binary expression errors
Fixes golang/go#11367.

Change-Id: Ic7f1bad51165685097c909fd18c245a57924cdc8
Reviewed-on: https://go-review.googlesource.com/11363
Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-23 19:43:07 +00:00