Before, Load() would just fail. Now, it gathers all frontend
errors (not just the first go/types error) in PackageInfo.Errors.
There are still cases where Load() can fail hard, e.g. errors in x_test.go
files. That case is trickier to fix and remains a TODO item.
Also, make godoc display all scanner/parser/type errors in the source view.
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/108940043
Blocks dominated by "if false" should be retained in the
initial SSA form so they remain visible to subsequent source
code analysis tools.
In any case, true compilers already need a stronger version of
this optimization so they can simplify CFGs such as this:
const x, y = ...
switch x {case y:...}
where a branch is constant but the comparison of constants
does not occur within an expression.
LGTM=gri
R=gri
CC=golang-codereviews, pcc
https://golang.org/cl/101250043
The SSA builder shouldn't be in the business of
interprocedural optimization, especially in the presence of
concurrency.
This causes the instruction count to increase by 0.03%.
LGTM=gri
R=gri, pcc
CC=golang-codereviews
https://golang.org/cl/105020045
It was making the unsound assumption that cgn==nil => v is one
of {Global,Function,Const,Capture} to avoid checking v's type,
which is what it now does. This caused more expensive
constraints to be generated, which is suboptimal though not
wrong exactly.
In one benchmark, this change reduces the number of complex
constraints by about 23% of loads and 53% of stores, and
increases the number of (simple) copy constraints by about 5%.
LGTM=crawshaw
R=crawshaw
CC=golang-codereviews
https://golang.org/cl/106940043
Files that import "C" are not valid Go source files and
require preprocessing. Until now, the loader has simply
hard-coded CGO_ENABLED=0 (in effect) which causes go/build to
use build tags select pure Go implementations where they exist
(e.g. in $GOROOT). Where they don't (e.g. arbitrary user
code) this leads to masses of spurious type errors.
(Reported by Guillaume Charmes, private correspondence.)
This change causes the loader to invoke the cgo preprocessor
on such files and to load the preprocessed files instead,
using the original names. This means that the syntax offset
position information is garbage, although thanks to //line
directives, the line numbers at least should be good.
See comment in cgo.go for details.
This CL changes the loader's default behaviour and may make it slower.
CGO_ENABLED=0 enables the old behaviour.
Tested via stdlib_test, which now loads all standard packages
using cgo, and also exercises CGO_ENABLED=0 for "net" and "os/user".
LGTM=gri
R=gri, rsc
CC=golang-codereviews, guillaume.charmes
https://golang.org/cl/86140043
Also:
- extend Parent() to all Values and add to interface:
(Builtin/Const/Global => nil; Function => Enclosing)
- hide Function.Enclosing since it's now redundant wrt Parent()
- make (*Function).String robust for synthetics without pkg object
LGTM=gri
R=gri
CC=golang-codereviews, khr
https://golang.org/cl/87580044
This optimization reduces solve time (typically >90% of the
total) by about 78% when analysing real programs. It also
makes the solver 100% deterministic since all iterations are
ordered.
Also:
- remove unnecessary nodeid parameter to solve() method.
- don't add a fieldInfo for singleton tuples (cosmetic fix).
- inline+simplify "worklist" type.
- replace "constraintset" type by a slice.
LGTM=crawshaw
R=crawshaw
CC=golang-codereviews
https://golang.org/cl/95240043
Until now, the same Function was used to represent a method
(T)func() and the "method expression" function func(T) formed
from it. So the SSA code for this:
var buf bytes.Buffer
f := Buffer.Bytes
f(buf)
buf.Bytes()
would involve an implicit cast (ChangeType) on line 2.
However, compilers based on go/ssa may want to use different
calling conventions for them, like gccgo does (see issue
7839). This change decouples them by using an anonymous
function called a "thunk", rather like this:
f := func(r *bytes.Buffer) []byte { return r.Bytes() }
Thunks are similar to method wrappers; both are created by
makeWrapper.
"Interface method wrappers" were a special case of thunks for
direct calls (no indirection/fields) of interface methods.
They are now subsumed by thunks and have been deleted. Now
that only the needed thunks are built, we don't need to
populate the concrete method sets of interface types at all,
so (*Program).Method and LookupMethod return nil for them.
This results in a slight reduction in function count (>1%) and
instruction count (<<1%).
Details:
go/ssa:
- API: ChangeType no longer supports func/method conversions.
- API: (*Program).FuncValue now returns nil for abstract
(interface) methods.
- API: (*Function).RelString simplified.
"$bound" is now a suffix not a prefix, and the receiver
type is rendered package-relative.
- API: Function.Object is now defined for all wrappers too.
- API: (*Program).Method and LookupMethod return nil for
abstract methods.
- emitConv no longer permits (non-identical)
Signature->Signature conversions. Added assertion.
- add and use isInterface helper
- sanity: we check packages after Build, not Create, otherwise
cross-package refs might fail.
go/pointer:
- update tests for new function strings.
- pointer_test: don't add non-pointerlike probes to analysis.
(The error was checked, but too late, causing a panic.)
- fixed a minor bug: if a test probe print(x) was the sole
reference to x, no nodes were generated for x.
- (reflect.Type).MethodByName: updated due to ssa API changes.
Also, fixed incorrect testdata/funcreflect.go expectation
for MethodByName on interfaces.
oracle:
- fix for new FuncValue semantics.
- a "pointsto" query on an I.f thunk now returns an error.
Fixesgolang/go#7839
LGTM=gri
R=gri
CC=golang-codereviews, pcc
https://golang.org/cl/93780044
- Replaced check.initDependencies with check.initOrder;
this is the only semantic change, it affects only the
value of Info.InitOrder.
- Added additional init order test cases and adjusted
existing tests.
- Moved orderedSetObjects from resolver.go to ordering.go.
Fixesgolang/go#7964.
LGTM=adonovan
R=adonovan
CC=golang-codereviews
https://golang.org/cl/91450043
Programs such as this cause the PtrTo solver to attempt to
enumerate an infinite set of types {T, *T, ..., *******T, etc}.
t := reflect.TypeOf(T{})
for {
t = reflect.PtrTo(t)
}
The fix is to bound the depth of reflectively created types at
about 4 map/chan/slice/pointer constructors.
+ test.
LGTM=gri
R=gri
CC=crawshaw, golang-codereviews
https://golang.org/cl/102030044
go/types doesn't correctly round the largest possible
float32 literal values and fails. Instead of relying
on Rat.Float64 and float32 conversion, we need a
Rat.Float32 implementation with correct rounding.
LGTM=adonovan
R=adonovan
CC=golang-codereviews
https://golang.org/cl/96540045
Very long instructions caused the printf width spec to go
negative, which causes right-padding, often several lines'
worth.
Also: print the basic block comment once on the RHS. It's too
verbose to print it each time we mention the block.
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/97490046
The previous implementation would cause the graph to contain
many duplicate edges resulting in very large cross products,
so that for some inputs (and random map iteration orders) the
running time of DeleteSyntheticNodes was many minutes---more
than the pointer analysis!
Duplicate edges can arise from an interface call that
dispatches to several different wrapper functions each
wrapping the same declared method.
For example, in the callgraph for go/types, a call to
Object.Pos() dispatches to the synthetic functions (*Type).Pos
and (*Var).Pos, each of which wrap (*object).Pos(). After
DeleteSyntheticNodes, Object.Pos() appeared to call
(*object).Pos() twice.
This change builds the set of all edges and avoids adding
edges already in the set.
Also, document findings.
LGTM=crawshaw
R=crawshaw
CC=golang-codereviews
https://golang.org/cl/96100043
Method expressions T.f are reported as having type (T)func(T),
i.e. T appears twice, as a receiver and a regular parameter.
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/96780044
During block optimization, degenerate conditional logic such
as "false && x" may result in single-predecessor blocks
containing φ-nodes. (Ideally such φ-nodes would be replaced
by their sole operand, but that requires Referrers information
which isn't computed until later.) It is obviously not safe
to fuse such blocks, so now we don't.
Fixesgolang/go#7840
LGTM=gri
R=gri
CC=golang-codereviews, pcc
https://golang.org/cl/90620043
GccgoInstallation.InitFromDriver currently parses
the output of gccgo -### to get the gcc version,
target triple, and library paths. At least with
Ubuntu's stock libgo5 package, the search path for
.gox files derived from the version is incorrect.
gccgo uses the DEFAULT_TARGET_VERSION macro when
constructing the search path; this value can be
retrieved from gccgo via the "-dumpversion" flag.
Fixesgolang/go#7772.
LGTM=iant, gri
R=golang-codereviews, iant, gri
CC=golang-codereviews
https://golang.org/cl/88150043
Otherwise on Windows the enumerated package "net\\http" will
be distinct from the imported package "net/http" leading to
strange errors. (A similar bug was fixed in go/ssa/stdlib_test.go.)
Fixesgolang/go#7189
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/86170043
Side-effect: Because interfaces are now type-checked in reverse order,
cycle errors in interface declarations appear at the "end" rather than
at the "beginning" of the cycle in the source code. This is harmless.
Eventually we may want to do dependency order determination and thus
cycle detection for all types before fully type-checking them, which
might simplify some code and also produce consistently positioned cycle
errors again.
Fixesgolang/go#7158.
LGTM=adonovan
R=adonovan
CC=golang-codereviews
https://golang.org/cl/83640043
This avoids confusion when trying to read correctly
encoded export data that happens to be encoded in
a different format (debug vs product).
LGTM=adonovan
R=adonovan
CC=golang-codereviews
https://golang.org/cl/82090043
Existing tools use the default value of zero; their behaviour is unchanged.
(*Config).ParseFile is used only from tests.
LGTM=crawshaw, rsc, gri
R=crawshaw, gri, rsc
CC=golang-codereviews
https://golang.org/cl/79290044
Also, add loader.Config.DisplayPath hook, which allows the
filename returned by build.Context.Import() to be transformed
prior to attaching to the AST. This allows a virtual file
system to be used without leaking into the user interface.
Eliminate parsePackageFiles hook; I don't think we need it any
more. The test that was using it has been rewritten to use
the build.Context hooks.
LGTM=gri
R=gri, crawshaw
CC=daniel.morsing, golang-codereviews, rsc
https://golang.org/cl/75520046
With this CL, an Object.Parent() Scope is always the scope in
which the object was originally declared. For dot-imported
objects, that is the package scope of the package from which
the objects are imported (not the file scope into which they
are imported).
Also:
- Changed Scope.Insert to be agnostic regarding blank
identifiers - blank identifiers must be handled outside.
- Fixed handling of blank labels: they are never declared.
Fixesgolang/go#7537.
LGTM=adonovan
R=adonovan
CC=golang-codereviews
https://golang.org/cl/75570043
e.g. file foo.go scope {...}
package math scope {...}
function f scope {...}
if scope {...}
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/74320043
This change renumbers nodes so that addressable ones
(that may appear in a points-to set) all have lower
numbers than non-addressable ones----initially at least:
reflection, SetFinalizer, etc add new nodes during
solving.
This improves the efficiency of sparse PTS
representations (to be added later). The largest int in
a PTS is now about 20% of the previous max.
Overview:
- move constraint stuff into constraint.go.
- add two methods to constraint:
(1) renumber(): renumbers all nodeids. The
implementations are very repetitive but simple. I
thought hard about other ways (mixins, reflection)
but decided this one was fine.
(2) indirect(): report the set of nodeids whose
points-to relations depend on the solver, not just
the initial constraint graph.
(This method is currently unused and is logically
part of a forthcoming change to implement PE/LE
presolver optimizations. (Perhaps I should comment
it out/remove it for now.)
- split up the population of the intrinsics map by file.
- delete analysis.probes (unused field)
- remove state="..." from panic message; unnecessary.
LGTM=crawshaw
R=crawshaw
CC=golang-codereviews
https://golang.org/cl/73320043
Now that go/types permits files to be added to a package
incrementally (fixing bug 7114), this CL extends the loader
to load and augment multiple test packages at once.
TESTED:
- go/loader/stdlib_test runs the type-checker on the entire
standard library loaded from source in a single gulp, with
each package augmented by tests.
- Manually tested on:
% ssadump -test -run unicode encoding/ascii85
Both sets of tests are run (and pass).
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/61060043
With this CL, it is now possible to type-check additional
package files to an already type-checked package through
repeated calls to Checker.Files.
Fixesgolang/go#7114.
LGTM=adonovan
R=adonovan
CC=golang-codereviews
https://golang.org/cl/72730043
Provide an extra field, "Soft" in types.Error.
A client may want to filter out soft errors and
only abort if there are "hard" errors.
Qualified a first set of errors as "soft".
Fixesgolang/go#7360.
LGTM=adonovan
R=adonovan
CC=golang-codereviews
https://golang.org/cl/71800043
Also:
- slightly better error messages for return statements
- more AST validity checking of parameter lists
- use secondary error message for clarifying message in type switch errors
Fixesgolang/go#7469.
LGTM=adonovan
R=adonovan
CC=golang-codereviews
https://golang.org/cl/71780044
The parser may return error positions referring to positions
immediately after rather than at a token position. Provide
mechanism to identify those positions in test cases.
Also: Don't compute position strings for each token in test
cases. Should speed up Check test.
Pending CL 70190046 in main repo.
LGTM=adonovan
R=adonovan
CC=golang-codereviews
https://golang.org/cl/71330043
(On Windows the two are not the same.)
Fixesgolang/go#7189.
Also: remove exception for code.google.com subtree; this was
an artifact of my unusual setup.
LGTM=gri
R=alex.brainman, gri
CC=golang-codereviews
https://golang.org/cl/70060048
Before, they were named func@line:col which made them easy to find in the source if you know the file, but hard if you don't, and it made tests fragile.
Now, they are named outer$1, outer$2, etc, which makes them
more informative in a UI since "outer" has meaning.
LGTM=crawshaw
R=crawshaw
CC=golang-codereviews
https://golang.org/cl/65630048
This is to avoid an internal error in pointer analysis from
bringing down a long-lived application such as godoc.
LGTM=crawshaw
R=crawshaw
CC=golang-codereviews
https://golang.org/cl/68930046
An identifier X in anonymous struct field struct{X} is both a
definition of a field (*Var) and reference to a type
(*TypeName). Now that we have split the map, we can capture
both of these aspects.
Interestingly, every client but one was going to extra effort
to iterate over just the uses or just the defs; this
simplifies them.
Also, fix two bug related to tagless switches:
- An entry was being recorded in the Object map for a piece of
synthetic syntax.
- The "true" identifier was being looked up in the current scope,
which allowed perverse users to locally redefine it. Now
we use the bool (not untyped boolean) constant true, per the
consequent clarification of the spec (issue 7404).
+ tests.
Fixesgolang/go#7276
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/68270044
If a -pos argument is specified, a 'callgraph' query reports only the
functions within the query package. This produces a far more manageable
amount of information, and because we don't need to package-qualify the
names, the result is easier to read.
Added tests:
- callgraph query with/without -pos
(The test driver was extended to allow "nopos" queries.)
- callers and callees queries don't return wrappers
Also, in go/callgraph:
- (*Node).String, (*Edge).String
- (*Graph).DeleteSyntheticNodes eliminates synthetic wrapper functions,
preserving topology. Used in all four oracle "call*" queries.
- (*Graph).DeleteNode
LGTM=crawshaw
R=crawshaw
CC=golang-codereviews
https://golang.org/cl/66240044
- clearer separation between package-level and object-level state
- lazy allocation of various package-level maps
- NewPackage automatically allocates a package scope
- steps towards enabling incremental checking of extra (test) files
after the (non-test) package is type-checked (not yet exposed)
LGTM=adonovan
R=adonovan
CC=golang-codereviews
https://golang.org/cl/66230044
1) We remove context sensitivity from API. The pointer analysis is
not sufficiently context-sensitive for the context information to
be worth exposing. (The actual analysis precision still benefits
from being context-sensitive, though.) Since all clients would
discard the context info, we now do that for them.
2) Make the graph doubly-linked. Edges are now shared by the Nodes
at both ends of the edge so it's possible to navigate more easily
(e.g. to the callers).
3) Graph and Node are now concrete, not interfaces.
Less code in every file!
LGTM=crawshaw
R=crawshaw
CC=golang-codereviews
https://golang.org/cl/66460043
Previously, each {Indirect,}Query would return a set of Pointers, one per context; now it returns (at most) one Pointer combining information from all contexts.
The old API was more faithful to the implementation concepts, but the analysis is not sufficiently context-sensitive that it makes sense: all existing clients simply throw away the context information---so now we do that for them.
(I may remove the context-sensitivity from the callgraph too, but I'll benchmark that first to see if it reduces precision.)
LGTM=crawshaw
R=crawshaw
CC=golang-codereviews
https://golang.org/cl/66130044
- pre-poluate typIndex and typList using a predefined list of types
- no need to handle basic types explicitly anymore
- removed basicTag
- go/types: exported UniverseByte and UniverseRune for now
LGTM=adonovan
R=adonovan
CC=cmang, golang-codereviews
https://golang.org/cl/65920044
If this flag is set, (*Config).Load will not return an
error even if some packages had type errors. Each individual
PackageInfo can be queried for its error state, now exposed as
TypeError.
In addition, each PackageInfo exposes whether it is
"transitively error-free". ssa.Create skips packages without
this flag since it is required for SSA construction.
+ Test.
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/62000045
Observation: not all alias facts are interesting.
- A channel-peers query also cares about pointers of kind chan.
- An oracle "points-to" query on an expression of kind map
only cares about maps.
- We always care about func, interface and reflect.Value,
since they're needed for sound analysis of dynamic dispatch.
We needn't bother collecting alias information for
uninteresting pointers, and this massively reduces the number
of labels flowing in to the constraint system.
The only constraints that create new labels are addressOf
and offsetAddr; both are now selectively emitted by type.
We compute the set of type kinds to track, based on the
{Indirect,}Query types. (We could enable tracking at an
even finer grain if we want.)
This requires that we can see all the {Indirect,}Query
value types a priori, which is not the case for the PrintCalls
mechanism used in the tests, so I have rewritten the latter
to use {Indirect,}Query instead.
This reduces the solver-phase time for the entire standard
library and tests from >8m to <2m. Similar speedups are
obtained on small and medium-sized programs.
Details:
- shouldTrack inspects the flattened form of a type to see if
it contains fields we must track. It memoizes the result.
- added precondition checks to (*Config).Add{,Indirect}Query.
- added (*ssa.Program).LookupMethod convenience method.
- added Example of how to use the Query mechanism.
- removed code made dead by a recent invariant:
the only pointerlike Const value is nil.
- don't generate constraints for any functions in "reflect".
(we had forgotten to skip synthetic wrappers too).
- write PTA warnings to the log.
- add annotations for more intrinsics.
LGTM=gri, crawshaw
R=crawshaw, gri
CC=golang-codereviews
https://golang.org/cl/62540043
Method Signatures (types.Signatures with a non-nil receiver) behave
like ordinary func Signatures in the hash function/equivalence relation
used by typemap.M, which leads to surprising incomplete traversal
over the type graph if an M is used to remember types already
visited.
This change avoids ever putting method Signatures in a typemap.
% go test -v code.google.com/p/go.tools/go/ssa
now repeatedly shows the exact same number of functions and
instructions.
(We should discuss how to avoid this problem more generally.)
Also:
- recur over the params/results of all the methods of
each type when computing necessary method sets.
- there's no longer any need to treat declarations of unexported
methods as a root for traversal. Added test case.
- enable SourceImports flag in stdlib_test, which was dropped
during a recent refactoring (d'oh).
- doc tweaks
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/65450043
This CL adds no-op stubs for intrinsics now required by tests:
runtime.Goexit
sync.runtime_Sem{acquire,release}
sync/atomic.AddUint{32,64}
Goexit needs more thought; for now I've disabled the interpreted
tests of the "testing" package to make the build green again.
TBR=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/65480044
- Add math.{Log,Ldexp} to externals
- Test now uses FromArgs
- Scan all initial packages for tests, don't assume Created[0] exists.
(It doesn't if we import a package that has only in-package tests.)
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/62010044
Previously, each word could be a package import path or a
comma-separated list of *.go file names. Now, if the
first word ends with ".go", all words are assumed to be
Go source files. This makes it impossible to specify
two ad-hoc packages from source files, but no-one needs that.
FromArgs also takes a boolean indicating whether tests
are wanted or not.
Also: ssadump: add -test flag to set that boolean.
For the oracle it's always true.
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/61470047
Method-set caching is now performed externally using a MethodSetCache (if desired), not by the Types themselves.
This a minor deoptimization due to the extra maps, but avoids a situation in which method-sets are computed and frozen prematurely. (See b/7114)
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/61430045
Move mutation of program to the final step.
+ 2 assertions;
+ switches.go: comment fix.
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/61510044
The "static bool" parameter to Implements was confusing; typically we think about interface implementation and type assertion as separate but related concepts, but here they were merged.
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/58540044
- Centralize "visited" check in check.objDecl.
- Assert that object-specific declX functions
are only called with objects that have no
type associated with them, yet.
- Added test case.
Thanks to Richard Musiol (neelance@gmail.com) for
finding the bug and providing an easy test case.
For a discussion of the bug see the issue.
Fixesgolang/go#7245.
TBR=adonovan
R=adonovan
CC=golang-codereviews
https://golang.org/cl/59210043
- consistently set underlying types of incoming named types in typInternal
- use underlying() helper function to resolve forward chains
- related consistency cleanups
LGTM=adonovan
R=adonovan
CC=golang-codereviews
https://golang.org/cl/53870044
This is (a) more efficient and (b) avoids the need for
constant error handling, since buffer writes can't fail.
Also:
- added WriteFunction and WritePackage functions,
similar to types.WriteFoo.
- *Function and *Package now implement io.WriterTo.
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/57930044
This results in significant improvement to type-checking time:
it reduces by 4% the entire running time of ssa/stdlib_test
(GOMAXPROCS=8, n=7).
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/57770043
Since Put() makes a dynamic function call, this CL includes a long-overdue change to supply a *frame (and thus the call
stack and the interpreter) to intrinsics too.
This fixes the tests that were broken by (sound) revision e4a4cb47c141.
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/57350043
CL 49530047 made the (over-)simplifying assumption that the
Path of an ad-hoc (Created) package should default to its
Name, i.e. package declaration.
With this change, the Name is still always computed from the
package declaration (by go/types) but the Path may be
specified by the loader.Config. If "", the value of the Name
is used, which is not globally unique.
R=gri, axwalk
CC=golang-codereviews
https://golang.org/cl/55180043
Was: Now:
call.Graph callgraph.Graph
call.GraphNode callgraph.Node
call.Edge callgraph.Edge
Though call.Graph was cute, the original naming was a mistake:
'call' is too useful a var name to waste on a package.
R=gri, crawshaw
CC=golang-codereviews
https://golang.org/cl/53190043
The goal is to allow clients to distinguish function-local objects
from package-local objects and identify the specific function by
examining its scope. For example, a compiler may need to mangle
object names according to the outer function name.
R=gri
CC=golang-codereviews, golang-dev
https://golang.org/cl/52280043
Both gc and gccgo appear to use this size, so it seems appropriate
to use it here.
R=gri
CC=golang-codereviews, golang-dev, iant, rsc
https://golang.org/cl/52290043
Alan Donovan
Robert Griesemer
Peter Collingbourne
Rob Pike
Andrew Wilkins
Gordon Klaus
Alex Brainman