Packages were not being created for all types.Packages,
specifically, indirectly imported packages were missing.
(*Program).CreatePackages now iterates over the type-checker's
package map too.
Also: removed all concurrency from importer. I think it was
incorrect (and hard to fix).
Also: change LoadInitialPackages so that all named packages
are loaded from source. This happens regardless of whether
GCImporter is used to satisfy imports.
Details:
- importer.Config.SourceImports flag determines whether to
load all packages from *.go source.
(Before, this was indicated by Config.Build != nil.)
- importer.Config.Build field effectively defaults to
&go/build.Default. A zero importer.Config is now usable.
- importer.Importer.Config field is now exported.
- LoadPackage renamed to ImportPackage since the resulting
packages may come from GCImporter (and be incomplete).
- doImport and ImportPackage fused.
Fixesgolang/go#7028
R=gri, axwalk
CC=golang-codereviews
https://golang.org/cl/48770043
(*CallCommon).Signature() now returns non-nil even for
built-ins. Builtins are now created with specialized types for
each use. Added sanity-check.
CallCommon.HasEllipsis field eliminated. It was an incorrect
memoization of Signature().IsVariadic() used only for
printing.
Also: introduce and use newTypeVar utility.
R=gri
CC=golang-codereviews
https://golang.org/cl/46880044
Although such a control-flow path is impossible by
construction, we must avoid no-arg returns (even if
unreachable) in non-void functions.
Tested by adding a sanity-check that ssa.Return has the correct arity.
Also: added ssautil.Switches() test.
Fixesgolang/go#7702
R=gri, axwalk
CC=golang-codereviews
https://golang.org/cl/46520043
Cause: emitExtract requires a type for each component of the
receive tuple; blank supplies no such type.
Solution: remove type parameter for emitExtract as it is no
longer needed: since rev b75cc03b4a56 it is always identical
to the tuple.Type().At(index).
+ tests.
Fixesgolang/go#6806.
R=gri, gri
CC=axwalk, golang-dev
https://golang.org/cl/30410043
Added sanity check to ensure Operands/Referrers are complete and dual.
Also: unexport Instruction.setBlock (=> no longer user-implementable).
R=gri
CC=golang-dev
https://golang.org/cl/22150043
Added test for []*map composite literals containing nested
literal subelements. This required implementing
(reflect.Value).Map{Keys,Index} in ssa/interp.
Plus two minor fixes in ssa/interp.
R=gri
CC=golang-dev
https://golang.org/cl/20470043
A DebugRef associates a source expression E with an ssa.Value
V, but until now did not record whether V was the value or the
address of E. So, we would guess from the "pointerness" of
the Value, leading to confusion in some cases, e.g.
type N *N
var n N
n = &n // lvalue and rvalue are both pointers
Now we explicitly record 'IsAddress bool' in DebugRef, and
plumb this everywhere: through (*Function).ValueForExpr and
(*Program).VarValue, all the way to forming the pointer
analysis query.
Also:
- VarValue now treats each reference to a global distinctly,
just like it does for other vars. So:
var g int
func f() {
g = 1 // VarValue(g) == Const(1:int), !isAddress
print(g) // VarValue(g) == Global(g), isAddress
}
- DebugRefs are not emitted for references to predeclared
identifiers (nil, built-in).
- DebugRefs no longer prevent lifting of an Alloc var into a
register; now we update or discard the debug info.
- TestValueForExpr: improve coverage of ssa.EnclosingFunction
by putting expectations in methods and init funcs, not just
normal funcs.
- oracle: fix golden file broken by recent
(*types.Var).IsField change.
R=gri
CC=golang-dev
https://golang.org/cl/16610045
This allows us to run/analyze multiple tests.
Also it causes the production code packages to be properly initialized.
Also:
- cmd/ssadump: improved usage message (add example;
incorporate LoadInitialPackages usage; explain how -run
finds main).
- pointer, oracle, ssa/interp: use CreateTestMainPackage.
- ssa/builder.go: remove 'rundefers' instruction from package init,
which no longer uses 'defer'.
R=gri
CC=golang-dev
https://golang.org/cl/15920047
Motivation:
Previously, we assumed that the set of types for which a
complete method set (containing all synthesized wrapper
functions) is required at runtime was the set of types
used as operands to some *ssa.MakeInterface instruction.
In fact, this is an underapproximation because types can
be derived from other ones via reflection, and some of
these may need methods. The reflect.Type API allows *T to
be derived from T, and these may have different method
sets. Reflection also allows almost any subcomponent of a
type to be accessed (with one exception: given T, defined
'type T struct{S}', you can reach S but not struct{S}).
As a result, the pointer analysis was unable to generate
all necessary constraints before running the solver,
causing a crash when reflection derives types whose
methods are unavailable. (A similar problem would afflict
an ahead-of-time compiler based on ssa. The ssa/interp
interpreter was immune only because it does not require
all wrapper methods to be created before execution
begins.)
Description:
This change causes the SSA builder to record, for each
package, the set of all types with non-empty method sets that
are referenced within that package. This set is accessed via
Packages.TypesWithMethodSets(). Program.TypesWithMethodSets()
returns its union across all packages.
The set of references that matter are:
- types of operands to some MakeInterface instruction (as before)
- types of all exported package members
- all subcomponents of the above, recursively.
This is a conservative approximation to the set of types
whose methods may be called dynamically.
We define the owning package of a type as follows:
- the owner of a named type is the package in which it is defined;
- the owner of a pointer-to-named type is the owner of that named type;
- the owner of all other types is nil.
A package must include the method sets for all types that it
owns, and all subcomponents of that type that are not owned by
another package, recursively. Types with an owner appear in
exactly one package; types with no owner (such as struct{T})
may appear within multiple packages.
(A typical Go compiler would emit multiple copies of these
methods as weak symbols; a typical linker would eliminate
duplicates.)
Also:
- go/types/typemap: implement hash function for *Tuple.
- pointer: generate nodes/constraints for all of
ssa.Program.TypesWithMethodSets().
Add rtti.go regression test.
- Add API test of Package.TypesWithMethodSets().
- Set Function.Pkg to nil (again) for wrapper functions,
since these may be shared by many packages.
- Remove a redundant logging statement.
- Document that ssa CREATE phase is in fact sequential.
Fixesgolang/go#6605
R=gri
CC=golang-dev
https://golang.org/cl/14920056
A function such as this:
func one() (x int) {
defer func() { recover() }()
x = 1
panic("return")
}
that combines named return parameters (NRPs) with deferred calls
that call recover, may return non-zero values despite the
fact it doesn't even contain a return statement. (!)
This requires a change to the SSA API: all functions'
control-flow graphs now have a second entry point, called
Recover, which is the block at which control flow resumes
after a recovered panic. The Recover block simply loads the
NRPs and returns them.
As an optimization, most functions don't need a Recover block,
so it is omitted. In fact it is only needed for functions that
have NRPs and defer a call to another function that _may_ call
recover.
Dataflow analysis of SSA now requires extra work, since every
may-panic instruction has an implicit control-flow edge to
the Recover block. The only dataflow analysis so far implemented
is SSA renaming, for which we make the following simplifying
assumption: the Recover block only loads the NRPs and returns.
This means we don't really need to analyze it, we can just
skip the "lifting" of such NRPs. We also special-case the Recover
block in the dominance computation.
Rejected alternative approaches:
- Specifying a Recover block for every defer instruction (like a
traditional exception handler).
This seemed like excessive generality, since Go programs
only need the same degenerate form of Recover block.
- Adding an instruction to set the Recover block immediately
after the named return values are set up, so that dominance
can be computed without special-casing.
This didn't seem worth the effort.
Interpreter:
- This CL completely reimplements the panic/recover/
defer logic in the interpreter. It's clearer and simpler
and closer to the model in the spec.
- Some runtime panic messages have been changed to be closer
to gc's, since tests depend on it.
- The interpreter now requires that the runtime.runtimeError
type be part of the SSA program. This requires that clients
import this package prior to invoking the interpreter.
This in turn requires (Importer).ImportPackage(path string),
which this CL adds.
- All $GOROOT/test/recover{,1,2,3}.go tests are now passing.
NB, the bug described in coverage.go (defer/recover in a concatenated
init function) remains. Will be fixed in a follow-up.
Fixesgolang/go#6381
R=gri
CC=crawshaw, golang-dev
https://golang.org/cl/13844043
Before, we would concatenate all the init() blocks together,
resulting in incorrect treatment of a recovered panic in one
init block: the implicit return would cause the subsequent ones
to be skipped.
The result is simpler, and closer to what gc does.
The additional functions are visible in the call graph,
so some tests required updating.
R=gri
CC=crawshaw, golang-dev
https://golang.org/cl/14671044
- removed support for nil constants from go/exact
- instead define a singleton Nil Object (the nil _value_)
- in assignments, follow more closely spec wording
(pending spec CL 14415043)
- removed use of goto in checker.unary
- cleanup around handling of isRepresentable for
constants, with better error messages
- fix missing checks in checker.convertUntyped
- added isTyped (== !isUntyped) and isInterface predicates
- fixed hasNil predicate: unsafe.Pointer also has nil
- adjusted ssa per adonovan
- implememted types.Implements (wrapper arounfd types.MissingMethod)
- use types.Implements in vet (and fix a bug)
R=adonovan, r
CC=golang-dev
https://golang.org/cl/14438052
Motivation: pointer analysis tools (like the oracle) want the
user to specify a set of initial packages, like 'go test'.
This change enables the user to specify a set of packages on
the command line using importer.LoadInitialPackages(args).
Each argument is interpreted as either:
- a comma-separated list of *.go source files together
comprising one non-importable ad-hoc package.
e.g. "src/pkg/net/http/triv.go" gives us [main].
- an import path, denoting both the imported package
and its non-importable external test package, if any.
e.g. "fmt" gives us [fmt, fmt_test].
Current type-checker limitations mean that only the first
import path may contribute tests: multiple packages augmented
by *_test.go files could create import cycles, which 'go test'
avoids by building a separate executable for each one.
That approach is less attractive for static analysis.
Details: (many files touched, but importer.go is the crux)
importer:
- PackageInfo.Importable boolean indicates whether
package is importable.
- un-expose Importer.Packages; expose AllPackages() instead.
- CreatePackageFromArgs has become LoadInitialPackages.
- imports() moved to util.go, renamed importsOf().
- InitialPackagesUsage usage message exported to clients.
- the package name for ad-hoc packages now comes from the
'package' decl, not "main".
ssa.Program:
- added CreatePackages() method
- PackagesByPath un-exposed, renamed 'imported'.
- expose AllPackages and ImportedPackage accessors.
oracle:
- describe: explain and workaround a go/types bug.
Misc:
- Removed various unnecessary error.Error() calls in Printf args.
R=crawshaw
CC=golang-dev
https://golang.org/cl/13579043
Before, VarValue looked for the ssa.Value for the 'var' object
in the same package as the object was defined, but this is
(obviously) wrong for a cross-package FieldVal selection,
expr.f. The caller must provide the package containing the
reference.
+ test.
Also:
- add 2 TODOs.
- split builder.expr into two functions so we don't need
defer, which makes panic dumps harder to read.
R=golang-dev, crawshaw
CC=golang-dev
https://golang.org/cl/13257045
Previously, if the result was not wanted, the received
(value, ok) tuple had no type for 'value'.
Now it is always set to the channel's element type.
Also: set the position on such receive instructions to that of
the = or := token, and document it.
+ (indirect) test via pointer analysis.
R=crawshaw, gri
CC=golang-dev
https://golang.org/cl/12956052
Map literals should use the same recursion logic as
struct/array/slice literals to apply an implicit &-operator to
the nested literals when a pointer is wanted.
+ test.
Also:
- ensure we set the source location for all Lookup and
MapUpdate instructions.
- remove obsolete address.object field.
R=gri, crawshaw
CC=golang-dev
https://golang.org/cl/12787048
&x.f, &x[0], x[i:j], &*x all must panic if x==nil.
The first three are already addressed by the semantics of
FieldAddr, IndexAddr, Slice; updated docs to reflect this.
The final case requires generation of an additional dynamic check.
See golang.org/s/go12nil for details.
Tested on $GOROOT/test/nilptr2.go (with patch from CL 13108043)
Also: remove a TODO where a no-op will do.
R=gri, crawshaw
CC=golang-dev, rsc
https://golang.org/cl/13064044
Remove its 'name' field and treat it just like any other
ssa.Register: it gets a temp name like "t1".
Instead, give it a comment field holding its purpose, e.g, "x"
for a source-level vare, or "new", "slicelit", "complit" or
"varargs".
This improves usability of tools whose UI needs to refer to a
particular allocation site.
R=gri
CC=golang-dev
https://golang.org/cl/12273043
The required changes were surprisingly minimal (I was hoping for a net
code deletion) because ssa is essentially doing its own type inference
for the three value,ok operators---and it continues to have to do so.
To see why, consider:
var i interface{}; var ok bool
i, ok := (map[string]string)(nil)[""]
Before, go/types inferred interface{} for the RHS of the assignment,
and now it infers (interface{}, bool), yet neither of these is what
ssa needs: it needs to know that the map values were strings so
that it can emit a lookup of the right type followed by a conversion
from string to interface{}.
TBR=gri
CC=golang-dev
https://golang.org/cl/11988044
stdlib_test runs the builder (in sanity-checking mode) over
the Go standard library. It also prints some stats about
the time and memory usage.
Also:
- importer.LoadPackage too (not just doImport) must consult
the cache to avoid creating duplicate Package instances for
the same import path when called serially from a test.
- importer: skip empty directories without an error.
- importer: print all errors, not just the first.
- visit.go: added AllFunctions utility for enumerating all
Functions in a Program.
- ssa.MethodSet is not safe to expose from the package since
it must be accessed under an (inaccessible) lock. (!!!)
This CL makes it unexported and restricts its use to the
single function Program.LookupMethod().
- Program.MethodSet() has gone.
Clients should instead iterate over the types.MethodSet
and call LookupMethod.
- Package.DumpTo(): improved efficiency of methodset printing
(by not creating wrappers) and accuracy (by showing * on
receiver type only when necessary).
- Program.CreatePackage: documented precondition and added
assertion.
R=gri
CC=golang-dev
https://golang.org/cl/12058048
Also:
- Implement Program.FuncValue for interface methods (+ test).
- go/types.Object.String(): don't package-qualify names unless
they are package level objects---otherwise you see "main.x" for
locals, struct fields, etc.
- go/types.Func.String(): don't assume Type() is *Signature;
it could be *Builtin.
R=gri
CC=golang-dev
https://golang.org/cl/12058045
buildDecl was visiting all decls in source order, but the spec
calls for visiting all vars and init() funcs in order, then
all remaining functions. These two passes are now called
buildInit(), buildFuncDecl().
+ Test.
Also:
- Added workaround to gcimporter for Func with pkg==nil.
- Prog.concreteMethods has been merged into Pkg.values.
- Prog.concreteMethod() renamed declaredFunc().
- s/mfunc/obj/ (name cleanup from recent gri CL)
R=gri
CC=golang-dev
https://golang.org/cl/12030044
A Method corresponds to a MethodVal Selection;
so the explicit Method object is not needed anymore.
- moved Selection code into separate file
- implemented Selection.String()
- improved and more consistent documentation
R=adonovan
CC=golang-dev
https://golang.org/cl/11950043
methprom.go covers method promotion.
Found bug: receiver() requires a following load under some
circumstances.
ifaceconv.go covers interface conversion.
Found bug: confusion about infallible and fallible conversions
led to use of TypeAssert in emitConv, which should never fail.
Changed semantics of ChangeInterface to make it infallible
and made some simplifications.
Also in this CL:
- SelectState.Pos now records the position of the
the '<-' operator for sends/receives done by a Select.
R=gri
CC=golang-dev
https://golang.org/cl/11931044
We now use LookupFieldOrMethod for all SelectorExprs, and
simplify the logic to discriminate the various cases.
We inline static calls to promoted/indirected functions,
dramatically reducing the number of functions created.
More tests are needed, but I'd like to submit this as-is.
In this CL, we:
- rely less on Id strings. Internally we now use
*types.Method (and its components) almost everywhere.
- stop thinking of types.Methods as objects. They don't
have stable identities. (Hopefully they will become
plain-old structs soon.)
- eliminate receiver indirection wrappers:
indirection and promotion are handled together by makeWrapper.
- Handle the interactions of promotion, indirection and
abstract methods much more cleanly.
- support receiver-bound interface method closures.
- break up builder.selectField so we can re-use parts
(emitFieldSelection).
- add importer.PackageInfo.classifySelector utility.
- delete interfaceMethodIndex()
- delete namedTypeMethodIndex()
- delete isSuperInterface() (replaced by types.IsAssignable)
- call memberFromObject on each declared concrete method's
*types.Func, not on every Method frem each method set, in the
CREATE phase for packages loaded by gcimporter.
go/types:
- document Func, Signature.Recv() better.
- use fmt in {Package,Label}.String
- reimplement Func.String to be prettier and to include method
receivers.
API changes:
- Function.method now holds the types.Method (soon to be
not-an-object) for synthetic wrappers.
- CallCommon.Method now contains an abstract (interface)
method object; was an abstract method index.
- CallCommon.MethodId() gone.
- Program.LookupMethod now takes a *Method not an Id string.
R=gri
CC=golang-dev
https://golang.org/cl/11674043
Now, in a "switch y := x.(type)", there is no object for the
outer y, only implicit objects, one per case (including
default).
Also: don't set obj=nil for blank idents (workaround suggested by gri).
R=gri
CC=golang-dev
https://golang.org/cl/11564046
- implemented objset for tracking duplicates of fields and methods
which permitted a simpler and faster scope implementation in turn
- related cleanups and internal renames
- fixed a couple of identifier reporting bugs
Speed of type-checking itself increased by almost 10%
(from ~71Kloc/s to ~78Kloc/s on one machine, measured
via go test -run=Self).
R=adonovan
CC=golang-dev
https://golang.org/cl/11750043
Reduces by 94% the number of wrappers created during the tests.
(No appreciable difference in running time, sadly.)
R=gri
CC=golang-dev
https://golang.org/cl/11619043
Details:
- emitImplicitSelections now emits common code for implicit
field selections in both method and field lookups.
The last iteration over the LookupFieldOrMethod indices---the explicit,
final index---is handled by the caller.
- anonFieldPath, candidate and the BFS algo in buildMethodSet are all gone.
R=gri
CC=golang-dev
https://golang.org/cl/11576045
ssa:
- Prog.CreatePackages inlined into all callers.
- Prog.CreatePackage is now exposed; idempotent; and checks for errors.
- '*address' not 'address' now implements lvalue (since it's 6 words).
- removed types.Method case from createMemberFromObject.
importer:
- added importer.PackageInfo.String method.
- simplifed importer.PackageInfo by putting types.Info in it.
- removed obsolete precondition from IsType.
R=gri
CC=golang-dev
https://golang.org/cl/11408045
(Motivation: "Literal" is a syntactic property, not a semantic one.)
Also: delete a "TODO: opt" that the lifting pass already does for us.
R=gri
CC=golang-dev
https://golang.org/cl/11351043
- removed a number of obsolete TODO(gri) comments.
- bring ssa.DefaultType back into sync with types.defaultType.
- re-enable types.Package.Path()!="" assertion.
- use Path() (not reflect pointer) in sort routine.
- make interp.checkInterface use types.MissingMethod.
- un-export ssa.MakeId function.
- inline pointer() into all callers, and delete.
- enable two more interp_tests: $GOROOT/test/{method3,cmp}.go
- add links to bugs to other interp_tests.
- add runtime.NumCPU to ssa/interp/externals.go
R=gri
CC=golang-dev
https://golang.org/cl/11353043
Alan Donovan
Robert Griesemer