close_issue_1
/
tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: close_issue_1:master
Branches Tags
close_issue_1:master
close_issue_1:release-branch.go1.12
close_issue_1:release-branch.go1.11
close_issue_1:release-branch.go1.10
close_issue_1:release-branch.go1.8
close_issue_1:release-branch.go1.9
close_issue_1:release-branch.go1.7
close_issue_1:release-branch.go1.6
close_issue_1:release-branch.go1.5
close_issue_1:release-branch.go1.4
close_issue_1:release-branch.go1.3
close_issue_1:release-branch.go1.2
close_issue_1:gopls/v0.1.3
close_issue_1:gopls/v0.1.2
close_issue_1:gopls/v0.1.1
close_issue_1:gopls/v0.1.0
..
compare: close_issue_1:release-branch.go1.6
Branches Tags
close_issue_1:master
close_issue_1:release-branch.go1.12
close_issue_1:release-branch.go1.11
close_issue_1:release-branch.go1.10
close_issue_1:release-branch.go1.8
close_issue_1:release-branch.go1.9
close_issue_1:release-branch.go1.7
close_issue_1:release-branch.go1.6
close_issue_1:release-branch.go1.5
close_issue_1:release-branch.go1.4
close_issue_1:release-branch.go1.3
close_issue_1:release-branch.go1.2
close_issue_1:gopls/v0.1.3
close_issue_1:gopls/v0.1.2
close_issue_1:gopls/v0.1.1
close_issue_1:gopls/v0.1.0

2 Commits
master ... release-br

Author SHA1 Message Date
Chris Broadfoot c887be1b2e [release-branch.go1.6] all: merge master into release-branch.go1.6 
Change-Id: I392f9307c9f9012fd5cebc0013debcee6526b762
 2016-02-03 15:20:29 -08:00
Brad Fitzpatrick 03f58a3ccf all: skip slow tests in short mode 
Updates golang/go#14113
Updates golang/go#11811

Change-Id: I61851de12ff474d3b738fc88f402742677973cae
 2016-01-28 00:09:23 +00:00
1286 changed files with 94240 additions and 117821 deletions
Show Stats Expand all files Collapse all files

15
CONTRIBUTING.md
View File

@ -4,15 +4,16 @@ Go is an open source project.
It is the work of hundreds of contributors. We appreciate your help!
## Filing issues
When [filing an issue](https://golang.org/issue/new), make sure to answer these five questions:
1. What version of Go are you using (`go version`)?
2. What operating system and processor architecture are you using?
3. What did you do?
4. What did you expect to see?
5. What did you see instead?
1. What version of Go are you using (`go version`)?
2. What operating system and processor architecture are you using?
3. What did you do?
4. What did you expect to see?
5. What did you see instead?
General questions should go to the [golang-nuts mailing list](https://groups.google.com/group/golang-nuts) instead of the issue tracker.
The gophers there will answer or ask you to file an issue if you've tripped over a bug.
@ -22,5 +23,9 @@ The gophers there will answer or ask you to file an issue if you've tripped over
Please read the [Contribution Guidelines](https://golang.org/doc/contribute.html)
before sending patches.
**We do not accept GitHub pull requests**
(we use [Gerrit](https://code.google.com/p/gerrit/) instead for code review).
Unless otherwise noted, the Go source files are distributed under
the BSD-style license found in the LICENSE file.

10
README Normal file
View File

@ -0,0 +1,10 @@
This subrepository holds the source for various packages and tools that support
the Go programming language.
Some of the tools, godoc and vet for example, are included in binary Go distributions.
Others, including the Go oracle and the test coverage tool, can be fetched with "go get".
Packages include a type-checker for Go and an implementation of the
Static Single Assignment form (SSA) representation for Go programs.
To submit changes to this repository, see http://golang.org/doc/contribute.html.

27
README.md
View File

@ -1,27 +0,0 @@
# Go Tools
This subrepository holds the source for various packages and tools that support
the Go programming language.
Some of the tools, `godoc` and `vet` for example, are included in binary Go
distributions.
Others, including the Go `guru` and the test coverage tool, can be fetched with
`go get`.
Packages include a type-checker for Go and an implementation of the
Static Single Assignment form (SSA) representation for Go programs.
## Download/Install
The easiest way to install is to run `go get -u golang.org/x/tools/...`. You can
also manually git clone the repository to `$GOPATH/src/golang.org/x/tools`.
## Report Issues / Send Patches
This repository uses Gerrit for code changes. To learn how to submit changes to
this repository, see https://golang.org/doc/contribute.html.
The main issue tracker for the tools repository is located at
https://github.com/golang/go/issues. Prefix your issue with "x/tools/(your
subdir):" in the subject line, so it is easy to find.

8
blog/atom/atom.go
View File

@ -34,12 +34,8 @@ type Entry struct {
}
type Link struct {
Rel string `xml:"rel,attr,omitempty"`
Href string `xml:"href,attr"`
Type string `xml:"type,attr,omitempty"`
HrefLang string `xml:"hreflang,attr,omitempty"`
Title string `xml:"title,attr,omitempty"`
Length uint `xml:"length,attr,omitempty"`
Rel string `xml:"rel,attr"`
Href string `xml:"href,attr"`
}
type Person struct {

44
blog/blog.go
View File

@ -24,14 +24,7 @@ import (
"golang.org/x/tools/present"
)
var (
validJSONPFunc = regexp.MustCompile(`(?i)^[a-z_][a-z0-9_.]*$`)
// used to serve relative paths when ServeLocalLinks is enabled.
golangOrgAbsLinkReplacer = strings.NewReplacer(
`href="https://golang.org/pkg`, `href="/pkg`,
`href="https://golang.org/cmd`, `href="/cmd`,
)
)
var validJSONPFunc = regexp.MustCompile(`(?i)^[a-z_][a-z0-9_.]*$`)
// Config specifies Server configuration values.
type Config struct {
@ -47,8 +40,7 @@ type Config struct {
FeedArticles int // Articles to include in Atom and JSON feeds.
FeedTitle string // The title of the Atom XML feed
PlayEnabled bool
ServeLocalLinks bool // rewrite golang.org/{pkg,cmd} links to host-less, relative paths.
PlayEnabled bool
}
// Doc represents an article adorned with presentation data.
@ -81,17 +73,10 @@ type Server struct {
func NewServer(cfg Config) (*Server, error) {
present.PlayEnabled = cfg.PlayEnabled
if notExist(cfg.TemplatePath) {
return nil, fmt.Errorf("template directory not found: %s", cfg.TemplatePath)
}
root := filepath.Join(cfg.TemplatePath, "root.tmpl")
parse := func(name string) (*template.Template, error) {
path := filepath.Join(cfg.TemplatePath, name)
if notExist(path) {
return nil, fmt.Errorf("template %s was not found in %s", name, cfg.TemplatePath)
}
t := template.New("").Funcs(funcMap)
return t.ParseFiles(root, path)
return t.ParseFiles(root, filepath.Join(cfg.TemplatePath, name))
}
s := &Server{cfg: cfg}
@ -199,8 +184,8 @@ func (s *Server) loadDocs(root string) error {
if err != nil {
return err
}
var html bytes.Buffer
err = d.Render(&html, s.template.doc)
html := new(bytes.Buffer)
err = d.Render(html, s.template.doc)
if err != nil {
return err
}
@ -425,18 +410,7 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
d.Doc = doc
t = s.template.article
}
var err error
if s.cfg.ServeLocalLinks {
var buf bytes.Buffer
err = t.ExecuteTemplate(&buf, "root", d)
if err != nil {
log.Println(err)
return
}
_, err = golangOrgAbsLinkReplacer.WriteString(w, buf.String())
} else {
err = t.ExecuteTemplate(w, "root", d)
}
err := t.ExecuteTemplate(w, "root", d)
if err != nil {
log.Println(err)
}
@ -448,9 +422,3 @@ type docsByTime []*Doc
func (s docsByTime) Len() int { return len(s) }
func (s docsByTime) Swap(i, j int) { s[i], s[j] = s[j], s[i] }
func (s docsByTime) Less(i, j int) bool { return s[i].Time.After(s[j].Time) }
// notExist reports whether the path exists or not.
func notExist(path string) bool {
_, err := os.Stat(path)
return os.IsNotExist(err)
}

44
blog/blog_test.go
View File

@ -1,44 +0,0 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package blog
import (
"bytes"
"testing"
)
func TestLinkRewrite(t *testing.T) {
tests := []struct {
input string
output string
}{
{
`For instance, the <a href="https://golang.org/pkg/bytes/" target="_blank">bytes package</a> from the standard library exports the <code>Buffer</code> type.`,
`For instance, the <a href="/pkg/bytes/" target="_blank">bytes package</a> from the standard library exports the <code>Buffer</code> type.`},
{
`(The <a href="https://golang.org/cmd/gofmt/" target="_blank">gofmt command</a> has a <code>-r</code> flag that provides a syntax-aware search and replace, making large-scale refactoring easier.)`,
`(The <a href="/cmd/gofmt/" target="_blank">gofmt command</a> has a <code>-r</code> flag that provides a syntax-aware search and replace, making large-scale refactoring easier.)`,
},
{
`<a href="//golang.org/LICENSE">BSD license</a>.<br> <a href="//golang.org/doc/tos.html">Terms of Service</a> `,
`<a href="//golang.org/LICENSE">BSD license</a>.<br> <a href="//golang.org/doc/tos.html">Terms of Service</a> `,
},
{
`For instance, the <code>websocket</code> package from the <code>go.net</code> sub-repository has an import path of <code>&#34;golang.org/x/net/websocket&#34;</code>.`,
`For instance, the <code>websocket</code> package from the <code>go.net</code> sub-repository has an import path of <code>&#34;golang.org/x/net/websocket&#34;</code>.`,
},
}
for _, test := range tests {
var buf bytes.Buffer
_, err := golangOrgAbsLinkReplacer.WriteString(&buf, test.input)
if err != nil {
t.Errorf("unexpected error during replacing links. Got: %#v, Want: nil.\n", err)
continue
}
if got, want := buf.String(), test.output; got != want {
t.Errorf("WriteString(%q) = %q. Expected: %q", test.input, got, want)
}
}
}

231
cmd/auth/authtest/authtest.go
View File

@ -1,231 +0,0 @@
// Copyright 2019 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// authtest is a diagnostic tool for implementations of the GOAUTH protocol
// described in https://golang.org/issue/26232.
//
// It accepts a single URL as an argument, and executes the GOAUTH protocol to
// fetch and display the headers for that URL.
//
// CAUTION: authtest logs the GOAUTH responses, which may include user
// credentials, to stderr. Do not post its output unless you are certain that
// all of the credentials involved are fake!
package main
import (
"bufio"
"bytes"
"flag"
"fmt"
"io"
"log"
"net/http"
"net/textproto"
"net/url"
"os"
"os/exec"
"path/filepath"
"strings"
)
var v = flag.Bool("v", false, "if true, log GOAUTH responses to stderr")
func main() {
log.SetFlags(log.LstdFlags | log.Lshortfile)
flag.Parse()
args := flag.Args()
if len(args) != 1 {
log.Fatalf("usage: [GOAUTH=CMD...] %s URL", filepath.Base(os.Args[0]))
}
resp := try(args[0], nil)
if resp.StatusCode == http.StatusOK {
return
}
resp = try(args[0], resp)
if resp.StatusCode != http.StatusOK {
os.Exit(1)
}
}
func try(url string, prev *http.Response) *http.Response {
req := new(http.Request)
if prev != nil {
*req = *prev.Request
} else {
var err error
req, err = http.NewRequest("HEAD", os.Args[1], nil)
if err != nil {
log.Fatal(err)
}
}
goauth:
for _, argList := range strings.Split(os.Getenv("GOAUTH"), ";") {
// TODO(golang.org/issue/26849): If we escape quoted strings in GOFLAGS, use
// the same quoting here.
args := strings.Split(argList, " ")
if len(args) == 0 || args[0] == "" {
log.Fatalf("invalid or empty command in GOAUTH")
}
creds, err := getCreds(args, prev)
if err != nil {
log.Fatal(err)
}
for _, c := range creds {
if c.Apply(req) {
fmt.Fprintf(os.Stderr, "# request to %s\n", req.URL)
fmt.Fprintf(os.Stderr, "%s %s %s\n", req.Method, req.URL, req.Proto)
req.Header.Write(os.Stderr)
fmt.Fprintln(os.Stderr)
break goauth
}
}
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK && resp.StatusCode < 400 || resp.StatusCode > 500 {
log.Fatalf("unexpected status: %v", resp.Status)
}
fmt.Fprintf(os.Stderr, "# response from %s\n", resp.Request.URL)
formatHead(os.Stderr, resp)
return resp
}
func formatHead(out io.Writer, resp *http.Response) {
fmt.Fprintf(out, "%s %s\n", resp.Proto, resp.Status)
if err := resp.Header.Write(out); err != nil {
log.Fatal(err)
}
fmt.Fprintln(out)
}
type Cred struct {
URLPrefixes []*url.URL
Header http.Header
}
func (c Cred) Apply(req *http.Request) bool {
if req.URL == nil {
return false
}
ok := false
for _, prefix := range c.URLPrefixes {
if prefix.Host == req.URL.Host &&
(req.URL.Path == prefix.Path ||
(strings.HasPrefix(req.URL.Path, prefix.Path) &&
(strings.HasSuffix(prefix.Path, "/") ||
req.URL.Path[len(prefix.Path)] == '/'))) {
ok = true
break
}
}
if !ok {
return false
}
for k, vs := range c.Header {
req.Header.Del(k)
for _, v := range vs {
req.Header.Add(k, v)
}
}
return true
}
func (c Cred) String() string {
var buf strings.Builder
for _, u := range c.URLPrefixes {
fmt.Fprintln(&buf, u)
}
buf.WriteString("\n")
c.Header.Write(&buf)
buf.WriteString("\n")
return buf.String()
}
func getCreds(args []string, resp *http.Response) ([]Cred, error) {
cmd := exec.Command(args[0], args[1:]...)
cmd.Stderr = os.Stderr
if resp != nil {
u := *resp.Request.URL
u.RawQuery = ""
cmd.Args = append(cmd.Args, u.String())
}
var head strings.Builder
if resp != nil {
formatHead(&head, resp)
}
cmd.Stdin = strings.NewReader(head.String())
fmt.Fprintf(os.Stderr, "# %s\n", strings.Join(cmd.Args, " "))
out, err := cmd.Output()
if err != nil {
return nil, fmt.Errorf("%s: %v", strings.Join(cmd.Args, " "), err)
}
os.Stderr.Write(out)
os.Stderr.WriteString("\n")
var creds []Cred
r := textproto.NewReader(bufio.NewReader(bytes.NewReader(out)))
line := 0
readLoop:
for {
var prefixes []*url.URL
for {
prefix, err := r.ReadLine()
if err == io.EOF {
if len(prefixes) > 0 {
return nil, fmt.Errorf("line %d: %v", line, io.ErrUnexpectedEOF)
}
break readLoop
}
line++
if prefix == "" {
if len(prefixes) == 0 {
return nil, fmt.Errorf("line %d: unexpected newline", line)
}
break
}
u, err := url.Parse(prefix)
if err != nil {
return nil, fmt.Errorf("line %d: malformed URL: %v", line, err)
}
if u.Scheme != "https" {
return nil, fmt.Errorf("line %d: non-HTTPS URL %q", line, prefix)
}
if len(u.RawQuery) > 0 {
return nil, fmt.Errorf("line %d: unexpected query string in URL %q", line, prefix)
}
if len(u.Fragment) > 0 {
return nil, fmt.Errorf("line %d: unexpected fragment in URL %q", line, prefix)
}
prefixes = append(prefixes, u)
}
header, err := r.ReadMIMEHeader()
if err != nil {
return nil, fmt.Errorf("headers at line %d: %v", line, err)
}
if len(header) > 0 {
creds = append(creds, Cred{
URLPrefixes: prefixes,
Header: http.Header(header),
})
}
}
return creds, nil
}

166
cmd/auth/cookieauth/cookieauth.go
View File

@ -1,166 +0,0 @@
// Copyright 2019 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// cookieauth uses a “Netscape cookie file” to implement the GOAUTH protocol
// described in https://golang.org/issue/26232.
// It expects the location of the file as the first command-line argument.
//
// Example GOAUTH usage:
// export GOAUTH="cookieauth $(git config --get http.cookieFile)"
//
// See http://www.cookiecentral.com/faq/#3.5 for a description of the Netscape
// cookie file format.
package main
import (
"bufio"
"fmt"
"io"
"log"
"net/http"
"net/http/cookiejar"
"net/url"
"os"
"strconv"
"strings"
"time"
"unicode"
)
func main() {
if len(os.Args) < 2 {
fmt.Fprintf(os.Stderr, "usage: %s COOKIEFILE [URL]\n", os.Args[0])
os.Exit(2)
}
log.SetPrefix("cookieauth: ")
f, err := os.Open(os.Args[1])
if err != nil {
log.Fatalf("failed to read cookie file: %v\n", os.Args[1])
os.Exit(1)
}
defer f.Close()
var (
targetURL *url.URL
targetURLs = map[string]*url.URL{}
)
if len(os.Args) == 3 {
targetURL, err = url.ParseRequestURI(os.Args[2])
if err != nil {
log.Fatalf("invalid request URI (%v): %q\n", err, os.Args[2])
}
targetURLs[targetURL.String()] = targetURL
} else if len(os.Args) > 3 {
// Extra arguments were passed: maybe the protocol was expanded?
// We don't know how to interpret the request, so ignore it.
return
}
entries, err := parseCookieFile(f.Name(), f)
if err != nil {
log.Fatalf("error reading cookie file: %v\n", f.Name())
}
jar, err := cookiejar.New(nil)
if err != nil {
log.Fatalf("failed to initialize cookie jar: %v\n", err)
}
for _, e := range entries {
u := &url.URL{
Scheme: "https",
Host: e.Host,
Path: e.Cookie.Path,
}
if targetURL == nil {
targetURLs[u.String()] = u
}
jar.SetCookies(u, []*http.Cookie{&e.Cookie})
}
for _, u := range targetURLs {
req := &http.Request{URL: u, Header: make(http.Header)}
for _, c := range jar.Cookies(req.URL) {
req.AddCookie(c)
}
fmt.Printf("%s\n\n", u)
req.Header.Write(os.Stdout)
fmt.Println()
}
}
type Entry struct {
Host string
Cookie http.Cookie
}
// parseCookieFile parses a Netscape cookie file as described in
// http://www.cookiecentral.com/faq/#3.5.
func parseCookieFile(name string, r io.Reader) ([]*Entry, error) {
var entries []*Entry
s := bufio.NewScanner(r)
line := 0
for s.Scan() {
line++
text := strings.TrimSpace(s.Text())
if len(text) < 2 || (text[0] == '#' && unicode.IsSpace(rune(text[1]))) {
continue
}
e, err := parseCookieLine(text)
if err != nil {
log.Printf("%s:%d: %v\n", name, line, err)
continue
}
entries = append(entries, e)
}
return entries, s.Err()
}
func parseCookieLine(line string) (*Entry, error) {
f := strings.Fields(line)
if len(f) < 7 {
return nil, fmt.Errorf("found %d columns; want 7", len(f))
}
e := new(Entry)
c := &e.Cookie
if domain := f[0]; strings.HasPrefix(domain, "#HttpOnly_") {
c.HttpOnly = true
e.Host = strings.TrimPrefix(domain[10:], ".")
} else {
e.Host = strings.TrimPrefix(domain, ".")
}
isDomain, err := strconv.ParseBool(f[1])
if err != nil {
return nil, fmt.Errorf("non-boolean domain flag: %v", err)
}
if isDomain {
c.Domain = e.Host
}
c.Path = f[2]
c.Secure, err = strconv.ParseBool(f[3])
if err != nil {
return nil, fmt.Errorf("non-boolean secure flag: %v", err)
}
expiration, err := strconv.ParseInt(f[4], 10, 64)
if err != nil {
return nil, fmt.Errorf("malformed expiration: %v", err)
}
c.Expires = time.Unix(expiration, 0)
c.Name = f[5]
c.Value = f[6]
return e, nil
}

149
cmd/auth/gitauth/gitauth.go
View File

@ -1,149 +0,0 @@
// Copyright 2019 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// gitauth uses 'git credential' to implement the GOAUTH protocol described in
// https://golang.org/issue/26232. It expects an absolute path to the working
// directory for the 'git' command as the first command-line argument.
//
// Example GOAUTH usage:
// export GOAUTH="gitauth $HOME"
//
// See https://git-scm.com/docs/gitcredentials or run 'man gitcredentials' for
// information on how to configure 'git credential'.
package main
import (
"bytes"
"fmt"
"log"
"net/http"
"net/url"
"os"
"os/exec"
"path/filepath"
"strings"
)
func main() {
if len(os.Args) < 2 || !filepath.IsAbs(os.Args[1]) {
fmt.Fprintf(os.Stderr, "usage: %s WORKDIR [URL]", os.Args[0])
os.Exit(2)
}
log.SetPrefix("gitauth: ")
if len(os.Args) != 3 {
// No explicit URL was passed on the command line, but 'git credential'
// provides no way to enumerate existing credentials.
// Wait for a request for a specific URL.
return
}
u, err := url.ParseRequestURI(os.Args[2])
if err != nil {
log.Fatalf("invalid request URI (%v): %q\n", err, os.Args[1])
}
var (
prefix *url.URL
lastHeader http.Header
lastStatus = http.StatusUnauthorized
)
for lastStatus == http.StatusUnauthorized {
cmd := exec.Command("git", "credential", "fill")
// We don't want to execute a 'git' command in an arbitrary directory, since
// that opens up a number of config-injection attacks (for example,
// https://golang.org/issue/29230). Instead, we have the user configure a
// directory explicitly on the command line.
cmd.Dir = os.Args[1]
cmd.Stdin = strings.NewReader(fmt.Sprintf("url=%s\n", u))
cmd.Stderr = os.Stderr
out, err := cmd.Output()
if err != nil {
log.Fatalf("'git credential fill' failed: %v\n", err)
}
prefix = new(url.URL)
var username, password string
lines := strings.Split(string(out), "\n")
for _, line := range lines {
frags := strings.SplitN(line, "=", 2)
if len(frags) != 2 {
continue // Ignore unrecognized response lines.
}
switch strings.TrimSpace(frags[0]) {
case "protocol":
prefix.Scheme = frags[1]
case "host":
prefix.Host = frags[1]
case "path":
prefix.Path = frags[1]
case "username":
username = frags[1]
case "password":
password = frags[1]
case "url":
// Write to a local variable instead of updating prefix directly:
// if the url field is malformed, we don't want to invalidate
// information parsed from the protocol, host, and path fields.
u, err := url.ParseRequestURI(frags[1])
if err == nil {
prefix = u
} else {
log.Printf("malformed URL from 'git credential fill' (%v): %q\n", err, frags[1])
// Proceed anyway: we might be able to parse the prefix from other fields of the response.
}
}
}
// Double-check that the URL Git gave us is a prefix of the one we requested.
if !strings.HasPrefix(u.String(), prefix.String()) {
log.Fatalf("requested a credential for %q, but 'git credential fill' provided one for %q\n", u, prefix)
}
// Send a HEAD request to try to detect whether the credential is valid.
// If the user just typed in a correct password and has caching enabled,
// we don't want to nag them for it again the next time they run a 'go' command.
req, err := http.NewRequest("HEAD", u.String(), nil)
if err != nil {
log.Fatalf("internal error constructing HTTP HEAD request: %v\n", err)
}
req.SetBasicAuth(username, password)
lastHeader = req.Header
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Printf("HTTPS HEAD request failed to connect: %v\n", err)
// Couldn't verify the credential, but we have no evidence that it is invalid either.
// Proceed, but don't update git's credential cache.
break
}
lastStatus = resp.StatusCode
if resp.StatusCode != http.StatusOK {
log.Printf("%s: %v %s\n", u, resp.StatusCode, http.StatusText(resp.StatusCode))
}
if resp.StatusCode == http.StatusOK || resp.StatusCode == http.StatusUnauthorized {
// We learned something about the credential: it either worked or it was invalid.
// Approve or reject the credential (on a best-effort basis)
// so that the git credential helper can update its cache as appropriate.
action := "approve"
if resp.StatusCode != http.StatusOK {
action = "reject"
}
cmd = exec.Command("git", "credential", action)
cmd.Stderr = os.Stderr
cmd.Stdout = os.Stderr
cmd.Stdin = bytes.NewReader(out)
_ = cmd.Run()
}
}
// Write out the credential in the format expected by the 'go' command.
fmt.Printf("%s\n\n", prefix)
lastHeader.Write(os.Stdout)
fmt.Println()
}

123
cmd/auth/netrcauth/netrcauth.go
View File

@ -1,123 +0,0 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// netrcauth uses a .netrc file (or _netrc file on Windows) to implement the
// GOAUTH protocol described in https://golang.org/issue/26232.
// It expects the location of the file as the first command-line argument.
//
// Example GOAUTH usage:
// export GOAUTH="netrcauth $HOME/.netrc"
//
// See https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html
// or run 'man 5 netrc' for a description of the .netrc file format.
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"net/url"
"os"
"strings"
)
func main() {
if len(os.Args) < 2 {
fmt.Fprintf(os.Stderr, "usage: %s NETRCFILE [URL]", os.Args[0])
os.Exit(2)
}
log.SetPrefix("netrcauth: ")
if len(os.Args) != 2 {
// An explicit URL was passed on the command line, but netrcauth does not
// have any URL-specific output: it dumps the entire .netrc file at the
// first call.
return
}
path := os.Args[1]
data, err := ioutil.ReadFile(path)
if err != nil {
if os.IsNotExist(err) {
return
}
log.Fatalf("failed to read %s: %v\n", path, err)
}
u := &url.URL{Scheme: "https"}
lines := parseNetrc(string(data))
for _, l := range lines {
u.Host = l.machine
fmt.Printf("%s\n\n", u)
req := &http.Request{Header: make(http.Header)}
req.SetBasicAuth(l.login, l.password)
req.Header.Write(os.Stdout)
fmt.Println()
}
}
// The following functions were extracted from src/cmd/go/internal/web2/web.go
// as of https://golang.org/cl/161698.
type netrcLine struct {
machine string
login string
password string
}
func parseNetrc(data string) []netrcLine {
// See https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html
// for documentation on the .netrc format.
var nrc []netrcLine
var l netrcLine
inMacro := false
for _, line := range strings.Split(data, "\n") {
if inMacro {
if line == "" {
inMacro = false
}
continue
}
f := strings.Fields(line)
i := 0
for ; i < len(f)-1; i += 2 {
// Reset at each "machine" token.
// “The auto-login process searches the .netrc file for a machine token
// that matches […]. Once a match is made, the subsequent .netrc tokens
// are processed, stopping when the end of file is reached or another
// machine or a default token is encountered.”
switch f[i] {
case "machine":
l = netrcLine{machine: f[i+1]}
case "default":
break
case "login":
l.login = f[i+1]
case "password":
l.password = f[i+1]
case "macdef":
// “A macro is defined with the specified name; its contents begin with
// the next .netrc line and continue until a null line (consecutive
// new-line characters) is encountered.”
inMacro = true
}
if l.machine != "" && l.login != "" && l.password != "" {
nrc = append(nrc, l)
l = netrcLine{}
}
}
if i < len(f) && f[i] == "default" {
// “There can be only one default token, and it must be after all machine tokens.”
break
}
}
return nrc
}

1
cmd/bundle/.gitignore vendored
View File

@ -1 +0,0 @@
testdata/out.got

377
cmd/bundle/main.go
View File

@ -2,81 +2,36 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Bundle creates a single-source-file version of a source package
// suitable for inclusion in a particular target package.
// +build go1.5
// The bundle command concatenates the source files of a package,
// renaming package-level names by adding a prefix and renaming
// identifiers as needed to preserve referential integrity.
//
// Usage:
// Example:
// $ bundle golang.org/x/net/http2 net/http http2
//
// bundle [-o file] [-dst path] [-pkg name] [-prefix p] [-import old=new] <src>
// The command above prints a single file containing the code of
// golang.org/x/net/http2, suitable for inclusion in package net/http,
// in which toplevel names have been prefixed with "http2".
//
// The src argument specifies the import path of the package to bundle.
// The bundling of a directory of source files into a single source file
// necessarily imposes a number of constraints.
// The package being bundled must not use cgo; must not use conditional
// file compilation, whether with build tags or system-specific file names
// like code_amd64.go; must not depend on any special comments, which
// may not be preserved; must not use any assembly sources;
// must not use renaming imports; and must not use reflection-based APIs
// that depend on the specific names of types or struct fields.
// Assumptions:
// - no file in the package imports "C", that is, uses cgo.
// - no file in the package has GOOS or GOARCH build tags or file names.
// - comments associated with the package or import declarations,
// may be discarded, as may comments associated with no top-level
// declaration at all.
// - neither the original package nor the destination package contains
// any identifiers starting with the designated prefix.
// (This allows us to avoid various conflict checks.)
// - there are no renaming imports.
// - test files are ignored.
// - none of the renamed identifiers is significant
// to reflection-based logic.
//
// By default, bundle writes the bundled code to standard output.
// If the -o argument is given, bundle writes to the named file
// and also includes a ``//go:generate'' comment giving the exact
// command line used, for regenerating the file with ``go generate.''
//
// Bundle customizes its output for inclusion in a particular package, the destination package.
// By default bundle assumes the destination is the package in the current directory,
// but the destination package can be specified explicitly using the -dst option,
// which takes an import path as its argument.
// If the source package imports the destination package, bundle will remove
// those imports and rewrite any references to use direct references to the
// corresponding symbols.
// Bundle also must write a package declaration in the output and must
// choose a name to use in that declaration.
// If the -package option is given, bundle uses that name.
// Otherwise, if the -dst option is given, bundle uses the last
// element of the destination import path.
// Otherwise, by default bundle uses the package name found in the
// package sources in the current directory.
//
// To avoid collisions, bundle inserts a prefix at the beginning of
// every package-level const, func, type, and var identifier in src's code,
// updating references accordingly. The default prefix is the package name
// of the source package followed by an underscore. The -prefix option
// specifies an alternate prefix.
//
// Occasionally it is necessary to rewrite imports during the bundling
// process. The -import option, which may be repeated, specifies that
// an import of "old" should be rewritten to import "new" instead.
//
// Example
//
// Bundle archive/zip for inclusion in cmd/dist:
//
// cd $GOROOT/src/cmd/dist
// bundle -o zip.go archive/zip
//
// Bundle golang.org/x/net/http2 for inclusion in net/http,
// prefixing all identifiers by "http2" instead of "http2_",
// and rewriting the import "golang.org/x/net/http2/hpack"
// to "internal/golang.org/x/net/http2/hpack":
//
// cd $GOROOT/src/net/http
// bundle -o h2_bundle.go \
// -prefix http2 \
// -import golang.org/x/net/http2/hpack=internal/golang.org/x/net/http2/hpack \
// golang.org/x/net/http2
//
// Two ways to update the http2 bundle:
//
// go generate net/http
//
// cd $GOROOT/src/net/http
// go generate
//
// Update both bundles, restricting ``go generate'' to running bundle commands:
//
// go generate -run bundle cmd/dist net/http
// Only package-level var, func, const, and type objects are renamed,
// and embedded fields of renamed types. No methods are renamed, so we
// needn't worry about preserving interface satisfaction.
//
package main
@ -88,124 +43,53 @@ import (
"go/build"
"go/format"
"go/parser"
"go/printer"
"go/token"
"go/types"
"io/ioutil"
"io"
"log"
"os"
"path"
"strconv"
"path/filepath"
"strings"
"golang.org/x/tools/go/loader"
)
var (
outputFile = flag.String("o", "", "write output to `file` (default standard output)")
dstPath = flag.String("dst", "", "set destination import `path` (default taken from current directory)")
pkgName = flag.String("pkg", "", "set destination package `name` (default taken from current directory)")
prefix = flag.String("prefix", "&_", "set bundled identifier prefix to `p` (default is \"&_\", where & stands for the original name)")
underscore = flag.Bool("underscore", false, "rewrite golang.org/x/* to internal/x/* imports; temporary workaround for golang.org/issue/16333")
importMap = map[string]string{}
)
func init() {
flag.Var(flagFunc(addImportMap), "import", "rewrite import using `map`, of form old=new (can be repeated)")
}
func addImportMap(s string) {
if strings.Count(s, "=") != 1 {
log.Fatal("-import argument must be of the form old=new")
}
i := strings.Index(s, "=")
old, new := s[:i], s[i+1:]
if old == "" || new == "" {
log.Fatal("-import argument must be of the form old=new; old and new must be non-empty")
}
importMap[old] = new
}
func usage() {
fmt.Fprintf(os.Stderr, "Usage: bundle [options] <src>\n")
flag.PrintDefaults()
}
func main() {
log.SetPrefix("bundle: ")
log.SetFlags(0)
flag.Usage = usage
flag.Parse()
args := flag.Args()
if len(args) != 1 {
usage()
os.Exit(2)
}
if len(args) != 3 {
log.Fatal(`Usage: bundle package dest prefix
if *dstPath != "" {
if *pkgName == "" {
*pkgName = path.Base(*dstPath)
}
} else {
wd, _ := os.Getwd()
pkg, err := build.ImportDir(wd, 0)
if err != nil {
log.Fatalf("cannot find package in current directory: %v", err)
}
*dstPath = pkg.ImportPath
if *pkgName == "" {
*pkgName = pkg.Name
}
Arguments:
package is the import path of the package to concatenate.
dest is the import path of the package in which the resulting file will reside.
prefix is the string to attach to all renamed identifiers.
`)
}
initialPkg, dest, prefix := args[0], args[1], args[2]
code, err := bundle(args[0], *dstPath, *pkgName, *prefix)
if err != nil {
if err := bundle(os.Stdout, initialPkg, dest, prefix); err != nil {
log.Fatal(err)
}
if *outputFile != "" {
err := ioutil.WriteFile(*outputFile, code, 0666)
if err != nil {
log.Fatal(err)
}
} else {
_, err := os.Stdout.Write(code)
if err != nil {
log.Fatal(err)
}
}
}
// isStandardImportPath is copied from cmd/go in the standard library.
func isStandardImportPath(path string) bool {
i := strings.Index(path, "/")
if i < 0 {
i = len(path)
}
elem := path[:i]
return !strings.Contains(elem, ".")
}
var ctxt = &build.Default
func bundle(src, dst, dstpkg, prefix string) ([]byte, error) {
func bundle(w io.Writer, initialPkg, dest, prefix string) error {
// Load the initial package.
conf := loader.Config{ParserMode: parser.ParseComments, Build: ctxt}
conf.TypeCheckFuncBodies = func(p string) bool { return p == src }
conf.Import(src)
conf.TypeCheckFuncBodies = func(p string) bool { return p == initialPkg }
conf.Import(initialPkg)
lprog, err := conf.Load()
if err != nil {
return nil, err
log.Fatal(err)
}
// Because there was a single Import call and Load succeeded,
// InitialPackages is guaranteed to hold the sole requested package.
info := lprog.InitialPackages()[0]
if strings.Contains(prefix, "&") {
prefix = strings.Replace(prefix, "&", info.Files[0].Name.Name, -1)
}
info := lprog.Package(initialPkg)
objsToUpdate := make(map[types.Object]bool)
var rename func(from types.Object)
@ -238,13 +122,8 @@ func bundle(src, dst, dstpkg, prefix string) ([]byte, error) {
var out bytes.Buffer
fmt.Fprintf(&out, "// Code generated by golang.org/x/tools/cmd/bundle. DO NOT EDIT.\n")
if *outputFile != "" {
fmt.Fprintf(&out, "//go:generate bundle %s\n", strings.Join(os.Args[1:], " "))
} else {
fmt.Fprintf(&out, "// $ bundle %s\n", strings.Join(os.Args[1:], " "))
}
fmt.Fprintf(&out, "\n")
fmt.Fprintf(&out, "// Code generated by golang.org/x/tools/cmd/bundle command:\n")
fmt.Fprintf(&out, "// $ bundle %s %s %s\n\n", initialPkg, dest, prefix)
// Concatenate package comments from all files...
for _, f := range info.Files {
@ -257,66 +136,29 @@ func bundle(src, dst, dstpkg, prefix string) ([]byte, error) {
// ...but don't let them become the actual package comment.
fmt.Fprintln(&out)
fmt.Fprintf(&out, "package %s\n\n", dstpkg)
// TODO(adonovan): don't assume pkg.name == basename(pkg.path).
fmt.Fprintf(&out, "package %s\n\n", filepath.Base(dest))
// BUG(adonovan,shurcooL): bundle may generate incorrect code
// due to shadowing between identifiers and imported package names.
//
// The generated code will either fail to compile or
// (unlikely) compile successfully but have different behavior
// than the original package. The risk of this happening is higher
// when the original package has renamed imports (they're typically
// renamed in order to resolve a shadow inside that particular .go file).
// TODO(adonovan,shurcooL):
// - detect shadowing issues, and either return error or resolve them
// Print a single declaration that imports all necessary packages.
// TODO(adonovan):
// - support renaming imports.
// - preserve comments from the original import declarations.
// pkgStd and pkgExt are sets of printed import specs. This is done
// to deduplicate instances of the same import name and path.
var pkgStd = make(map[string]bool)
var pkgExt = make(map[string]bool)
for _, f := range info.Files {
for _, imp := range f.Imports {
path, err := strconv.Unquote(imp.Path.Value)
if err != nil {
log.Fatalf("invalid import path string: %v", err) // Shouldn't happen here since conf.Load succeeded.
}
if path == dst {
continue
}
if newPath, ok := importMap[path]; ok {
path = newPath
}
var name string
if imp.Name != nil {
name = imp.Name.Name
}
spec := fmt.Sprintf("%s %q", name, path)
if isStandardImportPath(path) {
pkgStd[spec] = true
} else {
if *underscore {
spec = strings.Replace(spec, "golang.org/x/", "internal/x/", 1)
}
pkgExt[spec] = true
log.Fatalf("%s: renaming imports not supported",
lprog.Fset.Position(imp.Pos()))
}
}
}
// Print a single declaration that imports all necessary packages.
fmt.Fprintln(&out, "import (")
for p := range pkgStd {
fmt.Fprintf(&out, "\t%s\n", p)
for _, p := range info.Pkg.Imports() {
if p.Path() == dest {
continue
}
fmt.Fprintf(&out, "\t%q\n", p.Path())
}
if len(pkgExt) > 0 {
fmt.Fprintln(&out)
}
for p := range pkgExt {
fmt.Fprintf(&out, "\t%s\n", p)
}
fmt.Fprint(&out, ")\n\n")
fmt.Fprintln(&out, ")\n")
// Modify and print each file.
for _, f := range info.Files {
@ -339,7 +181,7 @@ func bundle(src, dst, dstpkg, prefix string) ([]byte, error) {
if sel, ok := n.(*ast.SelectorExpr); ok {
if id, ok := sel.X.(*ast.Ident); ok {
if obj, ok := info.Uses[id].(*types.PkgName); ok {
if obj.Imported().Path() == dst {
if obj.Imported().Path() == dest {
id.Name = "@@@"
}
}
@ -348,41 +190,25 @@ func bundle(src, dst, dstpkg, prefix string) ([]byte, error) {
return true
})
last := f.Package
if len(f.Imports) > 0 {
imp := f.Imports[len(f.Imports)-1]
last = imp.End()
if imp.Comment != nil {
if e := imp.Comment.End(); e > last {
last = e
}
}
}
// Pretty-print package-level declarations.
// but no package or import declarations.
//
// TODO(adonovan): this may cause loss of comments
// preceding or associated with the package or import
// declarations or not associated with any declaration.
// Check.
var buf bytes.Buffer
for _, decl := range f.Decls {
if decl, ok := decl.(*ast.GenDecl); ok && decl.Tok == token.IMPORT {
continue
}
beg, end := sourceRange(decl)
printComments(&out, f.Comments, last, beg)
buf.Reset()
format.Node(&buf, lprog.Fset, &printer.CommentedNode{Node: decl, Comments: f.Comments})
format.Node(&buf, lprog.Fset, decl)
// Remove each "@@@." in the output.
// TODO(adonovan): not hygienic.
out.Write(bytes.Replace(buf.Bytes(), []byte("@@@."), nil, -1))
last = printSameLineComment(&out, f.Comments, lprog.Fset, end)
out.WriteString("\n\n")
}
printLastComments(&out, f.Comments, last)
}
// Now format the entire thing.
@ -391,77 +217,6 @@ func bundle(src, dst, dstpkg, prefix string) ([]byte, error) {
log.Fatalf("formatting failed: %v", err)
}
return result, nil
_, err = w.Write(result)
return err
}
// sourceRange returns the [beg, end) interval of source code
// belonging to decl (incl. associated comments).
func sourceRange(decl ast.Decl) (beg, end token.Pos) {
beg = decl.Pos()
end = decl.End()
var doc, com *ast.CommentGroup
switch d := decl.(type) {
case *ast.GenDecl:
doc = d.Doc
if len(d.Specs) > 0 {
switch spec := d.Specs[len(d.Specs)-1].(type) {
case *ast.ValueSpec:
com = spec.Comment
case *ast.TypeSpec:
com = spec.Comment
}
}
case *ast.FuncDecl:
doc = d.Doc
}
if doc != nil {
beg = doc.Pos()
}
if com != nil && com.End() > end {
end = com.End()
}
return beg, end
}
func printComments(out *bytes.Buffer, comments []*ast.CommentGroup, pos, end token.Pos) {
for _, cg := range comments {
if pos <= cg.Pos() && cg.Pos() < end {
for _, c := range cg.List {
fmt.Fprintln(out, c.Text)
}
fmt.Fprintln(out)
}
}
}
const infinity = 1 << 30
func printLastComments(out *bytes.Buffer, comments []*ast.CommentGroup, pos token.Pos) {
printComments(out, comments, pos, infinity)
}
func printSameLineComment(out *bytes.Buffer, comments []*ast.CommentGroup, fset *token.FileSet, pos token.Pos) token.Pos {
tf := fset.File(pos)
for _, cg := range comments {
if pos <= cg.Pos() && tf.Line(cg.Pos()) == tf.Line(pos) {
for _, c := range cg.List {
fmt.Fprintln(out, c.Text)
}
return cg.End()
}
}
return pos
}
type flagFunc func(string)
func (f flagFunc) Set(s string) error {
f(s)
return nil
}
func (f flagFunc) String() string { return "" }

18
cmd/bundle/main_test.go
View File

@ -2,14 +2,11 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build go1.9
package main
import (
"bytes"
"io/ioutil"
"os"
"os/exec"
"runtime"
"testing"
@ -30,28 +27,23 @@ func TestBundle(t *testing.T) {
"initial": {
"a.go": load("testdata/src/initial/a.go"),
"b.go": load("testdata/src/initial/b.go"),
"c.go": load("testdata/src/initial/c.go"),
},
"domain.name/importdecl": {
"p.go": load("testdata/src/domain.name/importdecl/p.go"),
},
"fmt": {
"print.go": `package fmt; func Println(...interface{})`,
},
})
os.Args = os.Args[:1] // avoid e.g. -test=short in the output
out, err := bundle("initial", "github.com/dest", "dest", "prefix")
if err != nil {
var out bytes.Buffer
if err := bundle(&out, "initial", "dest", "prefix"); err != nil {
t.Fatal(err)
}
if got, want := string(out), load("testdata/out.golden"); got != want {
if got, want := out.String(), load("testdata/out.golden"); got != want {
t.Errorf("-- got --\n%s\n-- want --\n%s\n-- diff --", got, want)
if err := ioutil.WriteFile("testdata/out.got", out, 0644); err != nil {
if err := ioutil.WriteFile("testdata/out.got", out.Bytes(), 0644); err != nil {
t.Fatal(err)
}
t.Log(diff("testdata/out.golden", "testdata/out.got"))
t.Log(diff("testdata/out.got", "testdata/out.golden"))
}
}

45
cmd/bundle/testdata/out.golden vendored
View File

@ -1,5 +1,5 @@
// Code generated by golang.org/x/tools/cmd/bundle. DO NOT EDIT.
// $ bundle
// Code generated by golang.org/x/tools/cmd/bundle command:
// $ bundle initial dest prefix
// The package doc comment
//
@ -8,12 +8,6 @@ package dest
import (
"fmt"
. "fmt"
_ "fmt"
renamedfmt "fmt"
renamedfmt2 "fmt"
"domain.name/importdecl"
)
// init functions are not renamed
@ -23,40 +17,15 @@ func init() { prefixfoo() }
type prefixS struct {
prefixt
u int
} /* multi-line
comment
*/
// non-associated comment
/*
non-associated comment2
*/
}
// Function bar.
func prefixbar(s *prefixS) {
fmt.Println(s.prefixt, s.u) // comment inside function
}
func prefixbar(s *prefixS) { fmt.Println(s.prefixt, s.u) }
// file-end comment
type prefixt int
type prefixt int // type1
// const1
const prefixc = 1 // const2
const prefixc = 1
func prefixfoo() {
fmt.Println(importdecl.F())
}
// zinit
const (
prefixz1 = iota // z1
prefixz2 // z2
) // zend
func prefixbaz() {
renamedfmt.Println()
renamedfmt2.Println()
Println()
fmt.Println()
}

3
cmd/bundle/testdata/src/domain.name/importdecl/p.go vendored
View File

@ -1,3 +0,0 @@
package importdecl
func F() int { return 1 }

18
cmd/bundle/testdata/src/initial/a.go vendored
View File

@ -1,6 +1,6 @@
package initial
import "fmt" // this comment should not be visible
import "fmt"
// init functions are not renamed
func init() { foo() }
@ -9,19 +9,7 @@ func init() { foo() }
type S struct {
t
u int
} /* multi-line
comment
*/
// non-associated comment
/*
non-associated comment2
*/
// Function bar.
func bar(s *S) {
fmt.Println(s.t, s.u) // comment inside function
}
// file-end comment
// Function bar.
func bar(s *S) { fmt.Println(s.t, s.u) }

19
cmd/bundle/testdata/src/initial/b.go vendored
View File

@ -1,23 +1,12 @@
// The package doc comment
package initial
import (
"fmt"
import "fmt"
"domain.name/importdecl"
)
type t int
type t int // type1
// const1
const c = 1 // const2
const c = 1
func foo() {
fmt.Println(importdecl.F())
fmt.Println()
}
// zinit
const (
z1 = iota // z1
z2 // z2
) // zend

12
cmd/bundle/testdata/src/initial/c.go vendored
View File

@ -1,12 +0,0 @@
package initial
import _ "fmt"
import renamedfmt "fmt"
import renamedfmt2 "fmt"
import . "fmt"
func baz() {
renamedfmt.Println()
renamedfmt2.Println()
Println()
}

86
cmd/callgraph/main.go
View File

@ -37,7 +37,7 @@ import (
"golang.org/x/tools/go/callgraph/cha"
"golang.org/x/tools/go/callgraph/rta"
"golang.org/x/tools/go/callgraph/static"
"golang.org/x/tools/go/packages"
"golang.org/x/tools/go/loader"
"golang.org/x/tools/go/pointer"
"golang.org/x/tools/go/ssa"
"golang.org/x/tools/go/ssa/ssautil"
@ -67,7 +67,7 @@ const Usage = `callgraph: display the the call graph of a Go program.
Usage:
callgraph [-algo=static|cha|rta|pta] [-test] [-format=...] package...
callgraph [-algo=static|cha|rta|pta] [-test] [-format=...] <args>...
Flags:
@ -114,10 +114,12 @@ Flags:
Caller and Callee are *ssa.Function values, which print as
"(*sync/atomic.Mutex).Lock", but other attributes may be
derived from them, e.g. Caller.Pkg.Pkg.Path yields the
derived from them, e.g. Caller.Pkg.Object.Path yields the
import path of the enclosing package. Consult the go/ssa
API documentation for details.
` + loader.FromArgsUsage + `
Examples:
Show the call graph of the trivial web server application:
@ -126,7 +128,7 @@ Examples:
Same, but show only the packages of each function:
callgraph -format '{{.Caller.Pkg.Pkg.Path}} -> {{.Callee.Pkg.Pkg.Path}}' \
callgraph -format '{{.Caller.Pkg.Object.Path}} -> {{.Callee.Pkg.Object.Path}}' \
$GOROOT/src/net/http/triv.go | sort | uniq
Show functions that make dynamic calls into the 'fmt' test package,
@ -156,7 +158,7 @@ func init() {
func main() {
flag.Parse()
if err := doCallgraph("", "", *algoFlag, *formatFlag, *testFlag, flag.Args()); err != nil {
if err := doCallgraph(&build.Default, *algoFlag, *formatFlag, *testFlag, flag.Args()); err != nil {
fmt.Fprintf(os.Stderr, "callgraph: %s\n", err)
os.Exit(1)
}
@ -164,30 +166,28 @@ func main() {
var stdout io.Writer = os.Stdout
func doCallgraph(dir, gopath, algo, format string, tests bool, args []string) error {
func doCallgraph(ctxt *build.Context, algo, format string, tests bool, args []string) error {
conf := loader.Config{Build: ctxt}
if len(args) == 0 {
fmt.Fprintln(os.Stderr, Usage)
return nil
}
cfg := &packages.Config{
Mode: packages.LoadAllSyntax,
Tests: tests,
Dir: dir,
}
if gopath != "" {
cfg.Env = append(os.Environ(), "GOPATH="+gopath) // to enable testing
}
initial, err := packages.Load(cfg, args...)
// Use the initial packages from the command line.
args, err := conf.FromArgs(args, tests)
if err != nil {
return err
}
if packages.PrintErrors(initial) > 0 {
return fmt.Errorf("packages contain errors")
// Load, parse and type-check the whole program.
iprog, err := conf.Load()
if err != nil {
return err
}
// Create and build SSA-form program representation.
prog, pkgs := ssautil.AllPackages(initial, 0)
prog := ssautil.CreateProgram(iprog, 0)
prog.Build()
// -- call graph construction ------------------------------------------
@ -221,12 +221,12 @@ func doCallgraph(dir, gopath, algo, format string, tests bool, args []string) er
}
}
mains, err := mainPackages(pkgs)
main, err := mainPackage(prog, tests)
if err != nil {
return err
}
config := &pointer.Config{
Mains: mains,
Mains: []*ssa.Package{main},
BuildCallGraph: true,
Log: ptalog,
}
@ -237,13 +237,13 @@ func doCallgraph(dir, gopath, algo, format string, tests bool, args []string) er
cg = ptares.CallGraph
case "rta":
mains, err := mainPackages(pkgs)
main, err := mainPackage(prog, tests)
if err != nil {
return err
}
var roots []*ssa.Function
for _, main := range mains {
roots = append(roots, main.Func("init"), main.Func("main"))
roots := []*ssa.Function{
main.Func("init"),
main.Func("main"),
}
rtares := rta.Analyze(roots, true)
cg = rtares.CallGraph
@ -303,19 +303,35 @@ func doCallgraph(dir, gopath, algo, format string, tests bool, args []string) er
return nil
}
// mainPackages returns the main packages to analyze.
// Each resulting package is named "main" and has a main function.
func mainPackages(pkgs []*ssa.Package) ([]*ssa.Package, error) {
var mains []*ssa.Package
for _, p := range pkgs {
if p != nil && p.Pkg.Name() == "main" && p.Func("main") != nil {
mains = append(mains, p)
// mainPackage returns the main package to analyze.
// The resulting package has a main() function.
func mainPackage(prog *ssa.Program, tests bool) (*ssa.Package, error) {
pkgs := prog.AllPackages()
// TODO(adonovan): allow independent control over tests, mains and libraries.
// TODO(adonovan): put this logic in a library; we keep reinventing it.
if tests {
// If -test, use all packages' tests.
if len(pkgs) > 0 {
if main := prog.CreateTestMainPackage(pkgs...); main != nil {
return main, nil
}
}
return nil, fmt.Errorf("no tests")
}
// Otherwise, use the first package named main.
for _, pkg := range pkgs {
if pkg.Pkg.Name() == "main" {
if pkg.Func("main") == nil {
return nil, fmt.Errorf("no func main() in main package")
}
return pkg, nil
}
}
if len(mains) == 0 {
return nil, fmt.Errorf("no main packages")
}
return mains, nil
return nil, fmt.Errorf("no main package")
}
type Edge struct {

84
cmd/callgraph/main_test.go
View File

@ -5,44 +5,31 @@
// No testdata on Android.
// +build !android
// +build go1.11
package main
import (
"bytes"
"fmt"
"log"
"os"
"path/filepath"
"go/build"
"reflect"
"sort"
"strings"
"testing"
)
func init() {
// This test currently requires GOPATH mode.
// Explicitly disabling module mode should suffix, but
// we'll also turn off GOPROXY just for good measure.
if err := os.Setenv("GO111MODULE", "off"); err != nil {
log.Fatal(err)
}
if err := os.Setenv("GOPROXY", "off"); err != nil {
log.Fatal(err)
}
}
func TestCallgraph(t *testing.T) {
gopath, err := filepath.Abs("testdata")
if err != nil {
t.Fatal(err)
}
ctxt := build.Default // copy
ctxt.GOPATH = "testdata"
const format = "{{.Caller}} --> {{.Callee}}"
for _, test := range []struct {
algo string
tests bool
want []string
algo, format string
tests bool
want []string
}{
{"rta", false, []string{
{"rta", format, false, []string{
// rta imprecisely shows cross product of {main,main2} x {C,D}
`pkg.main --> (pkg.C).f`,
`pkg.main --> (pkg.D).f`,
@ -50,7 +37,7 @@ func TestCallgraph(t *testing.T) {
`pkg.main2 --> (pkg.C).f`,
`pkg.main2 --> (pkg.D).f`,
}},
{"pta", false, []string{
{"pta", format, false, []string{
// pta distinguishes main->C, main2->D. Also has a root node.
`<root> --> pkg.init`,
`<root> --> pkg.main`,
@ -58,42 +45,37 @@ func TestCallgraph(t *testing.T) {
`pkg.main --> pkg.main2`,
`pkg.main2 --> (pkg.D).f`,
}},
// tests: both the package's main and the test's main are called.
// The callgraph includes all the guts of the "testing" package.
{"rta", true, []string{
`pkg.test.main --> testing.MainStart`,
`testing.runExample --> pkg.Example`,
// tests: main is not called.
{"rta", format, true, []string{
`pkg.Example --> (pkg.C).f`,
`pkg.main --> (pkg.C).f`,
`test$main.init --> pkg.init`,
}},
{"pta", true, []string{
`<root> --> pkg.test.main`,
`<root> --> pkg.main`,
`pkg.test.main --> testing.MainStart`,
`testing.runExample --> pkg.Example`,
{"pta", format, true, []string{
`<root> --> pkg.Example`,
`<root> --> test$main.init`,
`pkg.Example --> (pkg.C).f`,
`pkg.main --> (pkg.C).f`,
`test$main.init --> pkg.init`,
}},
} {
const format = "{{.Caller}} --> {{.Callee}}"
stdout = new(bytes.Buffer)
if err := doCallgraph("testdata/src", gopath, test.algo, format, test.tests, []string{"pkg"}); err != nil {
if err := doCallgraph(&ctxt, test.algo, test.format, test.tests, []string{"pkg"}); err != nil {
t.Error(err)
continue
}
edges := make(map[string]bool)
for _, line := range strings.Split(fmt.Sprint(stdout), "\n") {
edges[line] = true
}
for _, edge := range test.want {
if !edges[edge] {
t.Errorf("callgraph(%q, %t): missing edge: %s",
test.algo, test.tests, edge)
}
}
if t.Failed() {
t.Log("got:\n", stdout)
got := sortedLines(fmt.Sprint(stdout))
if !reflect.DeepEqual(got, test.want) {
t.Errorf("callgraph(%q, %q, %t):\ngot:\n%s\nwant:\n%s",
test.algo, test.format, test.tests,
strings.Join(got, "\n"),
strings.Join(test.want, "\n"))
}
}
}
func sortedLines(s string) []string {
s = strings.TrimSpace(s)
lines := strings.Split(s, "\n")
sort.Strings(lines)
return lines
}

5
cmd/callgraph/testdata/src/pkg/pkg_test.go vendored
View File

@ -1,10 +1,7 @@
package main
// An Example function must have an "Output:" comment for the go build
// system to generate a call to it from the test main package.
// Don't import "testing", it adds a lot of callgraph edges.
func Example() {
C(0).f()
// Output:
}

504
cmd/compilebench/main.go
View File

@ -1,504 +0,0 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Compilebench benchmarks the speed of the Go compiler.
//
// Usage:
//
// compilebench [options]
//
// It times the compilation of various packages and prints results in
// the format used by package testing (and expected by golang.org/x/perf/cmd/benchstat).
//
// The options are:
//
// -alloc
// Report allocations.
//
// -compile exe
// Use exe as the path to the cmd/compile binary.
//
// -compileflags 'list'
// Pass the space-separated list of flags to the compilation.
//
// -link exe
// Use exe as the path to the cmd/link binary.
//
// -linkflags 'list'
// Pass the space-separated list of flags to the linker.
//
// -count n
// Run each benchmark n times (default 1).
//
// -cpuprofile file
// Write a CPU profile of the compiler to file.
//
// -go path
// Path to "go" command (default "go").
//
// -memprofile file
// Write a memory profile of the compiler to file.
//
// -memprofilerate rate
// Set runtime.MemProfileRate during compilation.
//
// -obj
// Report object file statistics.
//
// -pkg pkg
// Benchmark compiling a single package.
//
// -run regexp
// Only run benchmarks with names matching regexp.
//
// -short
// Skip long-running benchmarks.
//
// Although -cpuprofile and -memprofile are intended to write a
// combined profile for all the executed benchmarks to file,
// today they write only the profile for the last benchmark executed.
//
// The default memory profiling rate is one profile sample per 512 kB
// allocated (see ``go doc runtime.MemProfileRate'').
// Lowering the rate (for example, -memprofilerate 64000) produces
// a more fine-grained and therefore accurate profile, but it also incurs
// execution cost. For benchmark comparisons, never use timings
// obtained with a low -memprofilerate option.
//
// Example
//
// Assuming the base version of the compiler has been saved with
// ``toolstash save,'' this sequence compares the old and new compiler:
//
// compilebench -count 10 -compile $(toolstash -n compile) >old.txt
// compilebench -count 10 >new.txt
// benchstat old.txt new.txt
//
package main
import (
"bytes"
"encoding/json"
"flag"
"fmt"
"io/ioutil"
"log"
"os"
"os/exec"
"path/filepath"
"regexp"
"strconv"
"strings"
"time"
)
var (
goroot string
compiler string
linker string
runRE *regexp.Regexp
is6g bool
)
var (
flagGoCmd = flag.String("go", "go", "path to \"go\" command")
flagAlloc = flag.Bool("alloc", false, "report allocations")
flagObj = flag.Bool("obj", false, "report object file stats")
flagCompiler = flag.String("compile", "", "use `exe` as the cmd/compile binary")
flagCompilerFlags = flag.String("compileflags", "", "additional `flags` to pass to compile")
flagLinker = flag.String("link", "", "use `exe` as the cmd/link binary")
flagLinkerFlags = flag.String("linkflags", "", "additional `flags` to pass to link")
flagRun = flag.String("run", "", "run benchmarks matching `regexp`")
flagCount = flag.Int("count", 1, "run benchmarks `n` times")
flagCpuprofile = flag.String("cpuprofile", "", "write CPU profile to `file`")
flagMemprofile = flag.String("memprofile", "", "write memory profile to `file`")
flagMemprofilerate = flag.Int64("memprofilerate", -1, "set memory profile `rate`")
flagPackage = flag.String("pkg", "", "if set, benchmark the package at path `pkg`")
flagShort = flag.Bool("short", false, "skip long-running benchmarks")
)
type test struct {
name string
r runner
}
type runner interface {
long() bool
run(name string, count int) error
}
var tests = []test{
{"BenchmarkTemplate", compile{"html/template"}},
{"BenchmarkUnicode", compile{"unicode"}},
{"BenchmarkGoTypes", compile{"go/types"}},
{"BenchmarkCompiler", compile{"cmd/compile/internal/gc"}},
{"BenchmarkSSA", compile{"cmd/compile/internal/ssa"}},
{"BenchmarkFlate", compile{"compress/flate"}},
{"BenchmarkGoParser", compile{"go/parser"}},
{"BenchmarkReflect", compile{"reflect"}},
{"BenchmarkTar", compile{"archive/tar"}},
{"BenchmarkXML", compile{"encoding/xml"}},
{"BenchmarkLinkCompiler", link{"cmd/compile"}},
{"BenchmarkStdCmd", goBuild{[]string{"std", "cmd"}}},
{"BenchmarkHelloSize", size{"$GOROOT/test/helloworld.go", false}},
{"BenchmarkCmdGoSize", size{"cmd/go", true}},
}
func usage() {
fmt.Fprintf(os.Stderr, "usage: compilebench [options]\n")
fmt.Fprintf(os.Stderr, "options:\n")
flag.PrintDefaults()
os.Exit(2)
}
func main() {
log.SetFlags(0)
log.SetPrefix("compilebench: ")
flag.Usage = usage
flag.Parse()
if flag.NArg() != 0 {
usage()
}
s, err := exec.Command(*flagGoCmd, "env", "GOROOT").CombinedOutput()
if err != nil {
log.Fatalf("%s env GOROOT: %v", *flagGoCmd, err)
}
goroot = strings.TrimSpace(string(s))
os.Setenv("GOROOT", goroot) // for any subcommands
compiler = *flagCompiler
if compiler == "" {
var foundTool string
foundTool, compiler = toolPath("compile", "6g")
if foundTool == "6g" {
is6g = true
}
}
linker = *flagLinker
if linker == "" && !is6g { // TODO: Support 6l
_, linker = toolPath("link")
}
if is6g {
*flagMemprofilerate = -1
*flagAlloc = false
*flagCpuprofile = ""
*flagMemprofile = ""
}
if *flagRun != "" {
r, err := regexp.Compile(*flagRun)
if err != nil {
log.Fatalf("invalid -run argument: %v", err)
}
runRE = r
}
if *flagPackage != "" {
tests = []test{
{"BenchmarkPkg", compile{*flagPackage}},
{"BenchmarkPkgLink", link{*flagPackage}},
}
runRE = nil
}
for i := 0; i < *flagCount; i++ {
for _, tt := range tests {
if tt.r.long() && *flagShort {
continue
}
if runRE == nil || runRE.MatchString(tt.name) {
if err := tt.r.run(tt.name, i); err != nil {
log.Printf("%s: %v", tt.name, err)
}
}
}
}
}
func toolPath(names ...string) (found, path string) {
var out1 []byte
var err1 error
for i, name := range names {
out, err := exec.Command(*flagGoCmd, "tool", "-n", name).CombinedOutput()
if err == nil {
return name, strings.TrimSpace(string(out))
}
if i == 0 {
out1, err1 = out, err
}
}
log.Fatalf("go tool -n %s: %v\n%s", names[0], err1, out1)
return "", ""
}
type Pkg struct {
Dir string
GoFiles []string
}
func goList(dir string) (*Pkg, error) {
var pkg Pkg
out, err := exec.Command(*flagGoCmd, "list", "-json", dir).Output()
if err != nil {
return nil, fmt.Errorf("go list -json %s: %v\n", dir, err)
}
if err := json.Unmarshal(out, &pkg); err != nil {
return nil, fmt.Errorf("go list -json %s: unmarshal: %v", dir, err)
}
return &pkg, nil
}
func runCmd(name string, cmd *exec.Cmd) error {
start := time.Now()
out, err := cmd.CombinedOutput()
if err != nil {
return fmt.Errorf("%v\n%s", err, out)
}
fmt.Printf("%s 1 %d ns/op\n", name, time.Since(start).Nanoseconds())
return nil
}
type goBuild struct{ pkgs []string }
func (goBuild) long() bool { return true }
func (r goBuild) run(name string, count int) error {
args := []string{"build", "-a"}
if *flagCompilerFlags != "" {
args = append(args, "-gcflags", *flagCompilerFlags)
}
args = append(args, r.pkgs...)
cmd := exec.Command(*flagGoCmd, args...)
cmd.Dir = filepath.Join(goroot, "src")
return runCmd(name, cmd)
}
type size struct {
// path is either a path to a file ("$GOROOT/test/helloworld.go") or a package path ("cmd/go").
path string
isLong bool
}
func (r size) long() bool { return r.isLong }
func (r size) run(name string, count int) error {
if strings.HasPrefix(r.path, "$GOROOT/") {
r.path = goroot + "/" + r.path[len("$GOROOT/"):]
}
cmd := exec.Command(*flagGoCmd, "build", "-o", "_compilebenchout_", r.path)
cmd.Stdout = os.Stderr
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
return err
}
defer os.Remove("_compilebenchout_")
info, err := os.Stat("_compilebenchout_")
if err != nil {
return err
}
out, err := exec.Command("size", "_compilebenchout_").CombinedOutput()
if err != nil {
return fmt.Errorf("size: %v\n%s", err, out)
}
lines := strings.Split(string(out), "\n")
if len(lines) < 2 {
return fmt.Errorf("not enough output from size: %s", out)
}
f := strings.Fields(lines[1])
if strings.HasPrefix(lines[0], "__TEXT") && len(f) >= 2 { // OS X
fmt.Printf("%s 1 %s text-bytes %s data-bytes %v exe-bytes\n", name, f[0], f[1], info.Size())
} else if strings.Contains(lines[0], "bss") && len(f) >= 3 {
fmt.Printf("%s 1 %s text-bytes %s data-bytes %s bss-bytes %v exe-bytes\n", name, f[0], f[1], f[2], info.Size())
}
return nil
}
type compile struct{ dir string }
func (compile) long() bool { return false }
func (c compile) run(name string, count int) error {
// Make sure dependencies needed by go tool compile are installed to GOROOT/pkg.
out, err := exec.Command(*flagGoCmd, "build", "-i", c.dir).CombinedOutput()
if err != nil {
return fmt.Errorf("go build -i %s: %v\n%s", c.dir, err, out)
}
// Find dir and source file list.
pkg, err := goList(c.dir)
if err != nil {
return err
}
args := []string{"-o", "_compilebench_.o"}
args = append(args, strings.Fields(*flagCompilerFlags)...)
args = append(args, pkg.GoFiles...)
if err := runBuildCmd(name, count, pkg.Dir, compiler, args); err != nil {
return err
}
opath := pkg.Dir + "/_compilebench_.o"
if *flagObj {
// TODO(josharian): object files are big; just read enough to find what we seek.
data, err := ioutil.ReadFile(opath)
if err != nil {
log.Print(err)
}
// Find start of export data.
i := bytes.Index(data, []byte("\n$$B\n")) + len("\n$$B\n")
// Count bytes to end of export data.
nexport := bytes.Index(data[i:], []byte("\n$$\n"))
fmt.Printf(" %d object-bytes %d export-bytes", len(data), nexport)
}
fmt.Println()
os.Remove(opath)
return nil
}
type link struct{ dir string }
func (link) long() bool { return false }
func (r link) run(name string, count int) error {
if linker == "" {
// No linker. Skip the test.
return nil
}
// Build dependencies.
out, err := exec.Command(*flagGoCmd, "build", "-i", "-o", "/dev/null", r.dir).CombinedOutput()
if err != nil {
return fmt.Errorf("go build -i %s: %v\n%s", r.dir, err, out)
}
// Build the main package.
pkg, err := goList(r.dir)
if err != nil {
return err
}
args := []string{"-o", "_compilebench_.o"}
args = append(args, pkg.GoFiles...)
cmd := exec.Command(compiler, args...)
cmd.Dir = pkg.Dir
cmd.Stdout = os.Stderr
cmd.Stderr = os.Stderr
err = cmd.Run()
if err != nil {
return fmt.Errorf("compiling: %v", err)
}
defer os.Remove(pkg.Dir + "/_compilebench_.o")
// Link the main package.
args = []string{"-o", "_compilebench_.exe"}
args = append(args, strings.Fields(*flagLinkerFlags)...)
args = append(args, "_compilebench_.o")
if err := runBuildCmd(name, count, pkg.Dir, linker, args); err != nil {
return err
}
fmt.Println()
defer os.Remove(pkg.Dir + "/_compilebench_.exe")
return err
}
// runBuildCmd runs "tool args..." in dir, measures standard build
// tool metrics, and prints a benchmark line. The caller may print
// additional metrics and then must print a newline.
//
// This assumes tool accepts standard build tool flags like
// -memprofilerate, -memprofile, and -cpuprofile.
func runBuildCmd(name string, count int, dir, tool string, args []string) error {
var preArgs []string
if *flagMemprofilerate >= 0 {
preArgs = append(preArgs, "-memprofilerate", fmt.Sprint(*flagMemprofilerate))
}
if *flagAlloc || *flagCpuprofile != "" || *flagMemprofile != "" {
if *flagAlloc || *flagMemprofile != "" {
preArgs = append(preArgs, "-memprofile", "_compilebench_.memprof")
}
if *flagCpuprofile != "" {
preArgs = append(preArgs, "-cpuprofile", "_compilebench_.cpuprof")
}
}
cmd := exec.Command(tool, append(preArgs, args...)...)
cmd.Dir = dir
cmd.Stdout = os.Stderr
cmd.Stderr = os.Stderr
start := time.Now()
err := cmd.Run()
if err != nil {
return err
}
end := time.Now()
haveAllocs := false
var allocs, allocbytes int64
if *flagAlloc || *flagMemprofile != "" {
out, err := ioutil.ReadFile(dir + "/_compilebench_.memprof")
if err != nil {
log.Print("cannot find memory profile after compilation")
}
for _, line := range strings.Split(string(out), "\n") {
f := strings.Fields(line)
if len(f) < 4 || f[0] != "#" || f[2] != "=" {
continue
}
val, err := strconv.ParseInt(f[3], 0, 64)
if err != nil {
continue
}
haveAllocs = true
switch f[1] {
case "TotalAlloc":
allocbytes = val
case "Mallocs":
allocs = val
}
}
if !haveAllocs {
log.Println("missing stats in memprof (golang.org/issue/18641)")
}
if *flagMemprofile != "" {
outpath := *flagMemprofile
if *flagCount != 1 {
outpath = fmt.Sprintf("%s_%d", outpath, count)
}
if err := ioutil.WriteFile(outpath, out, 0666); err != nil {
log.Print(err)
}
}
os.Remove(dir + "/_compilebench_.memprof")
}
if *flagCpuprofile != "" {
out, err := ioutil.ReadFile(dir + "/_compilebench_.cpuprof")
if err != nil {
log.Print(err)
}
outpath := *flagCpuprofile
if *flagCount != 1 {
outpath = fmt.Sprintf("%s_%d", outpath, count)
}
if err := ioutil.WriteFile(outpath, out, 0666); err != nil {
log.Print(err)
}
os.Remove(dir + "/_compilebench_.cpuprof")
}
wallns := end.Sub(start).Nanoseconds()
userns := cmd.ProcessState.UserTime().Nanoseconds()
fmt.Printf("%s 1 %d ns/op %d user-ns/op", name, wallns, userns)
if haveAllocs {
fmt.Printf(" %d B/op %d allocs/op", allocbytes, allocs)
}
return nil
}

2
cmd/cover/cover.go
View File

@ -374,7 +374,7 @@ func trimComments(file *ast.File, fset *token.FileSet) []*ast.CommentGroup {
}
}
if list != nil {
comments = append(comments, &ast.CommentGroup{List: list})
comments = append(comments, &ast.CommentGroup{list})
}
}
return comments

3
cmd/cover/html.go
View File

@ -67,9 +67,6 @@ func htmlOutput(profile, outfile string) error {
} else {
out, err = os.Create(outfile)
}
if err != nil {
return err
}
err = htmlTemplate.Execute(out, d)
if err == nil {
err = out.Close()

245
cmd/digraph/digraph.go
View File

@ -1,83 +1,18 @@
// Copyright 2019 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
/*
The digraph command performs queries over unlabelled directed graphs
represented in text form. It is intended to integrate nicely with
typical UNIX command pipelines.
Usage:
your-application | digraph [command]
The support commands are:
nodes
the set of all nodes
degree
the in-degree and out-degree of each node
preds <node> ...
the set of immediate predecessors of the specified nodes
succs <node> ...
the set of immediate successors of the specified nodes
forward <node> ...
the set of nodes transitively reachable from the specified nodes
reverse <node> ...
the set of nodes that transitively reach the specified nodes
somepath <node> <node>
the list of nodes on some arbitrary path from the first node to the second
allpaths <node> <node>
the set of nodes on all paths from the first node to the second
sccs
all strongly connected components (one per line)
scc <node>
the set of nodes nodes strongly connected to the specified one
Input format:
Each line contains zero or more words. Words are separated by unquoted
whitespace; words may contain Go-style double-quoted portions, allowing spaces
and other characters to be expressed.
Each word declares a node, and if there are more than one, an edge from the
first to each subsequent one. The graph is provided on the standard input.
For instance, the following (acyclic) graph specifies a partial order among the
subtasks of getting dressed:
$ cat clothes.txt
socks shoes
"boxer shorts" pants
pants belt shoes
shirt tie sweater
sweater jacket
hat
The line "shirt tie sweater" indicates the two edges shirt -> tie and
shirt -> sweater, not shirt -> tie -> sweater.
Example usage:
Using digraph with existing Go tools:
$ go mod graph | digraph nodes # Operate on the Go module graph.
$ go list -m all | digraph nodes # Operate on the Go package graph.
Show the transitive closure of imports of the digraph tool itself:
$ go list -f '{{.ImportPath}} {{join .Imports " "}}' ... | digraph forward golang.org/x/tools/cmd/digraph
Show which clothes (see above) must be donned before a jacket:
$ digraph reverse jacket
*/
package main // import "golang.org/x/tools/cmd/digraph"
// The digraph command performs queries over unlabelled directed graphs
// represented in text form. It is intended to integrate nicely with
// typical UNIX command pipelines.
//
// Since directed graphs (import graphs, reference graphs, call graphs,
// etc) often arise during software tool development and debugging, this
// command is included in the go.tools repository.
//
// TODO(adonovan):
// - support input files other than stdin
// - support alternative formats (AT&T GraphViz, CSV, etc),
// - suport alternative formats (AT&T GraphViz, CSV, etc),
// a comment syntax, etc.
// - allow queries to nest, like Blaze query language.
//
package main // import "golang.org/x/tools/cmd/digraph"
import (
"bufio"
@ -93,41 +28,73 @@ import (
"unicode/utf8"
)
func usage() {
fmt.Fprintf(os.Stderr, `Usage: your-application | digraph [command]
const Usage = `digraph: queries over directed graphs in text form.
The support commands are:
nodes
the set of all nodes
degree
the in-degree and out-degree of each node
preds <node> ...
the set of immediate predecessors of the specified nodes
succs <node> ...
the set of immediate successors of the specified nodes
forward <node> ...
the set of nodes transitively reachable from the specified nodes
reverse <node> ...
the set of nodes that transitively reach the specified nodes
somepath <node> <node>
the list of nodes on some arbitrary path from the first node to the second
allpaths <node> <node>
the set of nodes on all paths from the first node to the second
sccs
all strongly connected components (one per line)
scc <node>
the set of nodes nodes strongly connected to the specified one
`)
os.Exit(2)
}
Graph format:
Each line contains zero or more words. Words are separated by
unquoted whitespace; words may contain Go-style double-quoted portions,
allowing spaces and other characters to be expressed.
Each field declares a node, and if there are more than one,
an edge from the first to each subsequent one.
The graph is provided on the standard input.
For instance, the following (acyclic) graph specifies a partial order
among the subtasks of getting dressed:
% cat clothes.txt
socks shoes
"boxer shorts" pants
pants belt shoes
shirt tie sweater
sweater jacket
hat
The line "shirt tie sweater" indicates the two edges shirt -> tie and
shirt -> sweater, not shirt -> tie -> sweater.
Supported queries:
nodes
the set of all nodes
degree
the in-degree and out-degree of each node.
preds <label> ...
the set of immediate predecessors of the specified nodes
succs <label> ...
the set of immediate successors of the specified nodes
forward <label> ...
the set of nodes transitively reachable from the specified nodes
reverse <label> ...
the set of nodes that transitively reach the specified nodes
somepath <label> <label>
the list of nodes on some arbitrary path from the first node to the second
allpaths <label> <label>
the set of nodes on all paths from the first node to the second
sccs
all strongly connected components (one per line)
scc <label>
the set of nodes nodes strongly connected to the specified one
Example usage:
Show the transitive closure of imports of the digraph tool itself:
% go list -f '{{.ImportPath}}{{.Imports}}' ... | tr '[]' ' ' |
digraph forward golang.org/x/tools/cmd/digraph
Show which clothes (see above) must be donned before a jacket:
% digraph reverse jacket <clothes.txt
`
func main() {
flag.Usage = usage
flag.Parse()
args := flag.Args()
if len(args) == 0 {
usage()
fmt.Println(Usage)
return
}
if err := digraph(args[0], args[1:]); err != nil {
@ -262,47 +229,6 @@ func (g graph) sccs() []nodeset {
return sccs
}
func (g graph) allpaths(from, to string) error {
// Mark all nodes to "to".
seen := make(nodeset) // value of seen[x] indicates whether x is on some path to "to"
var visit func(node string) bool
visit = func(node string) bool {
reachesTo, ok := seen[node]
if !ok {
reachesTo = node == to
seen[node] = reachesTo
for e := range g[node] {
if visit(e) {
reachesTo = true
}
}
if reachesTo && node != to {
seen[node] = true
}
}
return reachesTo
}
visit(from)
// For each marked node, collect its marked successors.
var edges []string
for n := range seen {
for succ := range g[n] {
if seen[succ] {
edges = append(edges, n+" "+succ)
}
}
}
// Sort (so that this method is deterministic) and print edges.
sort.Strings(edges)
for _, e := range edges {
fmt.Fprintln(stdout, e)
}
return nil
}
func parse(rd io.Reader) (graph, error) {
g := make(graph)
@ -325,7 +251,6 @@ func parse(rd io.Reader) (graph, error) {
return g, nil
}
// Overridable for testing purposes.
var stdin io.Reader = os.Stdin
var stdout io.Writer = os.Stdout
@ -440,7 +365,33 @@ func digraph(cmd string, args []string) error {
if g[to] == nil {
return fmt.Errorf("no such 'to' node %q", to)
}
g.allpaths(from, to)
seen := make(nodeset) // value of seen[x] indicates whether x is on some path to 'to'
var visit func(label string) bool
visit = func(label string) bool {
reachesTo, ok := seen[label]
if !ok {
reachesTo = label == to
seen[label] = reachesTo
for e := range g[label] {
if visit(e) {
reachesTo = true
}
}
seen[label] = reachesTo
}
return reachesTo
}
if !visit(from) {
return fmt.Errorf("no path from %q to %q", from, to)
}
for label, reachesTo := range seen {
if !reachesTo {
delete(seen, label)
}
}
seen.sort().println("\n")
case "sccs":
if len(args) != 0 {

152
cmd/digraph/digraph_test.go
View File

@ -1,6 +1,3 @@
// Copyright 2019 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
import (
@ -29,34 +26,35 @@ d c
`
for _, test := range []struct {
name string
input string
cmd string
args []string
want string
}{
{"nodes", g1, "nodes", nil, "belt\nhat\njacket\npants\nshirt\nshoes\nshorts\nsocks\nsweater\ntie\n"},
{"reverse", g1, "reverse", []string{"jacket"}, "jacket\nshirt\nsweater\n"},
{"forward", g1, "forward", []string{"socks"}, "shoes\nsocks\n"},
{"forward multiple args", g1, "forward", []string{"socks", "sweater"}, "jacket\nshoes\nsocks\nsweater\n"},
{"scss", g2, "sccs", nil, "a\nb\nc d\n"},
{"scc", g2, "scc", []string{"d"}, "c\nd\n"},
{"succs", g2, "succs", []string{"a"}, "b\nc\n"},
{"preds", g2, "preds", []string{"c"}, "a\nd\n"},
{"preds multiple args", g2, "preds", []string{"c", "d"}, "a\nb\nc\nd\n"},
} {
t.Run(test.name, func(t *testing.T) {
stdin = strings.NewReader(test.input)
stdout = new(bytes.Buffer)
if err := digraph(test.cmd, test.args); err != nil {
t.Fatal(err)
}
{g1, "nodes", nil, "belt\nhat\njacket\npants\nshirt\nshoes\nshorts\nsocks\nsweater\ntie\n"},
{g1, "reverse", []string{"jacket"}, "jacket\nshirt\nsweater\n"},
{g1, "forward", []string{"socks"}, "shoes\nsocks\n"},
{g1, "forward", []string{"socks", "sweater"}, "jacket\nshoes\nsocks\nsweater\n"},
got := stdout.(fmt.Stringer).String()
if got != test.want {
t.Errorf("digraph(%s, %s) = got %q, want %q", test.cmd, test.args, got, test.want)
}
})
{g2, "allpaths", []string{"a", "d"}, "a\nb\nc\nd\n"},
{g2, "sccs", nil, "a\nb\nc d\n"},
{g2, "scc", []string{"d"}, "c\nd\n"},
{g2, "succs", []string{"a"}, "b\nc\n"},
{g2, "preds", []string{"c"}, "a\nd\n"},
{g2, "preds", []string{"c", "d"}, "a\nb\nc\nd\n"},
} {
stdin = strings.NewReader(test.input)
stdout = new(bytes.Buffer)
if err := digraph(test.cmd, test.args); err != nil {
t.Error(err)
continue
}
got := stdout.(fmt.Stringer).String()
if got != test.want {
t.Errorf("digraph(%s, %s) = %q, want %q", test.cmd, test.args, got, test.want)
}
}
// TODO(adonovan):
@ -64,110 +62,6 @@ d c
// - test errors
}
func TestAllpaths(t *testing.T) {
for _, test := range []struct {
name string
in string
to string // from is always "A"
want string
}{
{
name: "Basic",
in: "A B\nB C",
to: "B",
want: "A B\n",
},
{
name: "Long",
in: "A B\nB C\n",
to: "C",
want: "A B\nB C\n",
},
{
name: "Cycle Basic",
in: "A B\nB A",
to: "B",
want: "A B\nB A\n",
},
{
name: "Cycle Path Out",
// A <-> B -> C -> D
in: "A B\nB A\nB C\nC D",
to: "C",
want: "A B\nB A\nB C\n",
},
{
name: "Cycle Path Out Further Out",
// A -> B <-> C -> D -> E
in: "A B\nB C\nC D\nC B\nD E",
to: "D",
want: "A B\nB C\nC B\nC D\n",
},
{
name: "Two Paths Basic",
// /-> C --\
// A -> B -- -> E -> F
// \-> D --/
in: "A B\nB C\nC E\nB D\nD E\nE F",
to: "E",
want: "A B\nB C\nB D\nC E\nD E\n",
},
{
name: "Two Paths With One Immediately From Start",
// /-> B -+ -> D
// A -- |
// \-> C <+
in: "A B\nA C\nB C\nB D",
to: "C",
want: "A B\nA C\nB C\n",
},
{
name: "Two Paths Further Up",
// /-> B --\
// A -- -> D -> E -> F
// \-> C --/
in: "A B\nA C\nB D\nC D\nD E\nE F",
to: "E",
want: "A B\nA C\nB D\nC D\nD E\n",
},
{
// We should include A - C - D even though it's further up the
// second path than D (which would already be in the graph by
// the time we get around to integrating the second path).
name: "Two Splits",
// /-> B --\ /-> E --\
// A -- -> D -- -> G -> H
// \-> C --/ \-> F --/
in: "A B\nA C\nB D\nC D\nD E\nD F\nE G\nF G\nG H",
to: "G",
want: "A B\nA C\nB D\nC D\nD E\nD F\nE G\nF G\n",
},
{
// D - E should not be duplicated.
name: "Two Paths - Two Splits With Gap",
// /-> B --\ /-> F --\
// A -- -> D -> E -- -> H -> I
// \-> C --/ \-> G --/
in: "A B\nA C\nB D\nC D\nD E\nE F\nE G\nF H\nG H\nH I",
to: "H",
want: "A B\nA C\nB D\nC D\nD E\nE F\nE G\nF H\nG H\n",
},
} {
t.Run(test.name, func(t *testing.T) {
stdin = strings.NewReader(test.in)
stdout = new(bytes.Buffer)
if err := digraph("allpaths", []string{"A", test.to}); err != nil {
t.Fatal(err)
}
got := stdout.(fmt.Stringer).String()
if got != test.want {
t.Errorf("digraph(allpaths, A, %s) = got %q, want %q", test.to, got, test.want)
}
})
}
}
func TestSplit(t *testing.T) {
for _, test := range []struct {
line string

4
cmd/eg/eg.go
View File

@ -7,8 +7,8 @@ import (
"flag"
"fmt"
"go/build"
"go/format"
"go/parser"
"go/printer"
"go/token"
"os"
"os/exec"
@ -139,7 +139,7 @@ func doMain() error {
hadErrors = true
}
} else {
format.Node(os.Stdout, iprog.Fset, file)
printer.Fprint(os.Stdout, iprog.Fset, file)
}
}
}

15
cmd/fiximports/main.go
View File

@ -491,18 +491,15 @@ func list(args ...string) ([]*listPackage, error) {
return pkgs, nil
}
// cwd contains the current working directory of the tool.
//
// It is initialized directly so that its value will be set for any other
// package variables or init functions that depend on it, such as the gopath
// variable in main_test.go.
var cwd string = func() string {
cwd, err := os.Getwd()
var cwd string
func init() {
var err error
cwd, err = os.Getwd()
if err != nil {
log.Fatalf("os.Getwd: %v", err)
}
return cwd
}()
}
// shortPath returns an absolute or relative name for path, whatever is shorter.
// Plundered from $GOROOT/src/cmd/go/build.go.

33
cmd/fiximports/main_test.go
View File

@ -10,7 +10,6 @@ package main
import (
"bytes"
"log"
"os"
"path/filepath"
"runtime"
@ -33,25 +32,11 @@ import (
// titanic.biz/bar -- domain is sinking; package has jumped ship to new.com/bar
// titanic.biz/foo -- domain is sinking but package has no import comment yet
var gopath = filepath.Join(cwd, "testdata")
func init() {
if err := os.Setenv("GOPATH", gopath); err != nil {
log.Fatal(err)
}
// This test currently requires GOPATH mode.
// Explicitly disabling module mode should suffix, but
// we'll also turn off GOPROXY just for good measure.
if err := os.Setenv("GO111MODULE", "off"); err != nil {
log.Fatal(err)
}
if err := os.Setenv("GOPROXY", "off"); err != nil {
log.Fatal(err)
}
}
func TestFixImports(t *testing.T) {
gopath := filepath.Join(cwd, "testdata")
if err := os.Setenv("GOPATH", gopath); err != nil {
t.Fatalf("os.Setenv: %v", err)
}
defer func() {
stderr = os.Stderr
*badDomains = "code.google.com"
@ -215,10 +200,7 @@ import (
}
// Compare stderr output.
if got := stderr.(*bytes.Buffer).String(); got != test.wantStderr {
if strings.Contains(got, "vendor/golang_org/x/text/unicode/norm") {
t.Skip("skipping known-broken test; see golang.org/issue/17417")
}
if stderr.(*bytes.Buffer).String() != test.wantStderr {
t.Errorf("#%d. stderr: got <<%s>>, want <<%s>>",
i, stderr, test.wantStderr)
}
@ -239,6 +221,11 @@ import (
// TestDryRun tests that the -n flag suppresses calls to writeFile.
func TestDryRun(t *testing.T) {
gopath := filepath.Join(cwd, "testdata")
if err := os.Setenv("GOPATH", gopath); err != nil {
t.Fatalf("os.Setenv: %v", err)
}
*dryrun = true
defer func() { *dryrun = false }() // restore
stderr = new(bytes.Buffer)

5
cmd/getgo/.dockerignore
View File

@ -1,5 +0,0 @@
.git
.dockerignore
LICENSE
README.md
.gitignore

3
cmd/getgo/.gitignore vendored
View File

@ -1,3 +0,0 @@
build
testgetgo
getgo

20
cmd/getgo/Dockerfile
View File

@ -1,20 +0,0 @@
FROM golang:latest
ENV SHELL /bin/bash
ENV HOME /root
WORKDIR $HOME
COPY . /go/src/golang.org/x/tools/cmd/getgo
RUN ( \
cd /go/src/golang.org/x/tools/cmd/getgo \
&& go build \
&& mv getgo /usr/local/bin/getgo \
)
# undo the adding of GOPATH to env for testing
ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ENV GOPATH ""
# delete /go and /usr/local/go for testing
RUN rm -rf /go /usr/local/go

27
cmd/getgo/LICENSE
View File

@ -1,27 +0,0 @@
Copyright (c) 2017 The Go Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

71
cmd/getgo/README.md
View File

@ -1,71 +0,0 @@
# getgo
A proof-of-concept command-line installer for Go.
This installer is designed to both install Go as well as do the initial configuration
of setting up the right environment variables and paths.
It will install the Go distribution (tools & stdlib) to "/.go" inside your home directory by default.
It will setup "$HOME/go" as your GOPATH.
This is where third party libraries and apps will be installed as well as where you will write your Go code.
If Go is already installed via this installer it will upgrade it to the latest version of Go.
Currently the installer supports Windows, \*nix and macOS on x86 & x64.
It supports Bash and Zsh on all of these platforms as well as powershell & cmd.exe on Windows.
## Usage
Windows Powershell/cmd.exe:
`(New-Object System.Net.WebClient).DownloadFile('https://get.golang.org/installer.exe', 'installer.exe'); Start-Process -Wait -NonewWindow installer.exe; Remove-Item installer.exe`
Shell (Linux/macOS/Windows):
`curl -LO https://get.golang.org/$(uname)/go_installer && chmod +x go_installer && ./go_installer && rm go_installer`
## To Do
* Check if Go is already installed (via a different method) and update it in place or at least notify the user
* Lots of testing. It's only had limited testing so far.
* Add support for additional shells.
## Development instructions
### Testing
There are integration tests in [`main_test.go`](main_test.go). Please add more
tests there.
#### On unix/linux with the Dockerfile
The Dockerfile automatically builds the binary, moves it to
`/usr/local/bin/getgo` and then unsets `$GOPATH` and removes all `$GOPATH` from
`$PATH`.
```bash
$ docker build --rm --force-rm -t getgo .
...
$ docker run --rm -it getgo bash
root@78425260fad0:~# getgo -v
Welcome to the Go installer!
Downloading Go version go1.8.3 to /usr/local/go
This may take a bit of time...
Adding "export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin" to /root/.bashrc
Downloaded!
Setting up GOPATH
Adding "export GOPATH=/root/go" to /root/.bashrc
Adding "export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin:/root/go/bin" to /root/.bashrc
GOPATH has been setup!
root@78425260fad0:~# which go
/usr/local/go/bin/go
root@78425260fad0:~# echo $GOPATH
/root/go
root@78425260fad0:~# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin:/root/go/bin
```
## Release instructions
To upload a new release of getgo, run `./make.bash && ./upload.bash`.

184
cmd/getgo/download.go
View File

@ -1,184 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !plan9
package main
import (
"archive/tar"
"archive/zip"
"compress/gzip"
"crypto/sha256"
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
"path/filepath"
"strings"
)
const (
currentVersionURL = "https://golang.org/VERSION?m=text"
downloadURLPrefix = "https://dl.google.com/go"
)
// downloadGoVersion downloads and upacks the specific go version to dest/go.
func downloadGoVersion(version, ops, arch, dest string) error {
suffix := "tar.gz"
if ops == "windows" {
suffix = "zip"
}
uri := fmt.Sprintf("%s/%s.%s-%s.%s", downloadURLPrefix, version, ops, arch, suffix)
verbosef("Downloading %s", uri)
req, err := http.NewRequest("GET", uri, nil)
if err != nil {
return err
}
req.Header.Add("User-Agent", fmt.Sprintf("golang.org-getgo/%s", version))
resp, err := http.DefaultClient.Do(req)
if err != nil {
return fmt.Errorf("Downloading Go from %s failed: %v", uri, err)
}
if resp.StatusCode > 299 {
return fmt.Errorf("Downloading Go from %s failed with HTTP status %s", uri, resp.Status)
}
defer resp.Body.Close()
tmpf, err := ioutil.TempFile("", "go")
if err != nil {
return err
}
defer os.Remove(tmpf.Name())
h := sha256.New()
w := io.MultiWriter(tmpf, h)
if _, err := io.Copy(w, resp.Body); err != nil {
return err
}
verbosef("Downloading SHA %s.sha256", uri)
sresp, err := http.Get(uri + ".sha256")
if err != nil {
return fmt.Errorf("Downloading Go sha256 from %s.sha256 failed: %v", uri, err)
}
defer sresp.Body.Close()
if sresp.StatusCode > 299 {
return fmt.Errorf("Downloading Go sha256 from %s.sha256 failed with HTTP status %s", uri, sresp.Status)
}
shasum, err := ioutil.ReadAll(sresp.Body)
if err != nil {
return err
}
// Check the shasum.
sum := fmt.Sprintf("%x", h.Sum(nil))
if sum != string(shasum) {
return fmt.Errorf("Shasum mismatch %s vs. %s", sum, string(shasum))
}
unpackFunc := unpackTar
if ops == "windows" {
unpackFunc = unpackZip
}
if err := unpackFunc(tmpf.Name(), dest); err != nil {
return fmt.Errorf("Unpacking Go to %s failed: %v", dest, err)
}
return nil
}
func unpack(dest, name string, fi os.FileInfo, r io.Reader) error {
if strings.HasPrefix(name, "go/") {
name = name[len("go/"):]
}
path := filepath.Join(dest, name)
if fi.IsDir() {
return os.MkdirAll(path, fi.Mode())
}
f, err := os.OpenFile(path, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, fi.Mode())
if err != nil {
return err
}
defer f.Close()
_, err = io.Copy(f, r)
return err
}
func unpackTar(src, dest string) error {
r, err := os.Open(src)
if err != nil {
return err
}
defer r.Close()
archive, err := gzip.NewReader(r)
if err != nil {
return err
}
defer archive.Close()
tarReader := tar.NewReader(archive)
for {
header, err := tarReader.Next()
if err == io.EOF {
break
} else if err != nil {
return err
}
if err := unpack(dest, header.Name, header.FileInfo(), tarReader); err != nil {
return err
}
}
return nil
}
func unpackZip(src, dest string) error {
zr, err := zip.OpenReader(src)
if err != nil {
return err
}
for _, f := range zr.File {
fr, err := f.Open()
if err != nil {
return err
}
if err := unpack(dest, f.Name, f.FileInfo(), fr); err != nil {
return err
}
fr.Close()
}
return nil
}
func getLatestGoVersion() (string, error) {
resp, err := http.Get(currentVersionURL)
if err != nil {
return "", fmt.Errorf("Getting current Go version failed: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode > 299 {
b, _ := ioutil.ReadAll(io.LimitReader(resp.Body, 1024))
return "", fmt.Errorf("Could not get current Go version: HTTP %d: %q", resp.StatusCode, b)
}
version, err := ioutil.ReadAll(resp.Body)
if err != nil {
return "", err
}
return strings.TrimSpace(string(version)), nil
}

36
cmd/getgo/download_test.go
View File

@ -1,36 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !plan9
package main
import (
"io/ioutil"
"os"
"path/filepath"
"testing"
)
func TestDownloadGoVersion(t *testing.T) {
if testing.Short() {
t.Skipf("Skipping download in short mode")
}
tmpd, err := ioutil.TempDir("", "go")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpd)
if err := downloadGoVersion("go1.8.1", "linux", "amd64", filepath.Join(tmpd, "go")); err != nil {
t.Fatal(err)
}
// Ensure the VERSION file exists.
vf := filepath.Join(tmpd, "go", "VERSION")
if _, err := os.Stat(vf); os.IsNotExist(err) {
t.Fatalf("file %s does not exist and should", vf)
}
}

117
cmd/getgo/main.go
View File

@ -1,117 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !plan9
// The getgo command installs Go to the user's system.
package main
import (
"bufio"
"context"
"errors"
"flag"
"fmt"
"os"
"os/exec"
"strings"
)
var (
interactive = flag.Bool("i", false, "Interactive mode, prompt for inputs.")
verbose = flag.Bool("v", false, "Verbose.")
setupOnly = flag.Bool("skip-dl", false, "Don't download - only set up environment variables")
goVersion = flag.String("version", "", `Version of Go to install (e.g. "1.8.3"). If empty, uses the latest version.`)
version = "devel"
)
var exitCleanly error = errors.New("exit cleanly sentinel value")
func main() {
flag.Parse()
if *goVersion != "" && !strings.HasPrefix(*goVersion, "go") {
*goVersion = "go" + *goVersion
}
ctx := context.Background()
verbosef("version " + version)
runStep := func(s step) {
err := s(ctx)
if err == exitCleanly {
os.Exit(0)
}
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
}
if !*setupOnly {
runStep(welcome)
runStep(checkOthers)
runStep(chooseVersion)
runStep(downloadGo)
}
runStep(setupGOPATH)
}
func verbosef(format string, v ...interface{}) {
if !*verbose {
return
}
fmt.Printf(format+"\n", v...)
}
func prompt(ctx context.Context, query, defaultAnswer string) (string, error) {
if !*interactive {
return defaultAnswer, nil
}
fmt.Printf("%s [%s]: ", query, defaultAnswer)
type result struct {
answer string
err error
}
ch := make(chan result, 1)
go func() {
s := bufio.NewScanner(os.Stdin)
if !s.Scan() {
ch <- result{"", s.Err()}
return
}
answer := s.Text()
if answer == "" {
answer = defaultAnswer
}
ch <- result{answer, nil}
}()
select {
case r := <-ch:
return r.answer, r.err
case <-ctx.Done():
return "", ctx.Err()
}
}
func runCommand(ctx context.Context, prog string, args ...string) ([]byte, error) {
verbosef("Running command: %s %v", prog, args)
cmd := exec.CommandContext(ctx, prog, args...)
out, err := cmd.CombinedOutput()
if err != nil {
return nil, fmt.Errorf("running cmd '%s %s' failed: %s err: %v", prog, strings.Join(args, " "), string(out), err)
}
if out != nil && err == nil && len(out) != 0 {
verbosef("%s", out)
}
return out, nil
}

173
cmd/getgo/main_test.go
View File

@ -1,173 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !plan9
package main
import (
"bytes"
"fmt"
"io/ioutil"
"os"
"os/exec"
"runtime"
"testing"
)
const (
testbin = "testgetgo"
)
var (
exeSuffix string // ".exe" on Windows
)
func init() {
if runtime.GOOS == "windows" {
exeSuffix = ".exe"
}
}
// TestMain creates a getgo command for testing purposes and
// deletes it after the tests have been run.
func TestMain(m *testing.M) {
if os.Getenv("GOGET_INTEGRATION") == "" {
fmt.Fprintln(os.Stderr, "main_test: Skipping integration tests with GOGET_INTEGRATION unset")
return
}
args := []string{"build", "-tags", testbin, "-o", testbin + exeSuffix}
out, err := exec.Command("go", args...).CombinedOutput()
if err != nil {
fmt.Fprintf(os.Stderr, "building %s failed: %v\n%s", testbin, err, out)
os.Exit(2)
}
// Don't let these environment variables confuse the test.
os.Unsetenv("GOBIN")
os.Unsetenv("GOPATH")
os.Unsetenv("GIT_ALLOW_PROTOCOL")
os.Unsetenv("PATH")
r := m.Run()
os.Remove(testbin + exeSuffix)
os.Exit(r)
}
func createTmpHome(t *testing.T) string {
tmpd, err := ioutil.TempDir("", "testgetgo")
if err != nil {
t.Fatalf("creating test tempdir failed: %v", err)
}
os.Setenv("HOME", tmpd)
return tmpd
}
// doRun runs the test getgo command, recording stdout and stderr and
// returning exit status.
func doRun(t *testing.T, args ...string) error {
var stdout, stderr bytes.Buffer
t.Logf("running %s %v", testbin, args)
cmd := exec.Command("./"+testbin+exeSuffix, args...)
cmd.Stdout = &stdout
cmd.Stderr = &stderr
cmd.Env = os.Environ()
status := cmd.Run()
if stdout.Len() > 0 {
t.Log("standard output:")
t.Log(stdout.String())
}
if stderr.Len() > 0 {
t.Log("standard error:")
t.Log(stderr.String())
}
return status
}
func TestCommandVerbose(t *testing.T) {
tmpd := createTmpHome(t)
defer os.RemoveAll(tmpd)
err := doRun(t, "-v")
if err != nil {
t.Fatal(err)
}
// make sure things are in path
shellConfig, err := shellConfigFile()
if err != nil {
t.Fatal(err)
}
b, err := ioutil.ReadFile(shellConfig)
if err != nil {
t.Fatal(err)
}
home, err := getHomeDir()
if err != nil {
t.Fatal(err)
}
expected := fmt.Sprintf(`
export PATH=$PATH:%s/.go/bin
export GOPATH=%s/go
export PATH=$PATH:%s/go/bin
`, home, home, home)
if string(b) != expected {
t.Fatalf("%s expected %q, got %q", shellConfig, expected, string(b))
}
}
func TestCommandPathExists(t *testing.T) {
tmpd := createTmpHome(t)
defer os.RemoveAll(tmpd)
// run once
err := doRun(t, "-skip-dl")
if err != nil {
t.Fatal(err)
}
// make sure things are in path
shellConfig, err := shellConfigFile()
if err != nil {
t.Fatal(err)
}
b, err := ioutil.ReadFile(shellConfig)
if err != nil {
t.Fatal(err)
}
home, err := getHomeDir()
if err != nil {
t.Fatal(err)
}
expected := fmt.Sprintf(`
export GOPATH=%s/go
export PATH=$PATH:%s/go/bin
`, home, home)
if string(b) != expected {
t.Fatalf("%s expected %q, got %q", shellConfig, expected, string(b))
}
// run twice
if err := doRun(t, "-skip-dl"); err != nil {
t.Fatal(err)
}
b, err = ioutil.ReadFile(shellConfig)
if err != nil {
t.Fatal(err)
}
if string(b) != expected {
t.Fatalf("%s expected %q, got %q", shellConfig, expected, string(b))
}
}

13
cmd/getgo/make.bash
View File

@ -1,13 +0,0 @@
#!/bin/bash
# Copyright 2017 The Go Authors. All rights reserved.
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file.
set -e -o -x
LDFLAGS="-X main.version=$(git describe --always --dirty='*')"
GOOS=windows GOARCH=386 go build -o build/installer.exe -ldflags="$LDFLAGS"
GOOS=linux GOARCH=386 go build -o build/installer_linux -ldflags="$LDFLAGS"
GOOS=darwin GOARCH=386 go build -o build/installer_darwin -ldflags="$LDFLAGS"

155
cmd/getgo/path.go
View File

@ -1,155 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !plan9
package main
import (
"bufio"
"context"
"fmt"
"os"
"os/user"
"path/filepath"
"runtime"
"strings"
)
const (
bashConfig = ".bash_profile"
zshConfig = ".zshrc"
)
// appendToPATH adds the given path to the PATH environment variable and
// persists it for future sessions.
func appendToPATH(value string) error {
if isInPATH(value) {
return nil
}
return persistEnvVar("PATH", pathVar+envSeparator+value)
}
func isInPATH(dir string) bool {
p := os.Getenv("PATH")
paths := strings.Split(p, envSeparator)
for _, d := range paths {
if d == dir {
return true
}
}
return false
}
func getHomeDir() (string, error) {
home := os.Getenv(homeKey)
if home != "" {
return home, nil
}
u, err := user.Current()
if err != nil {
return "", err
}
return u.HomeDir, nil
}
func checkStringExistsFile(filename, value string) (bool, error) {
file, err := os.OpenFile(filename, os.O_RDONLY, 0600)
if err != nil {
if os.IsNotExist(err) {
return false, nil
}
return false, err
}
defer file.Close()
scanner := bufio.NewScanner(file)
for scanner.Scan() {
line := scanner.Text()
if line == value {
return true, nil
}
}
return false, scanner.Err()
}
func appendToFile(filename, value string) error {
verbosef("Adding %q to %s", value, filename)
ok, err := checkStringExistsFile(filename, value)
if err != nil {
return err
}
if ok {
// Nothing to do.
return nil
}
f, err := os.OpenFile(filename, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0600)
if err != nil {
return err
}
defer f.Close()
_, err = f.WriteString(lineEnding + value + lineEnding)
return err
}
func isShell(name string) bool {
return strings.Contains(currentShell(), name)
}
// persistEnvVarWindows sets an environment variable in the Windows
// registry.
func persistEnvVarWindows(name, value string) error {
_, err := runCommand(context.Background(), "powershell", "-command",
fmt.Sprintf(`[Environment]::SetEnvironmentVariable("%s", "%s", "User")`, name, value))
return err
}
func persistEnvVar(name, value string) error {
if runtime.GOOS == "windows" {
if err := persistEnvVarWindows(name, value); err != nil {
return err
}
if isShell("cmd.exe") || isShell("powershell.exe") {
return os.Setenv(strings.ToUpper(name), value)
}
// User is in bash, zsh, etc.
// Also set the environment variable in their shell config.
}
rc, err := shellConfigFile()
if err != nil {
return err
}
line := fmt.Sprintf("export %s=%s", strings.ToUpper(name), value)
if err := appendToFile(rc, line); err != nil {
return err
}
return os.Setenv(strings.ToUpper(name), value)
}
func shellConfigFile() (string, error) {
home, err := getHomeDir()
if err != nil {
return "", err
}
switch {
case isShell("bash"):
return filepath.Join(home, bashConfig), nil
case isShell("zsh"):
return filepath.Join(home, zshConfig), nil
default:
return "", fmt.Errorf("%q is not a supported shell", currentShell())
}
}

58
cmd/getgo/path_test.go
View File

@ -1,58 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !plan9
package main
import (
"io/ioutil"
"os"
"path/filepath"
"strings"
"testing"
)
func TestAppendPath(t *testing.T) {
tmpd, err := ioutil.TempDir("", "go")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpd)
if err := os.Setenv("HOME", tmpd); err != nil {
t.Fatal(err)
}
GOPATH := os.Getenv("GOPATH")
if err := appendToPATH(filepath.Join(GOPATH, "bin")); err != nil {
t.Fatal(err)
}
shellConfig, err := shellConfigFile()
if err != nil {
t.Fatal(err)
}
b, err := ioutil.ReadFile(shellConfig)
if err != nil {
t.Fatal(err)
}
expected := "export PATH=" + pathVar + envSeparator + filepath.Join(GOPATH, "bin")
if strings.TrimSpace(string(b)) != expected {
t.Fatalf("expected: %q, got %q", expected, strings.TrimSpace(string(b)))
}
// Check that appendToPATH is idempotent.
if err := appendToPATH(filepath.Join(GOPATH, "bin")); err != nil {
t.Fatal(err)
}
b, err = ioutil.ReadFile(shellConfig)
if err != nil {
t.Fatal(err)
}
if strings.TrimSpace(string(b)) != expected {
t.Fatalf("expected: %q, got %q", expected, strings.TrimSpace(string(b)))
}
}

7
cmd/getgo/server/README.md
View File

@ -1,7 +0,0 @@
# getgo server
## Deployment
```
gcloud app deploy --promote --project golang-org
```

7
cmd/getgo/server/app.yaml
View File

@ -1,7 +0,0 @@
runtime: go
service: get
api_version: go1
handlers:
- url: /.*
script: _go_app

61
cmd/getgo/server/main.go
View File

@ -1,61 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Command server serves get.golang.org, redirecting users to the appropriate
// getgo installer based on the request path.
package server
import (
"fmt"
"net/http"
"strings"
"time"
)
const (
base = "https://dl.google.com/go/getgo/"
windowsInstaller = base + "installer.exe"
linuxInstaller = base + "installer_linux"
macInstaller = base + "installer_darwin"
)
// substring-based redirects.
var stringMatch = map[string]string{
// via uname, from bash
"MINGW": windowsInstaller, // Reported as MINGW64_NT-10.0 in git bash
"Linux": linuxInstaller,
"Darwin": macInstaller,
}
func init() {
http.HandleFunc("/", handler)
}
func handler(w http.ResponseWriter, r *http.Request) {
if containsIgnoreCase(r.URL.Path, "installer.exe") {
// cache bust
http.Redirect(w, r, windowsInstaller+cacheBust(), http.StatusFound)
return
}
for match, redirect := range stringMatch {
if containsIgnoreCase(r.URL.Path, match) {
http.Redirect(w, r, redirect, http.StatusFound)
return
}
}
http.NotFound(w, r)
}
func containsIgnoreCase(s, substr string) bool {
return strings.Contains(
strings.ToLower(s),
strings.ToLower(substr),
)
}
func cacheBust() string {
return fmt.Sprintf("?%d", time.Now().Nanosecond())
}

133
cmd/getgo/steps.go
View File

@ -1,133 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !plan9
package main
import (
"context"
"fmt"
"os"
"path/filepath"
"runtime"
"strings"
)
type step func(context.Context) error
func welcome(ctx context.Context) error {
fmt.Println("Welcome to the Go installer!")
answer, err := prompt(ctx, "Would you like to install Go? Y/n", "Y")
if err != nil {
return err
}
if strings.ToLower(answer) != "y" {
fmt.Println("Exiting install.")
return exitCleanly
}
return nil
}
func checkOthers(ctx context.Context) error {
// TODO: if go is currently installed install new version over that
path, err := whichGo(ctx)
if err != nil {
fmt.Printf("Cannot check if Go is already installed:\n%v\n", err)
}
if path == "" {
return nil
}
if path != installPath {
fmt.Printf("Go is already installed at %v; remove it from your PATH.\n", path)
}
return nil
}
func chooseVersion(ctx context.Context) error {
if *goVersion != "" {
return nil
}
var err error
*goVersion, err = getLatestGoVersion()
if err != nil {
return err
}
answer, err := prompt(ctx, fmt.Sprintf("The latest Go version is %s, install that? Y/n", *goVersion), "Y")
if err != nil {
return err
}
if strings.ToLower(answer) != "y" {
// TODO: handle passing a version
fmt.Println("Aborting install.")
return exitCleanly
}
return nil
}
func downloadGo(ctx context.Context) error {
answer, err := prompt(ctx, fmt.Sprintf("Download Go version %s to %s? Y/n", *goVersion, installPath), "Y")
if err != nil {
return err
}
if strings.ToLower(answer) != "y" {
fmt.Println("Aborting install.")
return exitCleanly
}
fmt.Printf("Downloading Go version %s to %s\n", *goVersion, installPath)
fmt.Println("This may take a bit of time...")
if err := downloadGoVersion(*goVersion, runtime.GOOS, arch, installPath); err != nil {
return err
}
if err := appendToPATH(filepath.Join(installPath, "bin")); err != nil {
return err
}
fmt.Println("Downloaded!")
return nil
}
func setupGOPATH(ctx context.Context) error {
answer, err := prompt(ctx, "Would you like us to setup your GOPATH? Y/n", "Y")
if err != nil {
return err
}
if strings.ToLower(answer) != "y" {
fmt.Println("Exiting and not setting up GOPATH.")
return exitCleanly
}
fmt.Println("Setting up GOPATH")
home, err := getHomeDir()
if err != nil {
return err
}
gopath := os.Getenv("GOPATH")
if gopath == "" {
// set $GOPATH
gopath = filepath.Join(home, "go")
if err := persistEnvVar("GOPATH", gopath); err != nil {
return err
}
fmt.Println("GOPATH has been set up!")
} else {
verbosef("GOPATH is already set to %s", gopath)
}
if err := appendToPATH(filepath.Join(gopath, "bin")); err != nil {
return err
}
return persistEnvChangesForSession()
}

38
cmd/getgo/system.go
View File

@ -1,38 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !plan9
package main
import (
"bytes"
"context"
"os/exec"
"runtime"
"strings"
)
// arch contains either amd64 or 386.
var arch = func() string {
cmd := exec.Command("uname", "-m") // "x86_64"
if runtime.GOOS == "windows" {
cmd = exec.Command("powershell", "-command", "(Get-WmiObject -Class Win32_ComputerSystem).SystemType") // "x64-based PC"
}
out, err := cmd.Output()
if err != nil {
// a sensible default?
return "amd64"
}
if bytes.Contains(out, []byte("64")) {
return "amd64"
}
return "386"
}()
func findGo(ctx context.Context, cmd string) (string, error) {
out, err := exec.CommandContext(ctx, cmd, "go").CombinedOutput()
return strings.TrimSpace(string(out)), err
}

55
cmd/getgo/system_unix.go
View File

@ -1,55 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build aix darwin dragonfly freebsd linux nacl netbsd openbsd solaris
package main
import (
"context"
"fmt"
"os"
"path/filepath"
)
const (
envSeparator = ":"
homeKey = "HOME"
lineEnding = "\n"
pathVar = "$PATH"
)
var installPath = func() string {
home, err := getHomeDir()
if err != nil {
return "/usr/local/go"
}
return filepath.Join(home, ".go")
}()
func whichGo(ctx context.Context) (string, error) {
return findGo(ctx, "which")
}
func isWindowsXP() bool {
return false
}
func currentShell() string {
return os.Getenv("SHELL")
}
func persistEnvChangesForSession() error {
shellConfig, err := shellConfigFile()
if err != nil {
return err
}
fmt.Println()
fmt.Printf("One more thing! Run `source %s` to persist the\n", shellConfig)
fmt.Println("new environment variables to your current session, or open a")
fmt.Println("new shell prompt.")
return nil
}

86
cmd/getgo/system_windows.go
View File

@ -1,86 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build windows
package main
import (
"context"
"log"
"os"
"syscall"
"unsafe"
)
const (
envSeparator = ";"
homeKey = "USERPROFILE"
lineEnding = "/r/n"
pathVar = "$env:Path"
)
var installPath = `c:\go`
func isWindowsXP() bool {
v, err := syscall.GetVersion()
if err != nil {
log.Fatalf("GetVersion failed: %v", err)
}
major := byte(v)
return major < 6
}
func whichGo(ctx context.Context) (string, error) {
return findGo(ctx, "where")
}
// currentShell reports the current shell.
// It might be "powershell.exe", "cmd.exe" or any of the *nix shells.
//
// Returns empty string if the shell is unknown.
func currentShell() string {
shell := os.Getenv("SHELL")
if shell != "" {
return shell
}
pid := os.Getppid()
pe, err := getProcessEntry(pid)
if err != nil {
verbosef("getting shell from process entry failed: %v", err)
return ""
}
return syscall.UTF16ToString(pe.ExeFile[:])
}
func getProcessEntry(pid int) (*syscall.ProcessEntry32, error) {
// From https://go.googlesource.com/go/+/go1.8.3/src/syscall/syscall_windows.go#941
snapshot, err := syscall.CreateToolhelp32Snapshot(syscall.TH32CS_SNAPPROCESS, 0)
if err != nil {
return nil, err
}
defer syscall.CloseHandle(snapshot)
var procEntry syscall.ProcessEntry32
procEntry.Size = uint32(unsafe.Sizeof(procEntry))
if err = syscall.Process32First(snapshot, &procEntry); err != nil {
return nil, err
}
for {
if procEntry.ProcessID == uint32(pid) {
return &procEntry, nil
}
if err := syscall.Process32Next(snapshot, &procEntry); err != nil {
return nil, err
}
}
}
func persistEnvChangesForSession() error {
return nil
}

19
cmd/getgo/upload.bash
View File

@ -1,19 +0,0 @@
#!/bin/bash
# Copyright 2017 The Go Authors. All rights reserved.
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file.
if ! command -v gsutil 2>&1 > /dev/null; then
echo "Install gsutil:"
echo
echo " https://cloud.google.com/storage/docs/gsutil_install#sdk-install"
fi
if [ ! -d build ]; then
echo "Run make.bash first"
fi
set -e -o -x
gsutil -m cp -a public-read build/* gs://golang/getgo

289
cmd/go-contrib-init/contrib.go
View File

@ -1,289 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// The go-contrib-init command helps new Go contributors get their development
// environment set up for the Go contribution process.
//
// It aims to be a complement or alternative to https://golang.org/doc/contribute.html.
package main
import (
"bytes"
"flag"
"fmt"
"go/build"
"io/ioutil"
"log"
"os"
"os/exec"
"path/filepath"
"regexp"
"runtime"
"strings"
)
var (
repo = flag.String("repo", detectrepo(), "Which go repo you want to contribute to. Use \"go\" for the core, or e.g. \"net\" for golang.org/x/net/*")
dry = flag.Bool("dry-run", false, "Fail with problems instead of trying to fix things.")
)
func main() {
log.SetFlags(0)
flag.Parse()
checkCLA()
checkGoroot()
checkWorkingDir()
checkGitOrigin()
checkGitCodeReview()
fmt.Print("All good. Happy hacking!\n" +
"Remember to squash your revised commits and preserve the magic Change-Id lines.\n" +
"Next steps: https://golang.org/doc/contribute.html#commit_changes\n")
}
func detectrepo() string {
wd, err := os.Getwd()
if err != nil {
return "go"
}
for _, path := range filepath.SplitList(build.Default.GOPATH) {
rightdir := filepath.Join(path, "src", "golang.org", "x") + string(os.PathSeparator)
if strings.HasPrefix(wd, rightdir) {
tail := wd[len(rightdir):]
end := strings.Index(tail, string(os.PathSeparator))
if end > 0 {
repo := tail[:end]
return repo
}
}
}
return "go"
}
var googleSourceRx = regexp.MustCompile(`(?m)^(go|go-review)?\.googlesource.com\b`)
func checkCLA() {
slurp, err := ioutil.ReadFile(cookiesFile())
if err != nil && !os.IsNotExist(err) {
log.Fatal(err)
}
if googleSourceRx.Match(slurp) {
// Probably good.
return
}
log.Fatal("Your .gitcookies file isn't configured.\n" +
"Next steps:\n" +
" * Submit a CLA (https://golang.org/doc/contribute.html#cla) if not done\n" +
" * Go to https://go.googlesource.com/ and click \"Generate Password\" at the top,\n" +
" then follow instructions.\n" +
" * Run go-contrib-init again.\n")
}
func expandUser(s string) string {
env := "HOME"
if runtime.GOOS == "windows" {
env = "USERPROFILE"
} else if runtime.GOOS == "plan9" {
env = "home"
}
home := os.Getenv(env)
if home == "" {
return s
}
if len(s) >= 2 && s[0] == '~' && os.IsPathSeparator(s[1]) {
if runtime.GOOS == "windows" {
s = filepath.ToSlash(filepath.Join(home, s[2:]))
} else {
s = filepath.Join(home, s[2:])
}
}
return os.Expand(s, func(env string) string {
if env == "HOME" {
return home
}
return os.Getenv(env)
})
}
func cookiesFile() string {
out, _ := exec.Command("git", "config", "http.cookiefile").Output()
if s := strings.TrimSpace(string(out)); s != "" {
if strings.HasPrefix(s, "~") {
s = expandUser(s)
}
return s
}
if runtime.GOOS == "windows" {
return filepath.Join(os.Getenv("USERPROFILE"), ".gitcookies")
}
return filepath.Join(os.Getenv("HOME"), ".gitcookies")
}
func checkGoroot() {
v := os.Getenv("GOROOT")
if v == "" {
return
}
if *repo == "go" {
if strings.HasPrefix(v, "/usr/") {
log.Fatalf("Your GOROOT environment variable is set to %q\n"+
"This is almost certainly not what you want. Either unset\n"+
"your GOROOT or set it to the path of your development version\n"+
"of Go.", v)
}
slurp, err := ioutil.ReadFile(filepath.Join(v, "VERSION"))
if err == nil {
slurp = bytes.TrimSpace(slurp)
log.Fatalf("Your GOROOT environment variable is set to %q\n"+
"But that path is to a binary release of Go, with VERSION file %q.\n"+
"You should hack on Go in a fresh checkout of Go. Fix or unset your GOROOT.\n",
v, slurp)
}
}
}
func checkWorkingDir() {
wd, err := os.Getwd()
if err != nil {
log.Fatal(err)
}
if *repo == "go" {
if inGoPath(wd) {
log.Fatalf(`You can't work on Go from within your GOPATH. Please checkout Go outside of your GOPATH
Current directory: %s
GOPATH: %s
`, wd, os.Getenv("GOPATH"))
}
return
}
gopath := firstGoPath()
if gopath == "" {
log.Fatal("Your GOPATH is not set, please set it")
}
rightdir := filepath.Join(gopath, "src", "golang.org", "x", *repo)
if !strings.HasPrefix(wd, rightdir) {
dirExists, err := exists(rightdir)
if err != nil {
log.Fatal(err)
}
if !dirExists {
log.Fatalf("The repo you want to work on is currently not on your system.\n"+
"Run %q to obtain this repo\n"+
"then go to the directory %q\n",
"go get -d golang.org/x/"+*repo, rightdir)
}
log.Fatalf("Your current directory is:%q\n"+
"Working on golang/x/%v requires you be in %q\n",
wd, *repo, rightdir)
}
}
func firstGoPath() string {
list := filepath.SplitList(build.Default.GOPATH)
if len(list) < 1 {
return ""
}
return list[0]
}
func exists(path string) (bool, error) {
_, err := os.Stat(path)
if os.IsNotExist(err) {
return false, nil
}
return true, err
}
func inGoPath(wd string) bool {
if os.Getenv("GOPATH") == "" {
return false
}
for _, path := range filepath.SplitList(os.Getenv("GOPATH")) {
if strings.HasPrefix(wd, filepath.Join(path, "src")) {
return true
}
}
return false
}
// mostly check that they didn't clone from github
func checkGitOrigin() {
if _, err := exec.LookPath("git"); err != nil {
log.Fatalf("You don't appear to have git installed. Do that.")
}
wantRemote := "https://go.googlesource.com/" + *repo
remotes, err := exec.Command("git", "remote", "-v").Output()
if err != nil {
msg := cmdErr(err)
if strings.Contains(msg, "Not a git repository") {
log.Fatalf("Your current directory is not in a git checkout of %s", wantRemote)
}
log.Fatalf("Error running git remote -v: %v", msg)
}
matches := 0
for _, line := range strings.Split(string(remotes), "\n") {
line = strings.TrimSpace(line)
if !strings.HasPrefix(line, "origin") {
continue
}
if !strings.Contains(line, wantRemote) {
curRemote := strings.Fields(strings.TrimPrefix(line, "origin"))[0]
// TODO: if not in dryRun mode, just fix it?
log.Fatalf("Current directory's git was cloned from %q; origin should be %q", curRemote, wantRemote)
}
matches++
}
if matches == 0 {
log.Fatalf("git remote -v output didn't contain expected %q. Got:\n%s", wantRemote, remotes)
}
}
func cmdErr(err error) string {
if ee, ok := err.(*exec.ExitError); ok && len(ee.Stderr) > 0 {
return fmt.Sprintf("%s: %s", err, ee.Stderr)
}
return fmt.Sprint(err)
}
func checkGitCodeReview() {
if _, err := exec.LookPath("git-codereview"); err != nil {
if *dry {
log.Fatalf("You don't appear to have git-codereview tool. While this is technically optional,\n" +
"almost all Go contributors use it. Our documentation and this tool assume it is used.\n" +
"To install it, run:\n\n\t$ go get golang.org/x/review/git-codereview\n\n(Then run go-contrib-init again)")
}
err := exec.Command("go", "get", "golang.org/x/review/git-codereview").Run()
if err != nil {
log.Fatalf("Error running go get golang.org/x/review/git-codereview: %v", cmdErr(err))
}
log.Printf("Installed git-codereview (ran `go get golang.org/x/review/git-codereview`)")
}
missing := false
for _, cmd := range []string{"change", "gofmt", "mail", "pending", "submit", "sync"} {
v, _ := exec.Command("git", "config", "alias."+cmd).Output()
if strings.Contains(string(v), "codereview") {
continue
}
if *dry {
log.Printf("Missing alias. Run:\n\t$ git config alias.%s \"codereview %s\"", cmd, cmd)
missing = true
} else {
err := exec.Command("git", "config", "alias."+cmd, "codereview "+cmd).Run()
if err != nil {
log.Fatalf("Error setting alias.%s: %v", cmd, cmdErr(err))
}
}
}
if missing {
log.Fatalf("Missing aliases. (While optional, this tool assumes you use them.)")
}
}

35
cmd/go-contrib-init/contrib_test.go
View File

@ -1,35 +0,0 @@
package main
import (
"os"
"runtime"
"testing"
)
func TestExpandUser(t *testing.T) {
env := "HOME"
if runtime.GOOS == "windows" {
env = "USERPROFILE"
} else if runtime.GOOS == "plan9" {
env = "home"
}
oldenv := os.Getenv(env)
os.Setenv(env, "/home/gopher")
defer os.Setenv(env, oldenv)
tests := []struct {
input string
want string
}{
{input: "~/foo", want: "/home/gopher/foo"},
{input: "${HOME}/foo", want: "/home/gopher/foo"},
{input: "/~/foo", want: "/~/foo"},
}
for _, tt := range tests {
got := expandUser(tt.input)
if got != tt.want {
t.Fatalf("want %q, but %q", tt.want, got)
}
}
}

8
cmd/godex/gc.go
View File

@ -2,12 +2,16 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build go1.5
// This file implements access to gc-generated export data.
package main
import "go/importer"
import (
"golang.org/x/tools/go/gcimporter"
)
func init() {
register("gc", importer.For("gc", nil))
register("gc", gcimporter.Import)
}

17
cmd/godex/gc14.go Normal file
View File

@ -0,0 +1,17 @@
// Copyright 2014 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !go1.5
// This file implements access to gc-generated export data.
package main
import (
"golang.org/x/tools/go/gcimporter"
)
func init() {
register("gc", gcimporter.Import)
}

38
cmd/godex/gccgo.go
View File

@ -2,33 +2,41 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build go1.5
// This file implements access to gccgo-generated export data.
package main
import (
"go/importer"
"go/types"
"golang.org/x/tools/go/gccgoimporter"
"golang.org/x/tools/go/types"
)
var (
initmap = make(map[*types.Package]gccgoimporter.InitData)
)
func init() {
register("gccgo", importer.For("gccgo", nil))
incpaths := []string{"/"}
// importer for default gccgo
var inst gccgoimporter.GccgoInstallation
inst.InitFromDriver("gccgo")
register("gccgo", inst.GetImporter(incpaths, initmap))
}
// Print the extra gccgo compiler data for this package, if it exists.
func (p *printer) printGccgoExtra(pkg *types.Package) {
// Disabled for now.
// TODO(gri) address this at some point.
if initdata, ok := initmap[pkg]; ok {
p.printf("/*\npriority %d\n", initdata.Priority)
// if initdata, ok := initmap[pkg]; ok {
// p.printf("/*\npriority %d\n", initdata.Priority)
p.printDecl("init", len(initdata.Inits), func() {
for _, init := range initdata.Inits {
p.printf("%s %s %d\n", init.Name, init.InitFunc, init.Priority)
}
})
// p.printDecl("init", len(initdata.Inits), func() {
// for _, init := range initdata.Inits {
// p.printf("%s %s %d\n", init.Name, init.InitFunc, init.Priority)
// }
// })
// p.print("*/\n")
// }
p.print("*/\n")
}
}

42
cmd/godex/gccgo14.go Normal file
View File

@ -0,0 +1,42 @@
// Copyright 2014 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !go1.5
// This file implements access to gccgo-generated export data.
package main
import (
"golang.org/x/tools/go/gccgoimporter"
"golang.org/x/tools/go/types"
)
var (
initmap = make(map[*types.Package]gccgoimporter.InitData)
)
func init() {
incpaths := []string{"/"}
// importer for default gccgo
var inst gccgoimporter.GccgoInstallation
inst.InitFromDriver("gccgo")
register("gccgo", inst.GetImporter(incpaths, initmap))
}
// Print the extra gccgo compiler data for this package, if it exists.
func (p *printer) printGccgoExtra(pkg *types.Package) {
if initdata, ok := initmap[pkg]; ok {
p.printf("/*\npriority %d\n", initdata.Priority)
p.printDecl("init", len(initdata.Inits), func() {
for _, init := range initdata.Inits {
p.printf("%s %s %d\n", init.Name, init.InitFunc, init.Priority)
}
})
p.print("*/\n")
}
}

48
cmd/godex/godex.go
View File

@ -2,6 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build go1.5
package main
import (
@ -9,11 +11,12 @@ import (
"flag"
"fmt"
"go/build"
"go/types"
"io/ioutil"
"os"
"path/filepath"
"strings"
"golang.org/x/tools/go/types"
)
var (
@ -28,6 +31,9 @@ var (
importFailed = errors.New("import failed")
)
// map of imported packages
var packages = make(map[string]*types.Package)
func usage() {
fmt.Fprintln(os.Stderr, "usage: godex [flags] {path|qualifiedIdent}")
flag.PrintDefaults()
@ -47,7 +53,7 @@ func main() {
report("no package name, path, or file provided")
}
var imp types.Importer = new(tryImporters)
imp := tryImports
if *source != "" {
imp = lookup(*source)
if imp == nil {
@ -65,7 +71,7 @@ func main() {
go genPrefixes(prefixes, !filepath.IsAbs(path) && !build.IsLocalImport(path))
// import package
pkg, err := tryPrefixes(prefixes, path, imp)
pkg, err := tryPrefixes(packages, prefixes, path, imp)
if err != nil {
logf("\t=> ignoring %q: %s\n", path, err)
continue
@ -109,7 +115,7 @@ func splitPathIdent(arg string) (path, name string) {
// tryPrefixes tries to import the package given by (the possibly partial) path using the given importer imp
// by prepending all possible prefixes to path. It returns with the first package that it could import, or
// with an error.
func tryPrefixes(prefixes chan string, path string, imp types.Importer) (pkg *types.Package, err error) {
func tryPrefixes(packages map[string]*types.Package, prefixes chan string, path string, imp types.Importer) (pkg *types.Package, err error) {
for prefix := range prefixes {
actual := path
if prefix == "" {
@ -121,7 +127,7 @@ func tryPrefixes(prefixes chan string, path string, imp types.Importer) (pkg *ty
actual = filepath.Join(prefix, path)
logf("\ttrying prefix %q\n", prefix)
}
pkg, err = imp.Import(actual)
pkg, err = imp(packages, actual)
if err == nil {
break
}
@ -130,14 +136,12 @@ func tryPrefixes(prefixes chan string, path string, imp types.Importer) (pkg *ty
return
}
// tryImporters is an importer that tries all registered importers
// tryImports is an importer that tries all registered importers
// successively until one of them succeeds or all of them failed.
type tryImporters struct{}
func (t *tryImporters) Import(path string) (pkg *types.Package, err error) {
func tryImports(packages map[string]*types.Package, path string) (pkg *types.Package, err error) {
for i, imp := range importers {
logf("\t\ttrying %s import\n", sources[i])
pkg, err = imp.Import(path)
pkg, err = imp(packages, path)
if err == nil {
break
}
@ -146,23 +150,17 @@ func (t *tryImporters) Import(path string) (pkg *types.Package, err error) {
return
}
type protector struct {
imp types.Importer
}
func (p *protector) Import(path string) (pkg *types.Package, err error) {
defer func() {
if recover() != nil {
pkg = nil
err = importFailed
}
}()
return p.imp.Import(path)
}
// protect protects an importer imp from panics and returns the protected importer.
func protect(imp types.Importer) types.Importer {
return &protector{imp}
return func(packages map[string]*types.Package, path string) (pkg *types.Package, err error) {
defer func() {
if recover() != nil {
pkg = nil
err = importFailed
}
}()
return imp(packages, path)
}
}
// register registers an importer imp for a given source src.

209
cmd/godex/godex14.go Normal file
View File

@ -0,0 +1,209 @@
// Copyright 2014 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !go1.5
package main
import (
"errors"
"flag"
"fmt"
"go/build"
"io/ioutil"
"os"
"path/filepath"
"strings"
"golang.org/x/tools/go/types"
)
var (
source = flag.String("s", "", "only consider packages from src, where src is one of the supported compilers")
verbose = flag.Bool("v", false, "verbose mode")
)
// lists of registered sources and corresponding importers
var (
sources []string
importers []types.Importer
importFailed = errors.New("import failed")
)
// map of imported packages
var packages = make(map[string]*types.Package)
func usage() {
fmt.Fprintln(os.Stderr, "usage: godex [flags] {path|qualifiedIdent}")
flag.PrintDefaults()
os.Exit(2)
}
func report(msg string) {
fmt.Fprintln(os.Stderr, "error: "+msg)
os.Exit(2)
}
func main() {
flag.Usage = usage
flag.Parse()
if flag.NArg() == 0 {
report("no package name, path, or file provided")
}
imp := tryImports
if *source != "" {
imp = lookup(*source)
if imp == nil {
report("source (-s argument) must be one of: " + strings.Join(sources, ", "))
}
}
for _, arg := range flag.Args() {
path, name := splitPathIdent(arg)
logf("\tprocessing %q: path = %q, name = %s\n", arg, path, name)
// generate possible package path prefixes
// (at the moment we do this for each argument - should probably cache the generated prefixes)
prefixes := make(chan string)
go genPrefixes(prefixes, !filepath.IsAbs(path) && !build.IsLocalImport(path))
// import package
pkg, err := tryPrefixes(packages, prefixes, path, imp)
if err != nil {
logf("\t=> ignoring %q: %s\n", path, err)
continue
}
// filter objects if needed
var filter func(types.Object) bool
if name != "" {
filter = func(obj types.Object) bool {
// TODO(gri) perhaps use regular expression matching here?
return obj.Name() == name
}
}
// print contents
print(os.Stdout, pkg, filter)
}
}
func logf(format string, args ...interface{}) {
if *verbose {
fmt.Fprintf(os.Stderr, format, args...)
}
}
// splitPathIdent splits a path.name argument into its components.
// All but the last path element may contain dots.
func splitPathIdent(arg string) (path, name string) {
if i := strings.LastIndex(arg, "."); i >= 0 {
if j := strings.LastIndex(arg, "/"); j < i {
// '.' is not part of path
path = arg[:i]
name = arg[i+1:]
return
}
}
path = arg
return
}
// tryPrefixes tries to import the package given by (the possibly partial) path using the given importer imp
// by prepending all possible prefixes to path. It returns with the first package that it could import, or
// with an error.
func tryPrefixes(packages map[string]*types.Package, prefixes chan string, path string, imp types.Importer) (pkg *types.Package, err error) {
for prefix := range prefixes {
actual := path
if prefix == "" {
// don't use filepath.Join as it will sanitize the path and remove
// a leading dot and then the path is not recognized as a relative
// package path by the importers anymore
logf("\ttrying no prefix\n")
} else {
actual = filepath.Join(prefix, path)
logf("\ttrying prefix %q\n", prefix)
}
pkg, err = imp(packages, actual)
if err == nil {
break
}
logf("\t=> importing %q failed: %s\n", actual, err)
}
return
}
// tryImports is an importer that tries all registered importers
// successively until one of them succeeds or all of them failed.
func tryImports(packages map[string]*types.Package, path string) (pkg *types.Package, err error) {
for i, imp := range importers {
logf("\t\ttrying %s import\n", sources[i])
pkg, err = imp(packages, path)
if err == nil {
break
}
logf("\t\t=> %s import failed: %s\n", sources[i], err)
}
return
}
// protect protects an importer imp from panics and returns the protected importer.
func protect(imp types.Importer) types.Importer {
return func(packages map[string]*types.Package, path string) (pkg *types.Package, err error) {
defer func() {
if recover() != nil {
pkg = nil
err = importFailed
}
}()
return imp(packages, path)
}
}
// register registers an importer imp for a given source src.
func register(src string, imp types.Importer) {
if lookup(src) != nil {
panic(src + " importer already registered")
}
sources = append(sources, src)
importers = append(importers, protect(imp))
}
// lookup returns the importer imp for a given source src.
func lookup(src string) types.Importer {
for i, s := range sources {
if s == src {
return importers[i]
}
}
return nil
}
func genPrefixes(out chan string, all bool) {
out <- ""
if all {
platform := build.Default.GOOS + "_" + build.Default.GOARCH
dirnames := append([]string{build.Default.GOROOT}, filepath.SplitList(build.Default.GOPATH)...)
for _, dirname := range dirnames {
walkDir(filepath.Join(dirname, "pkg", platform), "", out)
}
}
close(out)
}
func walkDir(dirname, prefix string, out chan string) {
fiList, err := ioutil.ReadDir(dirname)
if err != nil {
return
}
for _, fi := range fiList {
if fi.IsDir() && !strings.HasPrefix(fi.Name(), ".") {
prefix := filepath.Join(prefix, fi.Name())
out <- prefix
walkDir(filepath.Join(dirname, fi.Name()), prefix, out)
}
}
}

13
cmd/godex/isAlias18.go
View File

@ -1,13 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !go1.9
package main
import "go/types"
func isAlias(obj *types.TypeName) bool {
return false // there are no type aliases before Go 1.9
}

13
cmd/godex/isAlias19.go
View File

@ -1,13 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build go1.9
package main
import "go/types"
func isAlias(obj *types.TypeName) bool {
return obj.IsAlias()
}

49
cmd/godex/print.go
View File

@ -2,16 +2,19 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build go1.5
package main
import (
"bytes"
"fmt"
"go/constant"
"go/token"
"go/types"
"io"
"math/big"
"golang.org/x/tools/go/exact"
"golang.org/x/tools/go/types"
)
// TODO(gri) use tabwriter for alignment?
@ -141,13 +144,7 @@ func (p *printer) printPackage(pkg *types.Package, filter func(types.Object) boo
p.printDecl("type", len(typez), func() {
for _, obj := range typez {
p.printf("%s ", obj.Name())
typ := obj.Type()
if isAlias(obj) {
p.print("= ")
p.writeType(p.pkg, typ)
} else {
p.writeType(p.pkg, typ.Underlying())
}
p.writeType(p.pkg, obj.Type().Underlying())
p.print("\n")
}
})
@ -213,9 +210,9 @@ func (p *printer) printDecl(keyword string, n int, printGroup func()) {
// absInt returns the absolute value of v as a *big.Int.
// v must be a numeric value.
func absInt(v constant.Value) *big.Int {
func absInt(v exact.Value) *big.Int {
// compute big-endian representation of v
b := constant.Bytes(v) // little-endian
b := exact.Bytes(v) // little-endian
for i, j := 0, len(b)-1; i < j; i, j = i+1, j-1 {
b[i], b[j] = b[j], b[i]
}
@ -229,14 +226,14 @@ var (
// floatString returns the string representation for a
// numeric value v in normalized floating-point format.
func floatString(v constant.Value) string {
if constant.Sign(v) == 0 {
func floatString(v exact.Value) string {
if exact.Sign(v) == 0 {
return "0.0"
}
// x != 0
// convert |v| into a big.Rat x
x := new(big.Rat).SetFrac(absInt(constant.Num(v)), absInt(constant.Denom(v)))
x := new(big.Rat).SetFrac(absInt(exact.Num(v)), absInt(exact.Denom(v)))
// normalize x and determine exponent e
// (This is not very efficient, but also not speed-critical.)
@ -272,7 +269,7 @@ func floatString(v constant.Value) string {
if e != 0 {
s += fmt.Sprintf("e%+d", e)
}
if constant.Sign(v) < 0 {
if exact.Sign(v) < 0 {
s = "-" + s
}
@ -286,29 +283,29 @@ func floatString(v constant.Value) string {
// valString returns the string representation for the value v.
// Setting floatFmt forces an integer value to be formatted in
// normalized floating-point format.
// TODO(gri) Move this code into package constant.
func valString(v constant.Value, floatFmt bool) string {
// TODO(gri) Move this code into package exact.
func valString(v exact.Value, floatFmt bool) string {
switch v.Kind() {
case constant.Int:
case exact.Int:
if floatFmt {
return floatString(v)
}
case constant.Float:
case exact.Float:
return floatString(v)
case constant.Complex:
re := constant.Real(v)
im := constant.Imag(v)
case exact.Complex:
re := exact.Real(v)
im := exact.Imag(v)
var s string
if constant.Sign(re) != 0 {
if exact.Sign(re) != 0 {
s = floatString(re)
if constant.Sign(im) >= 0 {
if exact.Sign(im) >= 0 {
s += " + "
} else {
s += " - "
im = constant.UnaryOp(token.SUB, im, 0) // negate im
im = exact.UnaryOp(token.SUB, im, 0) // negate im
}
}
// im != 0, otherwise v would be constant.Int or constant.Float
// im != 0, otherwise v would be exact.Int or exact.Float
return s + floatString(im) + "i"
}
return v.String()

370
cmd/godex/print14.go Normal file
View File

@ -0,0 +1,370 @@
// Copyright 2014 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !go1.5
package main
import (
"bytes"
"fmt"
"go/token"
"io"
"math/big"
"golang.org/x/tools/go/exact"
"golang.org/x/tools/go/types"
)
// TODO(gri) use tabwriter for alignment?
func print(w io.Writer, pkg *types.Package, filter func(types.Object) bool) {
var p printer
p.pkg = pkg
p.printPackage(pkg, filter)
p.printGccgoExtra(pkg)
io.Copy(w, &p.buf)
}
type printer struct {
pkg *types.Package
buf bytes.Buffer
indent int // current indentation level
last byte // last byte written
}
func (p *printer) print(s string) {
// Write the string one byte at a time. We care about the presence of
// newlines for indentation which we will see even in the presence of
// (non-corrupted) Unicode; no need to read one rune at a time.
for i := 0; i < len(s); i++ {
ch := s[i]
if ch != '\n' && p.last == '\n' {
// Note: This could lead to a range overflow for very large
// indentations, but it's extremely unlikely to happen for
// non-pathological code.
p.buf.WriteString("\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t"[:p.indent])
}
p.buf.WriteByte(ch)
p.last = ch
}
}
func (p *printer) printf(format string, args ...interface{}) {
p.print(fmt.Sprintf(format, args...))
}
// methodsFor returns the named type and corresponding methods if the type
// denoted by obj is not an interface and has methods. Otherwise it returns
// the zero value.
func methodsFor(obj *types.TypeName) (*types.Named, []*types.Selection) {
named, _ := obj.Type().(*types.Named)
if named == nil {
// A type name's type can also be the
// exported basic type unsafe.Pointer.
return nil, nil
}
if _, ok := named.Underlying().(*types.Interface); ok {
// ignore interfaces
return nil, nil
}
methods := combinedMethodSet(named)
if len(methods) == 0 {
return nil, nil
}
return named, methods
}
func (p *printer) printPackage(pkg *types.Package, filter func(types.Object) bool) {
// collect objects by kind
var (
consts []*types.Const
typem []*types.Named // non-interface types with methods
typez []*types.TypeName // interfaces or types without methods
vars []*types.Var
funcs []*types.Func
builtins []*types.Builtin
methods = make(map[*types.Named][]*types.Selection) // method sets for named types
)
scope := pkg.Scope()
for _, name := range scope.Names() {
obj := scope.Lookup(name)
if obj.Exported() {
// collect top-level exported and possibly filtered objects
if filter == nil || filter(obj) {
switch obj := obj.(type) {
case *types.Const:
consts = append(consts, obj)
case *types.TypeName:
// group into types with methods and types without
if named, m := methodsFor(obj); named != nil {
typem = append(typem, named)
methods[named] = m
} else {
typez = append(typez, obj)
}
case *types.Var:
vars = append(vars, obj)
case *types.Func:
funcs = append(funcs, obj)
case *types.Builtin:
// for unsafe.Sizeof, etc.
builtins = append(builtins, obj)
}
}
} else if filter == nil {
// no filtering: collect top-level unexported types with methods
if obj, _ := obj.(*types.TypeName); obj != nil {
// see case *types.TypeName above
if named, m := methodsFor(obj); named != nil {
typem = append(typem, named)
methods[named] = m
}
}
}
}
p.printf("package %s // %q\n", pkg.Name(), pkg.Path())
p.printDecl("const", len(consts), func() {
for _, obj := range consts {
p.printObj(obj)
p.print("\n")
}
})
p.printDecl("var", len(vars), func() {
for _, obj := range vars {
p.printObj(obj)
p.print("\n")
}
})
p.printDecl("type", len(typez), func() {
for _, obj := range typez {
p.printf("%s ", obj.Name())
p.writeType(p.pkg, obj.Type().Underlying())
p.print("\n")
}
})
// non-interface types with methods
for _, named := range typem {
first := true
if obj := named.Obj(); obj.Exported() {
if first {
p.print("\n")
first = false
}
p.printf("type %s ", obj.Name())
p.writeType(p.pkg, named.Underlying())
p.print("\n")
}
for _, m := range methods[named] {
if obj := m.Obj(); obj.Exported() {
if first {
p.print("\n")
first = false
}
p.printFunc(m.Recv(), obj.(*types.Func))
p.print("\n")
}
}
}
if len(funcs) > 0 {
p.print("\n")
for _, obj := range funcs {
p.printFunc(nil, obj)
p.print("\n")
}
}
// TODO(gri) better handling of builtins (package unsafe only)
if len(builtins) > 0 {
p.print("\n")
for _, obj := range builtins {
p.printf("func %s() // builtin\n", obj.Name())
}
}
p.print("\n")
}
func (p *printer) printDecl(keyword string, n int, printGroup func()) {
switch n {
case 0:
// nothing to do
case 1:
p.printf("\n%s ", keyword)
printGroup()
default:
p.printf("\n%s (\n", keyword)
p.indent++
printGroup()
p.indent--
p.print(")\n")
}
}
// absInt returns the absolute value of v as a *big.Int.
// v must be a numeric value.
func absInt(v exact.Value) *big.Int {
// compute big-endian representation of v
b := exact.Bytes(v) // little-endian
for i, j := 0, len(b)-1; i < j; i, j = i+1, j-1 {
b[i], b[j] = b[j], b[i]
}
return new(big.Int).SetBytes(b)
}
var (
one = big.NewRat(1, 1)
ten = big.NewRat(10, 1)
)
// floatString returns the string representation for a
// numeric value v in normalized floating-point format.
func floatString(v exact.Value) string {
if exact.Sign(v) == 0 {
return "0.0"
}
// x != 0
// convert |v| into a big.Rat x
x := new(big.Rat).SetFrac(absInt(exact.Num(v)), absInt(exact.Denom(v)))
// normalize x and determine exponent e
// (This is not very efficient, but also not speed-critical.)
var e int
for x.Cmp(ten) >= 0 {
x.Quo(x, ten)
e++
}
for x.Cmp(one) < 0 {
x.Mul(x, ten)
e--
}
// TODO(gri) Values such as 1/2 are easier to read in form 0.5
// rather than 5.0e-1. Similarly, 1.0e1 is easier to read as
// 10.0. Fine-tune best exponent range for readability.
s := x.FloatString(100) // good-enough precision
// trim trailing 0's
i := len(s)
for i > 0 && s[i-1] == '0' {
i--
}
s = s[:i]
// add a 0 if the number ends in decimal point
if len(s) > 0 && s[len(s)-1] == '.' {
s += "0"
}
// add exponent and sign
if e != 0 {
s += fmt.Sprintf("e%+d", e)
}
if exact.Sign(v) < 0 {
s = "-" + s
}
// TODO(gri) If v is a "small" fraction (i.e., numerator and denominator
// are just a small number of decimal digits), add the exact fraction as
// a comment. For instance: 3.3333...e-1 /* = 1/3 */
return s
}
// valString returns the string representation for the value v.
// Setting floatFmt forces an integer value to be formatted in
// normalized floating-point format.
// TODO(gri) Move this code into package exact.
func valString(v exact.Value, floatFmt bool) string {
switch v.Kind() {
case exact.Int:
if floatFmt {
return floatString(v)
}
case exact.Float:
return floatString(v)
case exact.Complex:
re := exact.Real(v)
im := exact.Imag(v)
var s string
if exact.Sign(re) != 0 {
s = floatString(re)
if exact.Sign(im) >= 0 {
s += " + "
} else {
s += " - "
im = exact.UnaryOp(token.SUB, im, 0) // negate im
}
}
// im != 0, otherwise v would be exact.Int or exact.Float
return s + floatString(im) + "i"
}
return v.String()
}
func (p *printer) printObj(obj types.Object) {
p.print(obj.Name())
typ, basic := obj.Type().Underlying().(*types.Basic)
if basic && typ.Info()&types.IsUntyped != 0 {
// don't write untyped types
} else {
p.print(" ")
p.writeType(p.pkg, obj.Type())
}
if obj, ok := obj.(*types.Const); ok {
floatFmt := basic && typ.Info()&(types.IsFloat|types.IsComplex) != 0
p.print(" = ")
p.print(valString(obj.Val(), floatFmt))
}
}
func (p *printer) printFunc(recvType types.Type, obj *types.Func) {
p.print("func ")
sig := obj.Type().(*types.Signature)
if recvType != nil {
p.print("(")
p.writeType(p.pkg, recvType)
p.print(") ")
}
p.print(obj.Name())
p.writeSignature(p.pkg, sig)
}
// combinedMethodSet returns the method set for a named type T
// merged with all the methods of *T that have different names than
// the methods of T.
//
// combinedMethodSet is analogous to types/typeutil.IntuitiveMethodSet
// but doesn't require a MethodSetCache.
// TODO(gri) If this functionality doesn't change over time, consider
// just calling IntuitiveMethodSet eventually.
func combinedMethodSet(T *types.Named) []*types.Selection {
// method set for T
mset := types.NewMethodSet(T)
var res []*types.Selection
for i, n := 0, mset.Len(); i < n; i++ {
res = append(res, mset.At(i))
}
// add all *T methods with names different from T methods
pmset := types.NewMethodSet(types.NewPointer(T))
for i, n := 0, pmset.Len(); i < n; i++ {
pm := pmset.At(i)
if obj := pm.Obj(); mset.Lookup(obj.Pkg(), obj.Name()) == nil {
res = append(res, pm)
}
}
return res
}

12
cmd/godex/source.go
View File

@ -2,18 +2,20 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build go1.5
// This file implements access to export data from source.
package main
import "go/types"
import (
"golang.org/x/tools/go/types"
)
func init() {
register("source", sourceImporter{})
register("source", sourceImporter)
}
type sourceImporter struct{}
func (sourceImporter) Import(path string) (*types.Package, error) {
func sourceImporter(packages map[string]*types.Package, path string) (*types.Package, error) {
panic("unimplemented")
}

21
cmd/godex/source14.go Normal file
View File

@ -0,0 +1,21 @@
// Copyright 2014 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !go1.5
// This file implements access to export data from source.
package main
import (
"golang.org/x/tools/go/types"
)
func init() {
register("source", sourceImporter)
}
func sourceImporter(packages map[string]*types.Package, path string) (*types.Package, error) {
panic("unimplemented")
}

4
cmd/godex/writetype.go
View File

@ -2,6 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build go1.5
// This file implements writing of types. The functionality is lifted
// directly from go/types, but now contains various modifications for
// nicer output.
@ -12,7 +14,7 @@
package main
import "go/types"
import "golang.org/x/tools/go/types"
func (p *printer) writeType(this *types.Package, typ types.Type) {
p.writeTypeInternal(this, typ, make([]types.Type, 8))

244
cmd/godex/writetype14.go Normal file
View File

@ -0,0 +1,244 @@
// Copyright 2014 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !go1.5
// This file implements writing of types. The functionality is lifted
// directly from go/types, but now contains various modifications for
// nicer output.
//
// TODO(gri) back-port once we have a fixed interface and once the
// go/types API is not frozen anymore for the 1.3 release; and remove
// this implementation if possible.
package main
import "golang.org/x/tools/go/types"
func (p *printer) writeType(this *types.Package, typ types.Type) {
p.writeTypeInternal(this, typ, make([]types.Type, 8))
}
// From go/types - leave for now to ease back-porting this code.
const GcCompatibilityMode = false
func (p *printer) writeTypeInternal(this *types.Package, typ types.Type, visited []types.Type) {
// Theoretically, this is a quadratic lookup algorithm, but in
// practice deeply nested composite types with unnamed component
// types are uncommon. This code is likely more efficient than
// using a map.
for _, t := range visited {
if t == typ {
p.printf("○%T", typ) // cycle to typ
return
}
}
visited = append(visited, typ)
switch t := typ.(type) {
case nil:
p.print("<nil>")
case *types.Basic:
if t.Kind() == types.UnsafePointer {
p.print("unsafe.")
}
if GcCompatibilityMode {
// forget the alias names
switch t.Kind() {
case types.Byte:
t = types.Typ[types.Uint8]
case types.Rune:
t = types.Typ[types.Int32]
}
}
p.print(t.Name())
case *types.Array:
p.printf("[%d]", t.Len())
p.writeTypeInternal(this, t.Elem(), visited)
case *types.Slice:
p.print("[]")
p.writeTypeInternal(this, t.Elem(), visited)
case *types.Struct:
n := t.NumFields()
if n == 0 {
p.print("struct{}")
return
}
p.print("struct {\n")
p.indent++
for i := 0; i < n; i++ {
f := t.Field(i)
if !f.Anonymous() {
p.printf("%s ", f.Name())
}
p.writeTypeInternal(this, f.Type(), visited)
if tag := t.Tag(i); tag != "" {
p.printf(" %q", tag)
}
p.print("\n")
}
p.indent--
p.print("}")
case *types.Pointer:
p.print("*")
p.writeTypeInternal(this, t.Elem(), visited)
case *types.Tuple:
p.writeTuple(this, t, false, visited)
case *types.Signature:
p.print("func")
p.writeSignatureInternal(this, t, visited)
case *types.Interface:
// We write the source-level methods and embedded types rather
// than the actual method set since resolved method signatures
// may have non-printable cycles if parameters have anonymous
// interface types that (directly or indirectly) embed the
// current interface. For instance, consider the result type
// of m:
//
// type T interface{
// m() interface{ T }
// }
//
n := t.NumMethods()
if n == 0 {
p.print("interface{}")
return
}
p.print("interface {\n")
p.indent++
if GcCompatibilityMode {
// print flattened interface
// (useful to compare against gc-generated interfaces)
for i := 0; i < n; i++ {
m := t.Method(i)
p.print(m.Name())
p.writeSignatureInternal(this, m.Type().(*types.Signature), visited)
p.print("\n")
}
} else {
// print explicit interface methods and embedded types
for i, n := 0, t.NumExplicitMethods(); i < n; i++ {
m := t.ExplicitMethod(i)
p.print(m.Name())
p.writeSignatureInternal(this, m.Type().(*types.Signature), visited)
p.print("\n")
}
for i, n := 0, t.NumEmbeddeds(); i < n; i++ {
typ := t.Embedded(i)
p.writeTypeInternal(this, typ, visited)
p.print("\n")
}
}
p.indent--
p.print("}")
case *types.Map:
p.print("map[")
p.writeTypeInternal(this, t.Key(), visited)
p.print("]")
p.writeTypeInternal(this, t.Elem(), visited)
case *types.Chan:
var s string
var parens bool
switch t.Dir() {
case types.SendRecv:
s = "chan "
// chan (<-chan T) requires parentheses
if c, _ := t.Elem().(*types.Chan); c != nil && c.Dir() == types.RecvOnly {
parens = true
}
case types.SendOnly:
s = "chan<- "
case types.RecvOnly:
s = "<-chan "
default:
panic("unreachable")
}
p.print(s)
if parens {
p.print("(")
}
p.writeTypeInternal(this, t.Elem(), visited)
if parens {
p.print(")")
}
case *types.Named:
s := "<Named w/o object>"
if obj := t.Obj(); obj != nil {
if pkg := obj.Pkg(); pkg != nil {
if pkg != this {
p.print(pkg.Path())
p.print(".")
}
// TODO(gri): function-local named types should be displayed
// differently from named types at package level to avoid
// ambiguity.
}
s = obj.Name()
}
p.print(s)
default:
// For externally defined implementations of Type.
p.print(t.String())
}
}
func (p *printer) writeTuple(this *types.Package, tup *types.Tuple, variadic bool, visited []types.Type) {
p.print("(")
for i, n := 0, tup.Len(); i < n; i++ {
if i > 0 {
p.print(", ")
}
v := tup.At(i)
if name := v.Name(); name != "" {
p.print(name)
p.print(" ")
}
typ := v.Type()
if variadic && i == n-1 {
p.print("...")
typ = typ.(*types.Slice).Elem()
}
p.writeTypeInternal(this, typ, visited)
}
p.print(")")
}
func (p *printer) writeSignature(this *types.Package, sig *types.Signature) {
p.writeSignatureInternal(this, sig, make([]types.Type, 8))
}
func (p *printer) writeSignatureInternal(this *types.Package, sig *types.Signature, visited []types.Type) {
p.writeTuple(this, sig.Params(), sig.Variadic(), visited)
res := sig.Results()
n := res.Len()
if n == 0 {
// no result
return
}
p.print(" ")
if n == 1 && res.At(0).Name() == "" {
// single unnamed result
p.writeTypeInternal(this, res.At(0).Type(), visited)
return
}
// multiple or named result(s)
p.writeTuple(this, res, false, visited)
}

56
cmd/godoc/README.godoc-app Normal file
View File

@ -0,0 +1,56 @@
godoc on appengine
------------------
Prerequisites
-------------
* Go appengine SDK
https://developers.google.com/appengine/downloads#Google_App_Engine_SDK_for_Go
* Go sources at tip under $GOROOT
* Godoc sources at tip inside $GOPATH
(go get -d golang.org/x/tools/cmd/godoc)
Directory structure
-------------------
* Let $APPDIR be the directory containing the app engine files.
(e.g., $APPDIR=$HOME/godoc-app)
* $APPDIR contains the following entries (this may change depending on
app-engine release and version of godoc):
app.yaml
golang.org/x/tools/cmd/godoc
godoc.zip
index.split.*
* The app.yaml file is set up per app engine documentation.
For instance:
application: godoc-app
version: 1
runtime: go
api_version: go1
handlers:
- url: /.*
script: _go_app
Configuring and running godoc
-----------------------------
To configure godoc, run
bash setup-godoc-app.bash
to prepare an $APPDIR as described above. See the script for details on usage.
To run godoc locally, using the App Engine development server, run
<path to go_appengine>/dev_appserver.py $APPDIR
godoc should come up at http://localhost:8080 .

82
cmd/godoc/appinit.go Normal file
View File

@ -0,0 +1,82 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build appengine
package main
// This file replaces main.go when running godoc under app-engine.
// See README.godoc-app for details.
import (
"archive/zip"
"log"
"net/http"
"path"
"regexp"
"golang.org/x/tools/godoc"
"golang.org/x/tools/godoc/dl"
"golang.org/x/tools/godoc/proxy"
"golang.org/x/tools/godoc/short"
"golang.org/x/tools/godoc/static"
"golang.org/x/tools/godoc/vfs"
"golang.org/x/tools/godoc/vfs/mapfs"
"golang.org/x/tools/godoc/vfs/zipfs"
"google.golang.org/appengine"
)
func init() {
enforceHosts = !appengine.IsDevAppServer()
playEnabled = true
log.Println("initializing godoc ...")
log.Printf(".zip file = %s", zipFilename)
log.Printf(".zip GOROOT = %s", zipGoroot)
log.Printf("index files = %s", indexFilenames)
goroot := path.Join("/", zipGoroot) // fsHttp paths are relative to '/'
// read .zip file and set up file systems
const zipfile = zipFilename
rc, err := zip.OpenReader(zipfile)
if err != nil {
log.Fatalf("%s: %s\n", zipfile, err)
}
// rc is never closed (app running forever)
fs.Bind("/", zipfs.New(rc, zipFilename), goroot, vfs.BindReplace)
fs.Bind("/lib/godoc", mapfs.New(static.Files), "/", vfs.BindReplace)
corpus := godoc.NewCorpus(fs)
corpus.Verbose = false
corpus.MaxResults = 10000 // matches flag default in main.go
corpus.IndexEnabled = true
corpus.IndexFiles = indexFilenames
if err := corpus.Init(); err != nil {
log.Fatal(err)
}
corpus.IndexDirectory = indexDirectoryDefault
go corpus.RunIndexer()
pres = godoc.NewPresentation(corpus)
pres.TabWidth = 8
pres.ShowPlayground = true
pres.ShowExamples = true
pres.DeclLinks = true
pres.NotesRx = regexp.MustCompile("BUG")
readTemplates(pres, true)
mux := registerHandlers(pres)
dl.RegisterHandlers(mux)
short.RegisterHandlers(mux)
// Register /compile and /share handlers against the default serve mux
// so that other app modules can make plain HTTP requests to those
// hosts. (For reasons, HTTPS communication between modules is broken.)
proxy.RegisterHandlers(http.DefaultServeMux)
log.Println("godoc initialization complete")
}

26
cmd/godoc/blog.go
View File

@ -21,7 +21,7 @@ import (
const (
blogRepo = "golang.org/x/blog"
blogURL = "https://blog.golang.org/"
blogURL = "http://blog.golang.org/"
blogPath = "/blog/"
)
@ -34,19 +34,16 @@ var (
func init() {
// Initialize blog only when first accessed.
http.HandleFunc(blogPath, func(w http.ResponseWriter, r *http.Request) {
blogInitOnce.Do(func() {
blogInit(r.Host)
})
blogInitOnce.Do(blogInit)
blogServer.ServeHTTP(w, r)
})
}
func blogInit(host string) {
// Binary distributions included the blog content in "/blog".
// We stopped including this in Go 1.11.
func blogInit() {
// Binary distributions will include the blog content in "/blog".
root := filepath.Join(runtime.GOROOT(), "blog")
// Prefer content from the golang.org/x/blog repository if present.
// Prefer content from go.blog repository if present.
if pkg, err := build.Import(blogRepo, "", build.FindOnly); err == nil {
root = pkg.Dir
}
@ -60,13 +57,12 @@ func blogInit(host string) {
}
s, err := blog.NewServer(blog.Config{
BaseURL: blogPath,
BasePath: strings.TrimSuffix(blogPath, "/"),
ContentPath: filepath.Join(root, "content"),
TemplatePath: filepath.Join(root, "template"),
HomeArticles: 5,
PlayEnabled: playEnabled,
ServeLocalLinks: strings.HasPrefix(host, "localhost"),
BaseURL: blogPath,
BasePath: strings.TrimSuffix(blogPath, "/"),
ContentPath: filepath.Join(root, "content"),
TemplatePath: filepath.Join(root, "template"),
HomeArticles: 5,
PlayEnabled: playEnabled,
})
if err != nil {
log.Fatal(err)

2
cmd/godoc/dl.go
View File

@ -2,6 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !appengine
package main
import "net/http"

75
cmd/godoc/doc.go
View File

@ -6,19 +6,48 @@
Godoc extracts and generates documentation for Go programs.
It runs as a web server and presents the documentation as a
It has two modes.
Without the -http flag, it runs in command-line mode and prints plain text
documentation to standard output and exits. If both a library package and
a command with the same name exists, using the prefix cmd/ will force
documentation on the command rather than the library package. If the -src
flag is specified, godoc prints the exported interface of a package in Go
source form, or the implementation of a specific exported language entity:
godoc fmt # documentation for package fmt
godoc fmt Printf # documentation for fmt.Printf
godoc cmd/go # force documentation for the go command
godoc -src fmt # fmt package interface in Go source form
godoc -src fmt Printf # implementation of fmt.Printf
In command-line mode, the -q flag enables search queries against a godoc running
as a webserver. If no explicit server address is specified with the -server flag,
godoc first tries localhost:6060 and then http://golang.org.
godoc -q Reader
godoc -q math.Sin
godoc -server=:6060 -q sin
With the -http flag, it runs as a web server and presents the documentation as a
web page.
godoc -http=:6060
Usage:
godoc [flag]
godoc [flag] package [name ...]
The flags are:
-v
verbose mode
-q
arguments are considered search queries: a legal query is a
single identifier (such as ToLower) or a qualified identifier
(such as math.Sin)
-src
print (exported) source in command-line mode
-tabwidth=4
width of tabs in units of spaces
-timestamps=true
show timestamps with directory listings
-index
@ -32,12 +61,7 @@ The flags are:
to the indexer (the indexer will never finish), a value of 1.0
means that index creation is running at full throttle (other
goroutines may get no time while the index is built)
-index_interval=0
interval of indexing; a value of 0 sets it to 5 minutes, a
negative value indexes only once at startup
-play=false
enable playground
-links=true
-links=true:
link identifiers to their declarations
-write_index=false
write index to a file; the file name must be specified with
@ -48,17 +72,21 @@ The flags are:
-notes="BUG"
regular expression matching note markers to show
(e.g., "BUG|TODO", ".*")
-html
print HTML in command-line mode
-goroot=$GOROOT
Go root directory
-http=addr
HTTP service address (e.g., '127.0.0.1:6060' or just ':6060')
-server=addr
webserver address for command line searches
-analysis=type,pointer
comma-separated list of analyses to perform
"type": display identifier resolution, type info, method sets,
"type": display identifier resolution, type info, method sets,
'implements', and static callees
"pointer": display channel peers, callers and dynamic callees
"pointer" display channel peers, callers and dynamic callees
(significantly slower)
See https://golang.org/lib/godoc/analysis/help.html for details.
See http://golang.org/lib/godoc/analysis/help.html for details.
-templates=""
directory containing alternate template files; if set,
the directory may provide alternative template files
@ -73,7 +101,7 @@ By default, godoc looks at the packages it finds via $GOROOT and $GOPATH (if set
This behavior can be altered by providing an alternative $GOROOT with the -goroot
flag.
When the -index flag is set, a search index is maintained.
When godoc runs as a web server and -index is set, a search index is maintained.
The index is created at startup.
The index contains both identifier and full text search information (searchable
@ -81,19 +109,18 @@ via regular expressions). The maximum number of full text search results shown
can be set with the -maxresults flag; if set to 0, no full text results are
shown, and only an identifier index but no full text search index is created.
By default, godoc uses the system's GOOS/GOARCH. You can provide the URL parameters
"GOOS" and "GOARCH" to set the output on the web page for the target system.
The presentation mode of web pages served by godoc can be controlled with the
"m" URL parameter; it accepts a comma-separated list of flag names as value:
all show documentation for all declarations, not just the exported ones
methods show all embedded methods, not just those of unexported anonymous fields
src show the original source code rather than the extracted documentation
src show the original source code rather then the extracted documentation
text present the page in textual (command-line) form rather than HTML
flat present flat (not indented) directory listings using full paths
For instance, https://golang.org/pkg/math/big/?m=all shows the documentation
for all (not just the exported) declarations of package big.
For instance, http://golang.org/pkg/math/big/?m=all,text shows the documentation
for all (not just the exported) declarations of package big, in textual form (as
it would appear when using godoc from the command line: "godoc -src math/big .*").
By default, godoc serves files from the file system of the underlying OS.
Instead, a .zip file may be provided via the -zip flag, which contains
@ -102,18 +129,18 @@ slash ('/') as path separator; and they must be unrooted. $GOROOT (or -goroot)
must be set to the .zip file directory path containing the Go root directory.
For instance, for a .zip file created by the command:
zip -r go.zip $HOME/go
zip go.zip $HOME/go
one may run godoc as follows:
godoc -http=:6060 -zip=go.zip -goroot=$HOME/go
Godoc documentation is converted to HTML or to text using the go/doc package;
see https://golang.org/pkg/go/doc/#ToHTML for the exact rules.
see http://golang.org/pkg/go/doc/#ToHTML for the exact rules.
Godoc also shows example code that is runnable by the testing package;
see https://golang.org/pkg/testing/#hdr-Examples for the conventions.
see http://golang.org/pkg/testing/#hdr-Examples for the conventions.
See "Godoc: documenting Go code" for how to write good comments for godoc:
https://golang.org/doc/articles/godoc_documenting_go_code.html
http://golang.org/doc/articles/godoc_documenting_go_code.html
*/
package main // import "golang.org/x/tools/cmd/godoc"

294
cmd/godoc/godoc_test.go
View File

@ -8,7 +8,6 @@ import (
"bufio"
"bytes"
"fmt"
"go/build"
"io"
"io/ioutil"
"net"
@ -23,6 +22,50 @@ import (
"time"
)
var godocTests = []struct {
args []string
matches []string // regular expressions
dontmatch []string // regular expressions
}{
{
args: []string{"fmt"},
matches: []string{
`import "fmt"`,
`Package fmt implements formatted I/O`,
},
},
{
args: []string{"io", "WriteString"},
matches: []string{
`func WriteString\(`,
`WriteString writes the contents of the string s to w`,
},
},
{
args: []string{"nonexistingpkg"},
matches: []string{
// The last pattern (does not e) is for plan9:
// http://build.golang.org/log/2d8e5e14ed365bfa434b37ec0338cd9e6f8dd9bf
`no such file or directory|does not exist|cannot find the file|(?:' does not e)`,
},
},
{
args: []string{"fmt", "NonexistentSymbol"},
matches: []string{
`No match found\.`,
},
},
{
args: []string{"-src", "syscall", "Open"},
matches: []string{
`func Open\(`,
},
dontmatch: []string{
`No match found\.`,
},
},
}
// buildGodoc builds the godoc executable.
// It returns its path, and a cleanup function.
//
@ -32,10 +75,6 @@ func buildGodoc(t *testing.T) (bin string, cleanup func()) {
if runtime.GOARCH == "arm" {
t.Skip("skipping test on arm platforms; too slow")
}
if runtime.GOOS == "android" {
t.Skipf("the dependencies are not available on android")
}
tmp, err := ioutil.TempDir("", "godoc-regtest-")
if err != nil {
t.Fatal(err)
@ -58,6 +97,33 @@ func buildGodoc(t *testing.T) (bin string, cleanup func()) {
return bin, func() { os.RemoveAll(tmp) }
}
// Basic regression test for godoc command-line tool.
func TestCLI(t *testing.T) {
bin, cleanup := buildGodoc(t)
defer cleanup()
for _, test := range godocTests {
cmd := exec.Command(bin, test.args...)
cmd.Args[0] = "godoc"
out, err := cmd.CombinedOutput()
if err != nil {
t.Errorf("Running with args %#v: %v", test.args, err)
continue
}
for _, pat := range test.matches {
re := regexp.MustCompile(pat)
if !re.Match(out) {
t.Errorf("godoc %v =\n%s\nwanted /%v/", strings.Join(test.args, " "), out, pat)
}
}
for _, pat := range test.dontmatch {
re := regexp.MustCompile(pat)
if re.Match(out) {
t.Errorf("godoc %v =\n%s\ndid not want /%v/", strings.Join(test.args, " "), out, pat)
}
}
}
}
func serverAddress(t *testing.T) string {
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
@ -74,32 +140,19 @@ func waitForServerReady(t *testing.T, addr string) {
waitForServer(t,
fmt.Sprintf("http://%v/", addr),
"The Go Programming Language",
15*time.Second,
false)
5*time.Second)
}
func waitForSearchReady(t *testing.T, addr string) {
waitForServer(t,
fmt.Sprintf("http://%v/search?q=FALLTHROUGH", addr),
"The list of tokens.",
2*time.Minute,
false)
}
func waitUntilScanComplete(t *testing.T, addr string) {
waitForServer(t,
fmt.Sprintf("http://%v/pkg", addr),
"Scan is not yet complete",
2*time.Minute,
true,
)
// setting reverse as true, which means this waits
// until the string is not returned in the response anymore
2*time.Minute)
}
const pollInterval = 200 * time.Millisecond
func waitForServer(t *testing.T, url, match string, timeout time.Duration, reverse bool) {
func waitForServer(t *testing.T, url, match string, timeout time.Duration) {
// "health check" duplicated from x/tools/cmd/tipgodoc/tip.go
deadline := time.Now().Add(timeout)
for time.Now().Before(deadline) {
@ -110,74 +163,19 @@ func waitForServer(t *testing.T, url, match string, timeout time.Duration, rever
}
rbody, err := ioutil.ReadAll(res.Body)
res.Body.Close()
if err == nil && res.StatusCode == http.StatusOK {
if bytes.Contains(rbody, []byte(match)) && !reverse {
return
}
if !bytes.Contains(rbody, []byte(match)) && reverse {
return
}
if err == nil && res.StatusCode == http.StatusOK &&
bytes.Contains(rbody, []byte(match)) {
return
}
}
t.Fatalf("Server failed to respond in %v", timeout)
}
// hasTag checks whether a given release tag is contained in the current version
// of the go binary.
func hasTag(t string) bool {
for _, v := range build.Default.ReleaseTags {
if t == v {
return true
}
}
return false
}
func killAndWait(cmd *exec.Cmd) {
cmd.Process.Kill()
cmd.Wait()
}
func TestURL(t *testing.T) {
if runtime.GOOS == "plan9" {
t.Skip("skipping on plan9; fails to start up quickly enough")
}
bin, cleanup := buildGodoc(t)
defer cleanup()
testcase := func(url string, contents string) func(t *testing.T) {
return func(t *testing.T) {
stdout, stderr := new(bytes.Buffer), new(bytes.Buffer)
args := []string{fmt.Sprintf("-url=%s", url)}
cmd := exec.Command(bin, args...)
cmd.Stdout = stdout
cmd.Stderr = stderr
cmd.Args[0] = "godoc"
// Set GOPATH variable to non-existing path
// and GOPROXY=off to disable module fetches.
// We cannot just unset GOPATH variable because godoc would default it to ~/go.
// (We don't want the indexer looking at the local workspace during tests.)
cmd.Env = append(os.Environ(),
"GOPATH=does_not_exist",
"GOPROXY=off",
"GO111MODULE=off")
if err := cmd.Run(); err != nil {
t.Fatalf("failed to run godoc -url=%q: %s\nstderr:\n%s", url, err, stderr)
}
if !strings.Contains(stdout.String(), contents) {
t.Errorf("did not find substring %q in output of godoc -url=%q:\n%s", contents, url, stdout)
}
}
}
t.Run("index", testcase("/", "Go is an open source programming language"))
t.Run("fmt", testcase("/pkg/fmt", "Package fmt implements formatted I/O"))
}
// Basic integration test for godoc HTTP interface.
func TestWeb(t *testing.T) {
testWeb(t, false)
@ -193,9 +191,6 @@ func TestWebIndex(t *testing.T) {
// Basic integration test for godoc HTTP interface.
func testWeb(t *testing.T, withIndex bool) {
if runtime.GOOS == "plan9" {
t.Skip("skipping on plan9; fails to start up quickly enough")
}
bin, cleanup := buildGodoc(t)
defer cleanup()
addr := serverAddress(t)
@ -207,16 +202,7 @@ func testWeb(t *testing.T, withIndex bool) {
cmd.Stdout = os.Stderr
cmd.Stderr = os.Stderr
cmd.Args[0] = "godoc"
// Set GOPATH variable to non-existing path
// and GOPROXY=off to disable module fetches.
// We cannot just unset GOPATH variable because godoc would default it to ~/go.
// (We don't want the indexer looking at the local workspace during tests.)
cmd.Env = append(os.Environ(),
"GOPATH=does_not_exist",
"GOPROXY=off",
"GO111MODULE=off")
cmd.Env = godocEnv()
if err := cmd.Start(); err != nil {
t.Fatalf("failed to start godoc: %s", err)
}
@ -226,112 +212,73 @@ func testWeb(t *testing.T, withIndex bool) {
waitForSearchReady(t, addr)
} else {
waitForServerReady(t, addr)
waitUntilScanComplete(t, addr)
}
tests := []struct {
path string
contains []string // substring
match []string // regexp
notContains []string
needIndex bool
releaseTag string // optional release tag that must be in go/build.ReleaseTags
path string
match []string
dontmatch []string
needIndex bool
}{
{
path: "/",
contains: []string{"Go is an open source programming language"},
path: "/",
match: []string{"Go is an open source programming language"},
},
{
path: "/pkg/fmt/",
contains: []string{"Package fmt implements formatted I/O"},
path: "/pkg/fmt/",
match: []string{"Package fmt implements formatted I/O"},
},
{
path: "/src/fmt/",
contains: []string{"scan_test.go"},
path: "/src/fmt/",
match: []string{"scan_test.go"},
},
{
path: "/src/fmt/print.go",
contains: []string{"// Println formats using"},
path: "/src/fmt/print.go",
match: []string{"// Println formats using"},
},
{
path: "/pkg",
contains: []string{
match: []string{
"Standard library",
"Package fmt implements formatted I/O",
},
notContains: []string{
dontmatch: []string{
"internal/syscall",
"cmd/gc",
},
},
{
path: "/pkg/?m=all",
contains: []string{
match: []string{
"Standard library",
"Package fmt implements formatted I/O",
"internal/syscall/?m=all",
"internal/syscall",
},
notContains: []string{
dontmatch: []string{
"cmd/gc",
},
},
{
path: "/search?q=ListenAndServe",
contains: []string{
path: "/search?q=notwithstanding",
match: []string{
"/src",
},
notContains: []string{
dontmatch: []string{
"/pkg/bootstrap",
},
needIndex: true,
},
{
path: "/pkg/strings/",
contains: []string{
match: []string{
`href="/src/strings/strings.go"`,
},
},
{
path: "/cmd/compile/internal/amd64/",
contains: []string{
`href="/src/cmd/compile/internal/amd64/ssa.go"`,
},
},
{
path: "/pkg/math/bits/",
contains: []string{
`Added in Go 1.9`,
},
},
{
path: "/pkg/net/",
contains: []string{
`// IPv6 scoped addressing zone; added in Go 1.1`,
},
},
{
path: "/pkg/net/http/httptrace/",
match: []string{
`Got1xxResponse.*// Go 1\.11`,
`href="/src/cmd/compile/internal/amd64/reg.go"`,
},
releaseTag: "go1.11",
},
// Verify we don't add version info to a struct field added the same time
// as the struct itself:
{
path: "/pkg/net/http/httptrace/",
match: []string{
`(?m)GotFirstResponseByte func\(\)\s*$`,
},
},
// Remove trailing periods before adding semicolons:
{
path: "/pkg/database/sql/",
contains: []string{
"The number of connections currently in use; added in Go 1.11",
"The number of idle connections; added in Go 1.11",
},
releaseTag: "go1.11",
},
}
for _, test := range tests {
@ -345,34 +292,18 @@ func testWeb(t *testing.T, withIndex bool) {
continue
}
body, err := ioutil.ReadAll(resp.Body)
strBody := string(body)
resp.Body.Close()
if err != nil {
t.Errorf("GET %s: failed to read body: %s (response: %v)", url, err, resp)
}
isErr := false
for _, substr := range test.contains {
if test.releaseTag != "" && !hasTag(test.releaseTag) {
continue
}
for _, substr := range test.match {
if !bytes.Contains(body, []byte(substr)) {
t.Errorf("GET %s: wanted substring %q in body", url, substr)
isErr = true
}
}
for _, re := range test.match {
if test.releaseTag != "" && !hasTag(test.releaseTag) {
continue
}
if ok, err := regexp.MatchString(re, strBody); !ok || err != nil {
if err != nil {
t.Fatalf("Bad regexp %q: %v", re, err)
}
t.Errorf("GET %s: wanted to match %s in body", url, re)
isErr = true
}
}
for _, substr := range test.notContains {
for _, substr := range test.dontmatch {
if bytes.Contains(body, []byte(substr)) {
t.Errorf("GET %s: didn't want substring %q in body", url, substr)
isErr = true
@ -424,11 +355,14 @@ func main() { print(lib.V) }
defer cleanup()
addr := serverAddress(t)
cmd := exec.Command(bin, fmt.Sprintf("-http=%s", addr), "-analysis=type")
cmd.Env = os.Environ()
cmd.Env = append(cmd.Env, fmt.Sprintf("GOROOT=%s", filepath.Join(tmpdir, "goroot")))
cmd.Env = append(cmd.Env, fmt.Sprintf("GOPATH=%s", filepath.Join(tmpdir, "gopath")))
cmd.Env = append(cmd.Env, "GO111MODULE=off")
cmd.Env = append(cmd.Env, "GOPROXY=off")
for _, e := range os.Environ() {
if strings.HasPrefix(e, "GOROOT=") || strings.HasPrefix(e, "GOPATH=") {
continue
}
cmd.Env = append(cmd.Env, e)
}
cmd.Stdout = os.Stderr
stderr, err := cmd.StderrPipe()
if err != nil {
@ -506,3 +440,15 @@ tryagain:
}
}
}
// godocEnv returns the process environment without the GOPATH variable.
// (We don't want the indexer looking at the local workspace during tests.)
func godocEnv() (env []string) {
for _, v := range os.Environ() {
if strings.HasPrefix(v, "GOPATH=") {
continue
}
env = append(env, v)
}
return
}

74
cmd/godoc/goroot.go
View File

@ -1,74 +0,0 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
import (
"os"
"path/filepath"
"runtime"
)
// Copies of functions from src/cmd/go/internal/cfg/cfg.go for
// finding the GOROOT.
// Keep them in sync until support is moved to a common place, if ever.
func findGOROOT() string {
if env := os.Getenv("GOROOT"); env != "" {
return filepath.Clean(env)
}
def := filepath.Clean(runtime.GOROOT())
if runtime.Compiler == "gccgo" {
// gccgo has no real GOROOT, and it certainly doesn't
// depend on the executable's location.
return def
}
exe, err := os.Executable()
if err == nil {
exe, err = filepath.Abs(exe)
if err == nil {
if dir := filepath.Join(exe, "../.."); isGOROOT(dir) {
// If def (runtime.GOROOT()) and dir are the same
// directory, prefer the spelling used in def.
if isSameDir(def, dir) {
return def
}
return dir
}
exe, err = filepath.EvalSymlinks(exe)
if err == nil {
if dir := filepath.Join(exe, "../.."); isGOROOT(dir) {
if isSameDir(def, dir) {
return def
}
return dir
}
}
}
}
return def
}
// isGOROOT reports whether path looks like a GOROOT.
//
// It does this by looking for the path/pkg/tool directory,
// which is necessary for useful operation of the cmd/go tool,
// and is not typically present in a GOPATH.
func isGOROOT(path string) bool {
stat, err := os.Stat(filepath.Join(path, "pkg", "tool"))
if err != nil {
return false
}
return stat.IsDir()
}
// isSameDir reports whether dir1 and dir2 are the same directory.
func isSameDir(dir1, dir2 string) bool {
if dir1 == dir2 {
return true
}
info1, err1 := os.Stat(dir1)
info2, err2 := os.Stat(dir2)
return err1 == nil && err2 == nil && os.SameFile(info1, info2)
}

59
cmd/godoc/handlers.go
View File

@ -21,7 +21,6 @@ import (
"text/template"
"golang.org/x/tools/godoc"
"golang.org/x/tools/godoc/golangorgenv"
"golang.org/x/tools/godoc/redirect"
"golang.org/x/tools/godoc/vfs"
)
@ -31,20 +30,21 @@ var (
fs = vfs.NameSpace{}
)
var enforceHosts = false // set true in production on app engine
// hostEnforcerHandler redirects requests to "http://foo.golang.org/bar"
// to "https://golang.org/bar".
// It permits requests to the host "godoc-test.golang.org" for testing and
// golang.google.cn for Chinese users.
// It permits requests to the host "godoc-test.golang.org" for testing.
type hostEnforcerHandler struct {
h http.Handler
}
func (h hostEnforcerHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if !golangorgenv.EnforceHosts() {
if !enforceHosts {
h.h.ServeHTTP(w, r)
return
}
if !h.isHTTPS(r) || !h.validHost(r.Host) {
if r.TLS == nil || !h.validHost(r.Host) {
r.URL.Scheme = "https"
if h.validHost(r.Host) {
r.URL.Host = r.Host
@ -54,21 +54,12 @@ func (h hostEnforcerHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, r.URL.String(), http.StatusFound)
return
}
w.Header().Set("Strict-Transport-Security", "max-age=31536000; includeSubDomains; preload")
h.h.ServeHTTP(w, r)
}
func (h hostEnforcerHandler) isHTTPS(r *http.Request) bool {
return r.TLS != nil || r.Header.Get("X-Forwarded-Proto") == "https"
}
func (h hostEnforcerHandler) validHost(host string) bool {
switch strings.ToLower(host) {
case "golang.org", "golang.google.cn":
return true
}
if strings.HasSuffix(host, "-dot-golang-org.appspot.com") {
// staging/test
case "golang.org", "godoc-test.golang.org":
return true
}
return false
@ -112,23 +103,27 @@ func readTemplate(name string) *template.Template {
return t
}
func readTemplates(p *godoc.Presentation) {
codewalkHTML = readTemplate("codewalk.html")
codewalkdirHTML = readTemplate("codewalkdir.html")
p.CallGraphHTML = readTemplate("callgraph.html")
p.DirlistHTML = readTemplate("dirlist.html")
p.ErrorHTML = readTemplate("error.html")
p.ExampleHTML = readTemplate("example.html")
p.GodocHTML = readTemplate("godoc.html")
p.ImplementsHTML = readTemplate("implements.html")
p.MethodSetHTML = readTemplate("methodset.html")
p.PackageHTML = readTemplate("package.html")
p.PackageRootHTML = readTemplate("packageroot.html")
p.SearchHTML = readTemplate("search.html")
p.SearchDocHTML = readTemplate("searchdoc.html")
p.SearchCodeHTML = readTemplate("searchcode.html")
p.SearchTxtHTML = readTemplate("searchtxt.html")
p.SearchDescXML = readTemplate("opensearch.xml")
func readTemplates(p *godoc.Presentation, html bool) {
p.PackageText = readTemplate("package.txt")
p.SearchText = readTemplate("search.txt")
if html || p.HTMLMode {
codewalkHTML = readTemplate("codewalk.html")
codewalkdirHTML = readTemplate("codewalkdir.html")
p.CallGraphHTML = readTemplate("callgraph.html")
p.DirlistHTML = readTemplate("dirlist.html")
p.ErrorHTML = readTemplate("error.html")
p.ExampleHTML = readTemplate("example.html")
p.GodocHTML = readTemplate("godoc.html")
p.ImplementsHTML = readTemplate("implements.html")
p.MethodSetHTML = readTemplate("methodset.html")
p.PackageHTML = readTemplate("package.html")
p.SearchHTML = readTemplate("search.html")
p.SearchDocHTML = readTemplate("searchdoc.html")
p.SearchCodeHTML = readTemplate("searchcode.html")
p.SearchTxtHTML = readTemplate("searchtxt.html")
p.SearchDescXML = readTemplate("opensearch.xml")
}
}
type fmtResponse struct {

201
cmd/godoc/main.go
View File

@ -14,18 +14,28 @@
// (idea is if you say import "compress/zlib", you go to
// http://godoc/pkg/compress/zlib)
//
// Command-line interface:
//
// godoc packagepath [name ...]
//
// godoc compress/zlib
// - prints doc for package compress/zlib
// godoc crypto/block Cipher NewCMAC
// - prints doc for Cipher and NewCMAC in package crypto/block
// +build !appengine
package main
import (
"archive/zip"
"bytes"
_ "expvar" // to serve /debug/vars
"flag"
"fmt"
"go/build"
"log"
"net/http"
"net/http/httptest"
_ "net/http/pprof" // to serve /debug/pprof/*
"net/url"
"os"
@ -43,7 +53,10 @@ import (
"golang.org/x/tools/godoc/vfs/zipfs"
)
const defaultAddr = "localhost:6060" // default webserver address
const (
defaultAddr = ":6060" // default webserver address
toolsPath = "golang.org/x/tools/cmd/"
)
var (
// file system to serve
@ -56,21 +69,29 @@ var (
analysisFlag = flag.String("analysis", "", `comma-separated list of analyses to perform (supported: type, pointer). See http://golang.org/lib/godoc/analysis/help.html`)
// network
httpAddr = flag.String("http", defaultAddr, "HTTP service address")
httpAddr = flag.String("http", "", "HTTP service address (e.g., '"+defaultAddr+"')")
serverAddr = flag.String("server", "", "webserver address for command line searches")
// layout control
html = flag.Bool("html", false, "print HTML in command-line mode")
srcMode = flag.Bool("src", false, "print (exported) source in command-line mode")
urlFlag = flag.String("url", "", "print HTML for named URL")
// command-line searches
query = flag.Bool("q", false, "arguments are considered search queries")
verbose = flag.Bool("v", false, "verbose mode")
// file system roots
// TODO(gri) consider the invariant that goroot always end in '/'
goroot = flag.String("goroot", findGOROOT(), "Go root directory")
goroot = flag.String("goroot", runtime.GOROOT(), "Go root directory")
// layout control
tabWidth = flag.Int("tabwidth", 4, "tab width")
showTimestamps = flag.Bool("timestamps", false, "show timestamps with directory listings")
templateDir = flag.String("templates", "", "load templates/JS/CSS from disk in this directory")
showPlayground = flag.Bool("play", false, "enable playground")
templateDir = flag.String("templates", "", "directory containing alternate template files")
showPlayground = flag.Bool("play", false, "enable playground in web interface")
showExamples = flag.Bool("ex", false, "show examples in command line mode")
declLinks = flag.Bool("links", true, "link identifiers to their declarations")
// search index
@ -84,19 +105,10 @@ var (
notesRx = flag.String("notes", "BUG", "regular expression matching note markers to show")
)
// An httpResponseRecorder is an http.ResponseWriter
type httpResponseRecorder struct {
body *bytes.Buffer
header http.Header
code int
}
func (w *httpResponseRecorder) Header() http.Header { return w.header }
func (w *httpResponseRecorder) Write(b []byte) (int, error) { return w.body.Write(b) }
func (w *httpResponseRecorder) WriteHeader(code int) { w.code = code }
func usage() {
fmt.Fprintf(os.Stderr, "usage: godoc -http="+defaultAddr+"\n")
fmt.Fprintf(os.Stderr,
"usage: godoc package [name ...]\n"+
" godoc -http="+defaultAddr+"\n")
flag.PrintDefaults()
os.Exit(2)
}
@ -123,58 +135,40 @@ func handleURLFlag() {
// Invoke default HTTP handler to serve request
// to our buffering httpWriter.
w := &httpResponseRecorder{code: 200, header: make(http.Header), body: new(bytes.Buffer)}
w := httptest.NewRecorder()
http.DefaultServeMux.ServeHTTP(w, req)
// Return data, error, or follow redirect.
switch w.code {
switch w.Code {
case 200: // ok
os.Stdout.Write(w.body.Bytes())
os.Stdout.Write(w.Body.Bytes())
return
case 301, 302, 303, 307: // redirect
redirect := w.header.Get("Location")
redirect := w.HeaderMap.Get("Location")
if redirect == "" {
log.Fatalf("HTTP %d without Location header", w.code)
log.Fatalf("HTTP %d without Location header", w.Code)
}
urlstr = redirect
default:
log.Fatalf("HTTP error %d", w.code)
log.Fatalf("HTTP error %d", w.Code)
}
}
log.Fatalf("too many redirects")
}
func initCorpus(corpus *godoc.Corpus) {
err := corpus.Init()
if err != nil {
log.Fatal(err)
}
}
func main() {
flag.Usage = usage
flag.Parse()
if certInit != nil {
certInit()
}
playEnabled = *showPlayground
// Check usage.
if flag.NArg() > 0 {
fmt.Fprintln(os.Stderr, `Unexpected arguments. Use "go doc" for command-line help output instead. For example, "go doc fmt.Printf".`)
usage()
}
if *httpAddr == "" && *urlFlag == "" && !*writeIndex {
fmt.Fprintln(os.Stderr, "At least one of -http, -url, or -write_index must be set to a non-zero value.")
// Check usage: either server and no args, command line and args, or index creation mode
if (*httpAddr != "" || *urlFlag != "") != (flag.NArg() == 0) && !*writeIndex {
usage()
}
// Set the resolved goroot.
vfs.GOROOT = *goroot
fsGate := make(chan bool, 20)
var fsGate chan bool
fsGate = make(chan bool, 20)
// Determine file system to use.
if *zipfile == "" {
@ -201,6 +195,8 @@ func main() {
fs.Bind("/src", gatefs.New(vfs.OS(p), fsGate), "/src", vfs.BindAfter)
}
httpMode := *httpAddr != ""
var typeAnalysis, pointerAnalysis bool
if *analysisFlag != "" {
for _, a := range strings.Split(*analysisFlag, ",") {
@ -218,7 +214,7 @@ func main() {
corpus := godoc.NewCorpus(fs)
corpus.Verbose = *verbose
corpus.MaxResults = *maxResults
corpus.IndexEnabled = *indexEnabled
corpus.IndexEnabled = *indexEnabled && httpMode
if *maxResults == 0 {
corpus.IndexFullText = false
}
@ -226,27 +222,29 @@ func main() {
corpus.IndexDirectory = indexDirectoryDefault
corpus.IndexThrottle = *indexThrottle
corpus.IndexInterval = *indexInterval
if *writeIndex || *urlFlag != "" {
if *writeIndex {
corpus.IndexThrottle = 1.0
corpus.IndexEnabled = true
initCorpus(corpus)
} else {
go initCorpus(corpus)
}
if *writeIndex || httpMode || *urlFlag != "" {
if err := corpus.Init(); err != nil {
log.Fatal(err)
}
}
// Initialize the version info before readTemplates, which saves
// the map value in a method value.
corpus.InitVersionInfo()
pres = godoc.NewPresentation(corpus)
pres.TabWidth = *tabWidth
pres.ShowTimestamps = *showTimestamps
pres.ShowPlayground = *showPlayground
pres.ShowExamples = *showExamples
pres.DeclLinks = *declLinks
pres.SrcMode = *srcMode
pres.HTMLMode = *html
if *notesRx != "" {
pres.NotesRx = regexp.MustCompile(*notesRx)
}
readTemplates(pres)
readTemplates(pres, httpMode || *urlFlag != "")
registerHandlers(pres)
if *writeIndex {
@ -281,58 +279,51 @@ func main() {
return
}
var handler http.Handler = http.DefaultServeMux
if *verbose {
log.Printf("Go Documentation Server")
log.Printf("version = %s", runtime.Version())
log.Printf("address = %s", *httpAddr)
log.Printf("goroot = %s", *goroot)
switch {
case !*indexEnabled:
log.Print("search index disabled")
case *maxResults > 0:
log.Printf("full text index enabled (maxresults = %d)", *maxResults)
default:
log.Print("identifier search index enabled")
}
fs.Fprint(os.Stderr)
handler = loggingHandler(handler)
}
// Initialize search index.
if *indexEnabled {
go corpus.RunIndexer()
}
// Start type/pointer analysis.
if typeAnalysis || pointerAnalysis {
go analysis.Run(pointerAnalysis, &corpus.Analysis)
}
if runHTTPS != nil {
go func() {
if err := runHTTPS(handler); err != nil {
log.Fatalf("ListenAndServe TLS: %v", err)
if httpMode {
// HTTP server mode.
var handler http.Handler = http.DefaultServeMux
if *verbose {
log.Printf("Go Documentation Server")
log.Printf("version = %s", runtime.Version())
log.Printf("address = %s", *httpAddr)
log.Printf("goroot = %s", *goroot)
log.Printf("tabwidth = %d", *tabWidth)
switch {
case !*indexEnabled:
log.Print("search index disabled")
case *maxResults > 0:
log.Printf("full text index enabled (maxresults = %d)", *maxResults)
default:
log.Print("identifier search index enabled")
}
}()
fs.Fprint(os.Stderr)
handler = loggingHandler(handler)
}
// Initialize search index.
if *indexEnabled {
go corpus.RunIndexer()
}
// Start type/pointer analysis.
if typeAnalysis || pointerAnalysis {
go analysis.Run(pointerAnalysis, &corpus.Analysis)
}
// Start http server.
if err := http.ListenAndServe(*httpAddr, handler); err != nil {
log.Fatalf("ListenAndServe %s: %v", *httpAddr, err)
}
return
}
// Start http server.
if *verbose {
log.Println("starting HTTP server")
if *query {
handleRemoteSearch()
return
}
if wrapHTTPMux != nil {
handler = wrapHTTPMux(handler)
}
if err := http.ListenAndServe(*httpAddr, handler); err != nil {
log.Fatalf("ListenAndServe %s: %v", *httpAddr, err)
if err := godoc.CommandLine(os.Stdout, fs, pres, flag.Args()); err != nil {
log.Print(err)
}
}
// Hooks that are set non-nil in autocert.go if the "autocert" build tag
// is used.
var (
certInit func()
runHTTPS func(http.Handler) error
wrapHTTPMux func(http.Handler) http.Handler
)

2
cmd/godoc/play.go
View File

@ -2,6 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !appengine
package main
// This package registers "/compile" and "/share" handlers

72
cmd/godoc/remotesearch.go Normal file
View File

@ -0,0 +1,72 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !appengine
package main
import (
"errors"
"flag"
"io"
"log"
"net/http"
"net/url"
"os"
)
func handleRemoteSearch() {
// Command-line queries.
for i := 0; i < flag.NArg(); i++ {
res, err := remoteSearch(flag.Arg(i))
if err != nil {
log.Fatalf("remoteSearch: %s", err)
}
io.Copy(os.Stdout, res.Body)
}
return
}
// remoteSearchURL returns the search URL for a given query as needed by
// remoteSearch. If html is set, an html result is requested; otherwise
// the result is in textual form.
// Adjust this function as necessary if modeNames or FormValue parameters
// change.
func remoteSearchURL(query string, html bool) string {
s := "/search?m=text&q="
if html {
s = "/search?q="
}
return s + url.QueryEscape(query)
}
func remoteSearch(query string) (res *http.Response, err error) {
// list of addresses to try
var addrs []string
if *serverAddr != "" {
// explicit server address - only try this one
addrs = []string{*serverAddr}
} else {
addrs = []string{
defaultAddr,
"golang.org",
}
}
// remote search
search := remoteSearchURL(query, *html)
for _, addr := range addrs {
url := "http://" + addr + search
res, err = http.Get(url)
if err == nil && res.StatusCode == http.StatusOK {
break
}
}
if err == nil && res.StatusCode != http.StatusOK {
err = errors.New(res.Status)
}
return
}

134
cmd/godoc/setup-godoc-app.bash Executable file
View File

@ -0,0 +1,134 @@
#!/usr/bin/env bash
# Copyright 2011 The Go Authors. All rights reserved.
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file.
# This script creates a complete godoc app in $APPDIR.
# It copies the cmd/godoc and src/go/... sources from GOROOT,
# synthesizes an app.yaml file, and creates the .zip, index, and
# configuration files.
#
# If an argument is provided it is assumed to be the app-engine godoc directory.
# Without an argument, $APPDIR is used instead. If GOROOT is not set, "go env"
# is consulted to find the $GOROOT.
#
# The script creates a .zip file representing the $GOROOT file system
# and computes the correspondig search index files. These files are then
# copied to $APPDIR. A corresponding godoc configuration file is created
# in $APPDIR/appconfig.go.
ZIPFILE=godoc.zip
INDEXFILE=godoc.index
SPLITFILES=index.split.
GODOC=golang.org/x/tools/cmd/godoc
CONFIGFILE=$GODOC/appconfig.go
error() {
echo "error: $1"
exit 2
}
getArgs() {
if [ -z $APPENGINE_SDK ]; then
error "APPENGINE_SDK environment variable not set"
fi
if [ ! -x $APPENGINE_SDK/goapp ]; then
error "couldn't find goapp command in $APPENGINE_SDK"
fi
if [ -z $GOROOT ]; then
GOROOT=$(go env GOROOT)
echo "GOROOT not set explicitly, using go env value instead"
fi
if [ -z $APPDIR ]; then
if [ $# == 0 ]; then
error "APPDIR not set, and no argument provided"
fi
APPDIR=$1
echo "APPDIR not set, using argument instead"
fi
# safety checks
if [ ! -d $GOROOT ]; then
error "$GOROOT is not a directory"
fi
if [ -e $APPDIR ]; then
error "$APPDIR exists; check and remove it before trying again"
fi
# reporting
echo "GOROOT = $GOROOT"
echo "APPDIR = $APPDIR"
}
fetchGodoc() {
echo "*** Fetching godoc (if not already in GOPATH)"
unset GOBIN
go=$APPENGINE_SDK/goapp
$go get -d -tags appengine $GODOC
mkdir -p $APPDIR/$GODOC
cp $(find $($go list -f '{{.Dir}}' $GODOC) -mindepth 1 -maxdepth 1 -type f) $APPDIR/$GODOC/
}
makeAppYaml() {
echo "*** make $APPDIR/app.yaml"
cat > $APPDIR/app.yaml <<EOF
application: godoc
version: 1
runtime: go
api_version: go1.4beta
handlers:
- url: /.*
script: _go_app
EOF
}
makeZipfile() {
echo "*** make $APPDIR/$ZIPFILE"
zip -q -r $APPDIR/$ZIPFILE $GOROOT/*
}
makeIndexfile() {
echo "*** make $APPDIR/$INDEXFILE"
GOPATH= godoc -write_index -index_files=$APPDIR/$INDEXFILE -zip=$APPDIR/$ZIPFILE
}
splitIndexfile() {
echo "*** split $APPDIR/$INDEXFILE"
split -b8m $APPDIR/$INDEXFILE $APPDIR/$SPLITFILES
}
makeConfigfile() {
echo "*** make $APPDIR/$CONFIGFILE"
cat > $APPDIR/$CONFIGFILE <<EOF
package main
// GENERATED FILE - DO NOT MODIFY BY HAND.
// (generated by golang.org/x/tools/cmd/godoc/setup-godoc-app.bash)
const (
// .zip filename
zipFilename = "$ZIPFILE"
// goroot directory in .zip file
zipGoroot = "$GOROOT"
// glob pattern describing search index files
// (if empty, the index is built at run-time)
indexFilenames = "$SPLITFILES*"
)
EOF
}
getArgs "$@"
set -e
mkdir $APPDIR
fetchGodoc
makeAppYaml
makeZipfile
makeIndexfile
splitIndexfile
makeConfigfile
echo "*** setup complete"

89
cmd/godoc/x.go Normal file
View File

@ -0,0 +1,89 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// This file contains the handlers that serve go-import redirects for Go
// sub-repositories. It specifies the mapping from import paths like
// "golang.org/x/tools" to the actual repository locations.
package main
import (
"html/template"
"log"
"net/http"
"strings"
)
const xPrefix = "/x/"
type xRepo struct {
URL, VCS string
}
var xMap = map[string]xRepo{
"codereview": {"https://code.google.com/p/go.codereview", "hg"},
"arch": {"https://go.googlesource.com/arch", "git"},
"benchmarks": {"https://go.googlesource.com/benchmarks", "git"},
"blog": {"https://go.googlesource.com/blog", "git"},
"build": {"https://go.googlesource.com/build", "git"},
"crypto": {"https://go.googlesource.com/crypto", "git"},
"debug": {"https://go.googlesource.com/debug", "git"},
"exp": {"https://go.googlesource.com/exp", "git"},
"image": {"https://go.googlesource.com/image", "git"},
"mobile": {"https://go.googlesource.com/mobile", "git"},
"net": {"https://go.googlesource.com/net", "git"},
"oauth2": {"https://go.googlesource.com/oauth2", "git"},
"playground": {"https://go.googlesource.com/playground", "git"},
"review": {"https://go.googlesource.com/review", "git"},
"sync": {"https://go.googlesource.com/sync", "git"},
"sys": {"https://go.googlesource.com/sys", "git"},
"talks": {"https://go.googlesource.com/talks", "git"},
"term": {"https://go.googlesource.com/term", "git"},
"text": {"https://go.googlesource.com/text", "git"},
"time": {"https://go.googlesource.com/time", "git"},
"tools": {"https://go.googlesource.com/tools", "git"},
"tour": {"https://go.googlesource.com/tour", "git"},
}
func init() {
http.HandleFunc(xPrefix, xHandler)
}
func xHandler(w http.ResponseWriter, r *http.Request) {
head, tail := strings.TrimPrefix(r.URL.Path, xPrefix), ""
if i := strings.Index(head, "/"); i != -1 {
head, tail = head[:i], head[i:]
}
if head == "" {
http.Redirect(w, r, "https://godoc.org/-/subrepo", http.StatusTemporaryRedirect)
return
}
repo, ok := xMap[head]
if !ok {
http.NotFound(w, r)
return
}
data := struct {
Prefix, Head, Tail string
Repo xRepo
}{xPrefix, head, tail, repo}
if err := xTemplate.Execute(w, data); err != nil {
log.Println("xHandler:", err)
}
}
var xTemplate = template.Must(template.New("x").Parse(`<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
<meta name="go-import" content="golang.org{{.Prefix}}{{.Head}} {{.Repo.VCS}} {{.Repo.URL}}">
<meta name="go-source" content="golang.org{{.Prefix}}{{.Head}} https://github.com/golang/{{.Head}}/ https://github.com/golang/{{.Head}}/tree/master{/dir} https://github.com/golang/{{.Head}}/blob/master{/dir}/{file}#L{line}">
<meta http-equiv="refresh" content="0; url=https://godoc.org/golang.org{{.Prefix}}{{.Head}}{{.Tail}}">
</head>
<body>
Nothing to see here; <a href="https://godoc.org/golang.org{{.Prefix}}{{.Head}}{{.Tail}}">move along</a>.
</body>
</html>
`))

20
cmd/goimports/doc.go
View File

@ -5,14 +5,16 @@ adding missing ones and removing unreferenced ones.
$ go get golang.org/x/tools/cmd/goimports
In addition to fixing imports, goimports also formats
your code in the same style as gofmt so it can be used
as a replacement for your editor's gofmt-on-save hook.
It's a drop-in replacement for your editor's gofmt-on-save hook.
It has the same command-line interface as gofmt and formats
your code in the same way.
For emacs, make sure you have the latest go-mode.el:
https://github.com/dominikh/go-mode.el
Then in your .emacs file:
(setq gofmt-command "goimports")
(add-to-list 'load-path "/home/you/somewhere/emacs/")
(require 'go-mode-load)
(add-hook 'before-save-hook 'gofmt-before-save)
For vim, set "gofmt_command" to "goimports":
@ -25,18 +27,6 @@ For GoSublime, follow the steps described here:
For other editors, you probably know what to do.
To exclude directories in your $GOPATH from being scanned for Go
files, goimports respects a configuration file at
$GOPATH/src/.goimportsignore which may contain blank lines, comment
lines (beginning with '#'), or lines naming a directory relative to
the configuration file to ignore when scanning. No globbing or regex
patterns are allowed. Use the "-v" verbose flag to verify it's
working and see what goimports is doing.
File bugs or feature requests at:
https://golang.org/issues/new?title=x/tools/cmd/goimports:+
Happy hacking!
*/

216
cmd/goimports/goimports.go
View File

@ -5,24 +5,19 @@
package main
import (
"bufio"
"bytes"
"errors"
"flag"
"fmt"
"go/build"
"go/scanner"
"io"
"io/ioutil"
"log"
"os"
"os/exec"
"path/filepath"
"runtime"
"runtime/pprof"
"strings"
"golang.org/x/tools/internal/imports"
"golang.org/x/tools/imports"
)
var (
@ -30,32 +25,18 @@ var (
list = flag.Bool("l", false, "list files whose formatting differs from goimport's")
write = flag.Bool("w", false, "write result to (source) file instead of stdout")
doDiff = flag.Bool("d", false, "display diffs instead of rewriting files")
srcdir = flag.String("srcdir", "", "choose imports as if source code is from `dir`. When operating on a single file, dir may instead be the complete file name.")
verbose bool // verbose logging
cpuProfile = flag.String("cpuprofile", "", "CPU profile output")
memProfile = flag.String("memprofile", "", "memory profile output")
memProfileRate = flag.Int("memrate", 0, "if > 0, sets runtime.MemProfileRate")
options = &imports.Options{
TabWidth: 8,
TabIndent: true,
Comments: true,
Fragment: true,
// This environment, and its caches, will be reused for the whole run.
Env: &imports.ProcessEnv{
GOPATH: build.Default.GOPATH,
GOROOT: build.Default.GOROOT,
},
}
exitCode = 0
)
func init() {
flag.BoolVar(&options.AllErrors, "e", false, "report all errors (not just the first 10 on different lines)")
flag.StringVar(&options.Env.LocalPrefix, "local", "", "put imports beginning with this string after 3rd-party packages; comma-separated list")
flag.BoolVar(&options.FormatOnly, "format-only", false, "if true, don't fix imports and only format. In this mode, goimports is effectively gofmt, with the addition that imports are grouped into sections.")
}
func report(err error) {
@ -75,25 +56,9 @@ func isGoFile(f os.FileInfo) bool {
return !f.IsDir() && !strings.HasPrefix(name, ".") && strings.HasSuffix(name, ".go")
}
// argumentType is which mode goimports was invoked as.
type argumentType int
const (
// fromStdin means the user is piping their source into goimports.
fromStdin argumentType = iota
// singleArg is the common case from editors, when goimports is run on
// a single file.
singleArg
// multipleArg is when the user ran "goimports file1.go file2.go"
// or ran goimports on a directory tree.
multipleArg
)
func processFile(filename string, in io.Reader, out io.Writer, argType argumentType) error {
func processFile(filename string, in io.Reader, out io.Writer, stdin bool) error {
opt := options
if argType == fromStdin {
if stdin {
nopt := *options
nopt.Fragment = true
opt = &nopt
@ -113,35 +78,7 @@ func processFile(filename string, in io.Reader, out io.Writer, argType argumentT
return err
}
target := filename
if *srcdir != "" {
// Determine whether the provided -srcdirc is a directory or file
// and then use it to override the target.
//
// See https://github.com/dominikh/go-mode.el/issues/146
if isFile(*srcdir) {
if argType == multipleArg {
return errors.New("-srcdir value can't be a file when passing multiple arguments or when walking directories")
}
target = *srcdir
} else if argType == singleArg && strings.HasSuffix(*srcdir, ".go") && !isDir(*srcdir) {
// For a file which doesn't exist on disk yet, but might shortly.
// e.g. user in editor opens $DIR/newfile.go and newfile.go doesn't yet exist on disk.
// The goimports on-save hook writes the buffer to a temp file
// first and runs goimports before the actual save to newfile.go.
// The editor's buffer is named "newfile.go" so that is passed to goimports as:
// goimports -srcdir=/gopath/src/pkg/newfile.go /tmp/gofmtXXXXXXXX.go
// and then the editor reloads the result from the tmp file and writes
// it to newfile.go.
target = *srcdir
} else {
// Pretend that file is from *srcdir in order to decide
// visible imports correctly.
target = filepath.Join(*srcdir, filepath.Base(filename))
}
}
res, err := imports.Process(target, src, opt)
res, err := imports.Process(filename, src, opt)
if err != nil {
return err
}
@ -152,24 +89,17 @@ func processFile(filename string, in io.Reader, out io.Writer, argType argumentT
fmt.Fprintln(out, filename)
}
if *write {
if argType == fromStdin {
// filename is "<standard input>"
return errors.New("can't use -w on stdin")
}
err = ioutil.WriteFile(filename, res, 0)
if err != nil {
return err
}
}
if *doDiff {
if argType == fromStdin {
filename = "stdin.go" // because <standard input>.orig looks silly
}
data, err := diff(src, res, filename)
data, err := diff(src, res)
if err != nil {
return fmt.Errorf("computing diff: %s", err)
}
fmt.Printf("diff -u %s %s\n", filepath.ToSlash(filename+".orig"), filepath.ToSlash(filename))
fmt.Printf("diff %s gofmt/%s\n", filename, filename)
out.Write(data)
}
}
@ -183,7 +113,7 @@ func processFile(filename string, in io.Reader, out io.Writer, argType argumentT
func visitFile(path string, f os.FileInfo, err error) error {
if err == nil && isGoFile(f) {
err = processFile(path, nil, os.Stdout, multipleArg)
err = processFile(path, nil, os.Stdout, false)
}
if err != nil {
report(err)
@ -208,58 +138,14 @@ func main() {
// parseFlags parses command line flags and returns the paths to process.
// It's a var so that custom implementations can replace it in other files.
var parseFlags = func() []string {
flag.BoolVar(&verbose, "v", false, "verbose logging")
flag.Parse()
return flag.Args()
}
func bufferedFileWriter(dest string) (w io.Writer, close func()) {
f, err := os.Create(dest)
if err != nil {
log.Fatal(err)
}
bw := bufio.NewWriter(f)
return bw, func() {
if err := bw.Flush(); err != nil {
log.Fatalf("error flushing %v: %v", dest, err)
}
if err := f.Close(); err != nil {
log.Fatal(err)
}
}
}
func gofmtMain() {
flag.Usage = usage
paths := parseFlags()
if *cpuProfile != "" {
bw, flush := bufferedFileWriter(*cpuProfile)
pprof.StartCPUProfile(bw)
defer flush()
defer pprof.StopCPUProfile()
}
// doTrace is a conditionally compiled wrapper around runtime/trace. It is
// used to allow goimports to compile under gccgo, which does not support
// runtime/trace. See https://golang.org/issue/15544.
defer doTrace()()
if *memProfileRate > 0 {
runtime.MemProfileRate = *memProfileRate
bw, flush := bufferedFileWriter(*memProfile)
defer func() {
runtime.GC() // materialize all statistics
if err := pprof.WriteHeapProfile(bw); err != nil {
log.Fatal(err)
}
flush()
}()
}
if verbose {
log.SetFlags(log.LstdFlags | log.Lmicroseconds)
options.Env.Debug = true
}
if options.TabWidth < 0 {
fmt.Fprintf(os.Stderr, "negative tabwidth %d\n", options.TabWidth)
exitCode = 2
@ -267,17 +153,12 @@ func gofmtMain() {
}
if len(paths) == 0 {
if err := processFile("<standard input>", os.Stdin, os.Stdout, fromStdin); err != nil {
if err := processFile("<standard input>", os.Stdin, os.Stdout, true); err != nil {
report(err)
}
return
}
argType := singleArg
if len(paths) > 1 {
argType = multipleArg
}
for _, path := range paths {
switch dir, err := os.Stat(path); {
case err != nil:
@ -285,93 +166,36 @@ func gofmtMain() {
case dir.IsDir():
walkDir(path)
default:
if err := processFile(path, nil, os.Stdout, argType); err != nil {
if err := processFile(path, nil, os.Stdout, false); err != nil {
report(err)
}
}
}
}
func writeTempFile(dir, prefix string, data []byte) (string, error) {
file, err := ioutil.TempFile(dir, prefix)
if err != nil {
return "", err
}
_, err = file.Write(data)
if err1 := file.Close(); err == nil {
err = err1
}
if err != nil {
os.Remove(file.Name())
return "", err
}
return file.Name(), nil
}
func diff(b1, b2 []byte, filename string) (data []byte, err error) {
f1, err := writeTempFile("", "gofmt", b1)
func diff(b1, b2 []byte) (data []byte, err error) {
f1, err := ioutil.TempFile("", "gofmt")
if err != nil {
return
}
defer os.Remove(f1)
defer os.Remove(f1.Name())
defer f1.Close()
f2, err := writeTempFile("", "gofmt", b2)
f2, err := ioutil.TempFile("", "gofmt")
if err != nil {
return
}
defer os.Remove(f2)
defer os.Remove(f2.Name())
defer f2.Close()
cmd := "diff"
if runtime.GOOS == "plan9" {
cmd = "/bin/ape/diff"
}
f1.Write(b1)
f2.Write(b2)
data, err = exec.Command(cmd, "-u", f1, f2).CombinedOutput()
data, err = exec.Command("diff", "-u", f1.Name(), f2.Name()).CombinedOutput()
if len(data) > 0 {
// diff exits with a non-zero status when the files don't match.
// Ignore that failure as long as we get output.
return replaceTempFilename(data, filename)
err = nil
}
return
}
// replaceTempFilename replaces temporary filenames in diff with actual one.
//
// --- /tmp/gofmt316145376 2017-02-03 19:13:00.280468375 -0500
// +++ /tmp/gofmt617882815 2017-02-03 19:13:00.280468375 -0500
// ...
// ->
// --- path/to/file.go.orig 2017-02-03 19:13:00.280468375 -0500
// +++ path/to/file.go 2017-02-03 19:13:00.280468375 -0500
// ...
func replaceTempFilename(diff []byte, filename string) ([]byte, error) {
bs := bytes.SplitN(diff, []byte{'\n'}, 3)
if len(bs) < 3 {
return nil, fmt.Errorf("got unexpected diff for %s", filename)
}
// Preserve timestamps.
var t0, t1 []byte
if i := bytes.LastIndexByte(bs[0], '\t'); i != -1 {
t0 = bs[0][i:]
}
if i := bytes.LastIndexByte(bs[1], '\t'); i != -1 {
t1 = bs[1][i:]
}
// Always print filepath with slash separator.
f := filepath.ToSlash(filename)
bs[0] = []byte(fmt.Sprintf("--- %s%s", f+".orig", t0))
bs[1] = []byte(fmt.Sprintf("+++ %s%s", f, t1))
return bytes.Join(bs, []byte{'\n'}), nil
}
// isFile reports whether name is a file.
func isFile(name string) bool {
fi, err := os.Stat(name)
return err == nil && fi.Mode().IsRegular()
}
// isDir reports whether name is a directory.
func isDir(name string) bool {
fi, err := os.Stat(name)
return err == nil && fi.IsDir()
}

26
cmd/goimports/goimports_gc.go
View File

@ -1,26 +0,0 @@
// Copyright 2016 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build gc
package main
import (
"flag"
"runtime/trace"
)
var traceProfile = flag.String("trace", "", "trace profile output")
func doTrace() func() {
if *traceProfile != "" {
bw, flush := bufferedFileWriter(*traceProfile)
trace.Start(bw)
return func() {
flush()
trace.Stop()
}
}
return func() {}
}

2
cmd/gomvpkg/main.go
View File

@ -19,7 +19,7 @@ import (
var (
fromFlag = flag.String("from", "", "Import path of package to be moved")
toFlag = flag.String("to", "", "Destination import path for package")
vcsMvCmdFlag = flag.String("vcs_mv_cmd", "", `A template for the version control system's "move directory" command, e.g. "git mv {{.Src}} {{.Dst}}"`)
vcsMvCmdFlag = flag.String("vcs_mv_cmd", "", `A template for the version control system's "move directory" command, e.g. "git mv {{.Src}} {{.Dst}}`)
helpFlag = flag.Bool("help", false, "show usage message")
)

3
cmd/gopls/integration/vscode/.gitignore vendored
View File

@ -1,3 +0,0 @@
gopls*.vsix
out
node_modules

20
cmd/gopls/integration/vscode/.vscode/launch.json vendored
View File

@ -1,20 +0,0 @@
{
"version": "0.2.0",
"configurations": [
{
"name": "Launch Extension",
"type": "extensionHost",
"request": "launch",
"runtimeExecutable": "${execPath}",
"args": [
"--extensionDevelopmentPath=${workspaceFolder}"
],
"stopOnEntry": false,
"sourceMaps": true,
"outFiles": [
"${workspaceFolder}/out/**/*.js"
],
"preLaunchTask": "npm"
},
]
}

29
cmd/gopls/integration/vscode/.vscode/tasks.json vendored
View File

@ -1,29 +0,0 @@
// Available variables which can be used inside of strings.
// ${workspaceFolder}: the root folder of the team
// ${file}: the current opened file
// ${fileBasename}: the current opened file's basename
// ${fileDirname}: the current opened file's dirname
// ${fileExtname}: the current opened file's extension
// ${cwd}: the current working directory of the spawned process
// A task runner that calls the Typescript compiler (tsc) and
// compiles the extension.
{
"version": "2.0.0",
// we want to run npm
"command": "npm",
// the command is a shell script
"type": "shell",
// show the output window only if unrecognized errors occur.
"presentation": {
"reveal": "silent"
},
// we run the custom script "compile" as defined in package.json
"args": [
"run",
"compile"
],
// The tsc compiler is started in watching mode
"isBackground": true,
// use the standard tsc in watch mode problem matcher to find compile problems in the output.
"problemMatcher": "$tsc-watch"
}

21
cmd/gopls/integration/vscode/README.md
View File

@ -1,21 +0,0 @@
# gopls testing extension
An extension for debugging the Go Language Server provided by
https://golang.org/x/tools/cmd/gopls. The code for this extension comes from
a combination of
https://github.com/Microsoft/vscode-extension-samples/blob/master/lsp-sample
and https://github.com/Microsoft/vscode-go.
## Features
* Diagnostics (on file change)
* Completion (Ctrl + Space)
* Jump to definition (F12 or right-click -> Go to Definition)
* Signature help (Ctrl + Shift + Space)
## Installation
To package the extension, run `vsce package` from this directory. To install
the extension, navigate to the "Extensions" panel in VSCode, and select
"Install from VSIX..." from the menu in the top right corner. Choose the
`gopls-1.0.0.vsix file` and reload VSCode.

2065
cmd/gopls/integration/vscode/package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

58
cmd/gopls/integration/vscode/package.json
View File

@ -1,58 +0,0 @@
{
"name": "gopls",
"description": "Go Language Server Client for testing",
"author": "The Go authors",
"license": "SEE LICENSE IN ../../../../LICENSE",
"version": "1.0.0",
"repository": {
"type": "git",
"url": "https://golang.org/x/tools"
},
"publisher": "golang",
"engines": {
"vscode": "^1.23.0"
},
"activationEvents": [
"onLanguage:go"
],
"main": "./out/extension",
"scripts": {
"vscode:prepublish": "tsc -p ./",
"compile": "tsc -watch -p ./",
"postinstall": "node ./node_modules/vscode/bin/install",
"lint": "node ./node_modules/tslint/bin/tslint ./src/*.ts"
},
"extensionDependencies": [],
"dependencies": {
"vscode-languageclient": "~4.3.0"
},
"devDependencies": {
"@types/mocha": "^5.2.5",
"@types/node": "^8.10.39",
"tslint": "^5.11.0",
"typescript": "^3.1.3",
"vscode": "^1.1.24"
},
"contributes": {
"configuration": {
"title": "gopls",
"properties": {
"gopls.flags": {
"type": "array",
"items": {
"type": "string"
},
"default": [],
"description": "Flags to pass to gopls",
"scope": "resource"
},
"gopls.command": {
"type": "string",
"default": "gopls",
"description": "Name of the gopls binary",
"scope": "resource"
}
}
}
}
}

96
cmd/gopls/integration/vscode/src/extension.ts
View File

@ -1,96 +0,0 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
'use strict';
import fs = require('fs');
import lsp = require('vscode-languageclient');
import vscode = require('vscode');
import path = require('path');
export function activate(ctx: vscode.ExtensionContext): void {
// The handleDiagnostics middleware writes to the diagnostics log, in order
// to confirm the order in which the extension received diagnostics.
let r = Math.floor(Math.random() * 100000000);
let diagnosticsLog = fs.openSync('/tmp/diagnostics' + r + '.log', 'w');
let document = vscode.window.activeTextEditor.document;
let config = vscode.workspace.getConfiguration('gopls', document.uri);
let goplsCommand: string = config['command'];
let goplsFlags: string[] = config['flags'];
let serverOptions:
lsp.ServerOptions = {command: getBinPath(goplsCommand), args: goplsFlags};
let clientOptions: lsp.LanguageClientOptions = {
initializationOptions: {},
documentSelector: ['go'],
uriConverters: {
code2Protocol: (uri: vscode.Uri): string =>
(uri.scheme ? uri : uri.with({scheme: 'file'})).toString(),
protocol2Code: (uri: string) => vscode.Uri.parse(uri),
},
middleware: {
handleDiagnostics: (uri: vscode.Uri, diagnostics: vscode.Diagnostic[], next: lsp.HandleDiagnosticsSignature) => {
let diagString = "-------------------------------------------\n";
diagString += uri.toString(); + ": " + diagnostics.length + "\n";
if (diagnostics.length > 0) {
diagString += "\n";
for (const diag of diagnostics) {
diagString += diag.message + "\n";
}
}
fs.writeSync(diagnosticsLog, diagString);
return next(uri, diagnostics);
}
},
revealOutputChannelOn: lsp.RevealOutputChannelOn.Never,
};
const c = new lsp.LanguageClient('gopls', serverOptions, clientOptions);
c.onReady().then(() => {
const capabilities = c.initializeResult && c.initializeResult.capabilities;
if (!capabilities) {
return vscode.window.showErrorMessage(
'The language server is not able to serve any features at the moment.');
}
});
ctx.subscriptions.push(c.start());
}
function getBinPath(toolName: string): string {
toolName = correctBinname(toolName);
let tool = findToolIn(toolName, 'PATH', false);
if (tool) {
return tool;
}
return findToolIn(toolName, 'GOPATH', true);
}
function findToolIn(
toolName: string, envVar: string, appendBinToPath: boolean): string {
let value = process.env[envVar];
if (value) {
let paths = value.split(path.delimiter);
for (let i = 0; i < paths.length; i++) {
let binpath = path.join(paths[i], appendBinToPath ? 'bin' : '', toolName);
if (fileExists(binpath)) {
return binpath;
}
}
}
return null;
}
function fileExists(filePath: string): boolean {
try {
return fs.statSync(filePath).isFile();
} catch (e) {
return false;
}
}
function correctBinname(toolName: string) {
if (process.platform === 'win32')
return toolName + '.exe';
else
return toolName;
}

15
cmd/gopls/integration/vscode/tsconfig.json
View File

@ -1,15 +0,0 @@
{
"compilerOptions": {
"module": "commonjs",
"target": "es6",
"outDir": "out",
"rootDir": "src",
"lib": [
"es6"
],
"sourceMap": true
},
"exclude": [
"node_modules"
]
}

12
cmd/gopls/integration/vscode/tslint.json
View File

@ -1,12 +0,0 @@
{
"rules": {
"indent": [
true,
"tabs"
],
"semicolon": [
true,
"always"
]
}
}

23
cmd/gopls/main.go
View File

@ -1,23 +0,0 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// The gopls command is an LSP server for Go.
// The Language Server Protocol allows any text editor
// to be extended with IDE-like features;
// see https://langserver.org/ for details.
package main // import "golang.org/x/tools/cmd/gopls"
import (
"context"
"os"
"golang.org/x/tools/internal/lsp/cmd"
"golang.org/x/tools/internal/lsp/debug"
"golang.org/x/tools/internal/tool"
)
func main() {
debug.Version += "-cmd.gopls"
tool.Main(context.Background(), cmd.New("gopls-legacy", "", nil), os.Args[1:])
}

385
cmd/gorename/gorename_test.go
View File

@ -1,385 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main_test
import (
"fmt"
"io/ioutil"
"os"
"os/exec"
"path/filepath"
"runtime"
"strconv"
"strings"
"testing"
)
var haveCGO bool
type test struct {
offset, from, to string // specify the arguments
fileSpecified bool // true if the offset or from args specify a specific file
pkgs map[string][]string
wantErr bool
wantOut string // a substring expected to be in the output
packages map[string][]string // a map of the package name to the files contained within, which will be numbered by i.go where i is the index
}
// Test that renaming that would modify cgo files will produce an error and not modify the file.
func TestGeneratedFiles(t *testing.T) {
if !haveCGO {
t.Skipf("skipping test: no cgo")
}
tmp, bin, cleanup := buildGorename(t)
defer cleanup()
srcDir := filepath.Join(tmp, "src")
err := os.Mkdir(srcDir, os.ModePerm)
if err != nil {
t.Fatal(err)
}
var env = []string{fmt.Sprintf("GOPATH=%s", tmp)}
for _, envVar := range os.Environ() {
if !strings.HasPrefix(envVar, "GOPATH=") {
env = append(env, envVar)
}
}
// gorename currently requires GOPATH mode.
env = append(env, "GO111MODULE=off")
// Testing renaming in packages that include cgo files:
for iter, renameTest := range []test{
{
// Test: variable not used in any cgo file -> no error
from: `"mytest"::f`, to: "g",
packages: map[string][]string{
"mytest": []string{`package mytest; func f() {}`,
`package mytest
// #include <stdio.h>
import "C"
func z() {C.puts(nil)}`},
},
wantErr: false,
wantOut: "Renamed 1 occurrence in 1 file in 1 package.",
}, {
// Test: to name used in cgo file -> rename error
from: `"mytest"::f`, to: "g",
packages: map[string][]string{
"mytest": []string{`package mytest; func f() {}`,
`package mytest
// #include <stdio.h>
import "C"
func g() {C.puts(nil)}`},
},
wantErr: true,
wantOut: "conflicts with func in same block",
},
{
// Test: from name in package in cgo file -> error
from: `"mytest"::f`, to: "g",
packages: map[string][]string{
"mytest": []string{`package mytest
// #include <stdio.h>
import "C"
func f() { C.puts(nil); }
`},
},
wantErr: true,
wantOut: "gorename: refusing to modify generated file containing DO NOT EDIT marker:",
}, {
// Test: from name in cgo file -> error
from: filepath.Join("mytest", "0.go") + `::f`, to: "g",
fileSpecified: true,
packages: map[string][]string{
"mytest": []string{`package mytest
// #include <stdio.h>
import "C"
func f() { C.puts(nil); }
`},
},
wantErr: true,
wantOut: "gorename: refusing to modify generated file containing DO NOT EDIT marker:",
}, {
// Test: offset in cgo file -> identifier in cgo error
offset: filepath.Join("main", "0.go") + `:#78`, to: "bar",
fileSpecified: true,
wantErr: true,
packages: map[string][]string{
"main": {`package main
// #include <unistd.h>
import "C"
import "fmt"
func main() {
foo := 1
C.close(2)
fmt.Println(foo)
}
`},
},
wantOut: "cannot rename identifiers in generated file containing DO NOT EDIT marker:",
}, {
// Test: from identifier appears in cgo file in another package -> error
from: `"test"::Foo`, to: "Bar",
packages: map[string][]string{
"test": []string{
`package test
func Foo(x int) (int){
return x * 2
}
`,
},
"main": []string{
`package main
import "test"
import "fmt"
// #include <unistd.h>
import "C"
func fun() {
x := test.Foo(3)
C.close(3)
fmt.Println(x)
}
`,
},
},
wantErr: true,
wantOut: "gorename: refusing to modify generated file containing DO NOT EDIT marker:",
}, {
// Test: from identifier doesn't appear in cgo file that includes modified package -> rename successful
from: `"test".Foo::x`, to: "y",
packages: map[string][]string{
"test": []string{
`package test
func Foo(x int) (int){
return x * 2
}
`,
},
"main": []string{
`package main
import "test"
import "fmt"
// #include <unistd.h>
import "C"
func fun() {
x := test.Foo(3)
C.close(3)
fmt.Println(x)
}
`,
},
},
wantErr: false,
wantOut: "Renamed 2 occurrences in 1 file in 1 package.",
}, {
// Test: from name appears in cgo file in same package -> error
from: `"mytest"::f`, to: "g",
packages: map[string][]string{
"mytest": []string{`package mytest; func f() {}`,
`package mytest
// #include <stdio.h>
import "C"
func z() {C.puts(nil); f()}`,
`package mytest
// #include <unistd.h>
import "C"
func foo() {C.close(3); f()}`,
},
},
wantErr: true,
wantOut: "gorename: refusing to modify generated files containing DO NOT EDIT marker:",
}, {
// Test: from name in file, identifier not used in cgo file -> rename successful
from: filepath.Join("mytest", "0.go") + `::f`, to: "g",
fileSpecified: true,
packages: map[string][]string{
"mytest": []string{`package mytest; func f() {}`,
`package mytest
// #include <stdio.h>
import "C"
func z() {C.puts(nil)}`},
},
wantErr: false,
wantOut: "Renamed 1 occurrence in 1 file in 1 package.",
}, {
// Test: from identifier imported to another package but does not modify cgo file -> rename successful
from: `"test".Foo`, to: "Bar",
packages: map[string][]string{
"test": []string{
`package test
func Foo(x int) (int){
return x * 2
}
`,
},
"main": []string{
`package main
// #include <unistd.h>
import "C"
func fun() {
C.close(3)
}
`,
`package main
import "test"
import "fmt"
func g() { fmt.Println(test.Foo(3)) }
`,
},
},
wantErr: false,
wantOut: "Renamed 2 occurrences in 2 files in 2 packages.",
},
} {
// Write the test files
testCleanup := setUpPackages(t, srcDir, renameTest.packages)
// Set up arguments
var args []string
var arg, val string
if renameTest.offset != "" {
arg, val = "-offset", renameTest.offset
} else {
arg, val = "-from", renameTest.from
}
prefix := fmt.Sprintf("%d: %s %q -to %q", iter, arg, val, renameTest.to)
if renameTest.fileSpecified {
// add the src dir to the value of the argument
val = filepath.Join(srcDir, val)
}
args = append(args, arg, val, "-to", renameTest.to)
// Run command
cmd := exec.Command(bin, args...)
cmd.Args[0] = "gorename"
cmd.Env = env
// Check the output
out, err := cmd.CombinedOutput()
// errors should result in no changes to files
if err != nil {
if !renameTest.wantErr {
t.Errorf("%s: received unexpected error %s", prefix, err)
}
// Compare output
if ok := strings.Contains(string(out), renameTest.wantOut); !ok {
t.Errorf("%s: unexpected command output: %s (want: %s)", prefix, out, renameTest.wantOut)
}
// Check that no files were modified
if modified := modifiedFiles(t, srcDir, renameTest.packages); len(modified) != 0 {
t.Errorf("%s: files unexpectedly modified: %s", prefix, modified)
}
} else {
if !renameTest.wantErr {
if ok := strings.Contains(string(out), renameTest.wantOut); !ok {
t.Errorf("%s: unexpected command output: %s (want: %s)", prefix, out, renameTest.wantOut)
}
} else {
t.Errorf("%s: command succeeded unexpectedly, output: %s", prefix, out)
}
}
testCleanup()
}
}
// buildGorename builds the gorename executable.
// It returns its path, and a cleanup function.
func buildGorename(t *testing.T) (tmp, bin string, cleanup func()) {
if runtime.GOOS == "android" {
t.Skipf("the dependencies are not available on android")
}
tmp, err := ioutil.TempDir("", "gorename-regtest-")
if err != nil {
t.Fatal(err)
}
defer func() {
if cleanup == nil { // probably, go build failed.
os.RemoveAll(tmp)
}
}()
bin = filepath.Join(tmp, "gorename")
if runtime.GOOS == "windows" {
bin += ".exe"
}
cmd := exec.Command("go", "build", "-o", bin)
if err := cmd.Run(); err != nil {
t.Fatalf("Building gorename: %v", err)
}
return tmp, bin, func() { os.RemoveAll(tmp) }
}
// setUpPackages sets up the files in a temporary directory provided by arguments.
func setUpPackages(t *testing.T, dir string, packages map[string][]string) (cleanup func()) {
var pkgDirs []string
for pkgName, files := range packages {
// Create a directory for the package.
pkgDir := filepath.Join(dir, pkgName)
pkgDirs = append(pkgDirs, pkgDir)
if err := os.Mkdir(pkgDir, os.ModePerm); err != nil {
t.Fatal(err)
}
// Write the packages files
for i, val := range files {
file := filepath.Join(pkgDir, strconv.Itoa(i)+".go")
if err := ioutil.WriteFile(file, []byte(val), os.ModePerm); err != nil {
t.Fatal(err)
}
}
}
return func() {
for _, dir := range pkgDirs {
os.RemoveAll(dir)
}
}
}
// modifiedFiles returns a list of files that were renamed (without the prefix dir).
func modifiedFiles(t *testing.T, dir string, packages map[string][]string) (results []string) {
for pkgName, files := range packages {
pkgDir := filepath.Join(dir, pkgName)
for i, val := range files {
file := filepath.Join(pkgDir, strconv.Itoa(i)+".go")
// read file contents and compare to val
if contents, err := ioutil.ReadFile(file); err != nil {
t.Fatalf("File missing: %s", err)
} else if string(contents) != val {
results = append(results, strings.TrimPrefix(dir, file))
}
}
}
return results
}

62
cmd/gotype/doc.go Normal file
View File

@ -0,0 +1,62 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
/*
The gotype command does syntactic and semantic analysis of Go files
and packages like the front-end of a Go compiler. Errors are reported
if the analysis fails; otherwise gotype is quiet (unless -v is set).
Without a list of paths, gotype reads from standard input, which
must provide a single Go source file defining a complete package.
If a single path is specified that is a directory, gotype checks
the Go files in that directory; they must all belong to the same
package.
Otherwise, each path must be the filename of Go file belonging to
the same package.
Usage:
gotype [flags] [path...]
The flags are:
-a
use all (incl. _test.go) files when processing a directory
-e
report all errors (not just the first 10)
-v
verbose mode
-gccgo
use gccimporter instead of gcimporter
Debugging flags:
-seq
parse sequentially, rather than in parallel
-ast
print AST (forces -seq)
-trace
print parse trace (forces -seq)
-comments
parse comments (ignored unless -ast or -trace is provided)
Examples:
To check the files a.go, b.go, and c.go:
gotype a.go b.go c.go
To check an entire package in the directory dir and print the processed files:
gotype -v dir
To check an entire package including tests in the local directory:
gotype -a .
To verify the output of a pipe:
echo "package foo" | gotype
*/
package main // import "golang.org/x/tools/cmd/gotype"

218
cmd/gotype/gotype.go
View File

@ -2,87 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// gotype.go is a copy of the original source maintained
// in $GOROOT/src/go/types/gotype.go, but with the call
// to types.SizesFor factored out so we can provide a local
// implementation when compiling against Go 1.8 and earlier.
//
// This code is here for the sole purpose of satisfying historic
// references to this location, and for making gotype accessible
// via 'go get'.
//
// Do NOT make changes to this version as they will not be maintained
// (and possibly overwritten). Any changes should be made to the original
// and then ported to here.
// +build go1.5
/*
The gotype command, like the front-end of a Go compiler, parses and
type-checks a single Go package. Errors are reported if the analysis
fails; otherwise gotype is quiet (unless -v is set).
Without a list of paths, gotype reads from standard input, which
must provide a single Go source file defining a complete package.
With a single directory argument, gotype checks the Go files in
that directory, comprising a single package. Use -t to include the
(in-package) _test.go files. Use -x to type check only external
test files.
Otherwise, each path must be the filename of a Go file belonging
to the same package.
Imports are processed by importing directly from the source of
imported packages (default), or by importing from compiled and
installed packages (by setting -c to the respective compiler).
The -c flag must be set to a compiler ("gc", "gccgo") when type-
checking packages containing imports with relative import paths
(import "./mypkg") because the source importer cannot know which
files to include for such packages.
Usage:
gotype [flags] [path...]
The flags are:
-t
include local test files in a directory (ignored if -x is provided)
-x
consider only external test files in a directory
-e
report all errors (not just the first 10)
-v
verbose mode
-c
compiler used for installed packages (gc, gccgo, or source); default: source
Flags controlling additional output:
-ast
print AST (forces -seq)
-trace
print parse trace (forces -seq)
-comments
parse comments (ignored unless -ast or -trace is provided)
Examples:
To check the files a.go, b.go, and c.go:
gotype a.go b.go c.go
To check an entire package including (in-package) tests in the directory dir and print the processed files:
gotype -t -v dir
To check the external test package (if any) in the current directory, based on installed packages compiled with
cmd/compile:
gotype -c=gc -x .
To verify the output of a pipe:
echo "package foo" | gotype
*/
package main
import (
@ -98,19 +19,18 @@ import (
"io/ioutil"
"os"
"path/filepath"
"sync"
"time"
)
var (
// main operation modes
testFiles = flag.Bool("t", false, "include in-package test files in a directory")
xtestFiles = flag.Bool("x", false, "consider only external test files in a directory")
allErrors = flag.Bool("e", false, "report all errors, not just the first 10")
verbose = flag.Bool("v", false, "verbose mode")
compiler = flag.String("c", defaultCompiler, "compiler used for installed packages (gc, gccgo, or source)")
allFiles = flag.Bool("a", false, "use all (incl. _test.go) files when processing a directory")
allErrors = flag.Bool("e", false, "report all errors (not just the first 10)")
verbose = flag.Bool("v", false, "verbose mode")
gccgo = flag.Bool("gccgo", false, "use gccgoimporter instead of gcimporter")
// additional output control
// debugging support
sequential = flag.Bool("seq", false, "parse sequentially, rather than in parallel")
printAST = flag.Bool("ast", false, "print AST (forces -seq)")
printTrace = flag.Bool("trace", false, "print parse trace (forces -seq)")
parseComments = flag.Bool("comments", false, "parse comments (ignored unless -ast or -trace is provided)")
@ -119,55 +39,36 @@ var (
var (
fset = token.NewFileSet()
errorCount = 0
sequential = false
parserMode parser.Mode
sizes types.Sizes
)
func initParserMode() {
if *allErrors {
parserMode |= parser.AllErrors
}
if *printAST {
sequential = true
}
if *printTrace {
parserMode |= parser.Trace
sequential = true
}
if *parseComments && (*printAST || *printTrace) {
parserMode |= parser.ParseComments
}
}
const usageString = `usage: gotype [flags] [path ...]
The gotype command, like the front-end of a Go compiler, parses and
type-checks a single Go package. Errors are reported if the analysis
fails; otherwise gotype is quiet (unless -v is set).
Without a list of paths, gotype reads from standard input, which
must provide a single Go source file defining a complete package.
With a single directory argument, gotype checks the Go files in
that directory, comprising a single package. Use -t to include the
(in-package) _test.go files. Use -x to type check only external
test files.
Otherwise, each path must be the filename of a Go file belonging
to the same package.
Imports are processed by importing directly from the source of
imported packages (default), or by importing from compiled and
installed packages (by setting -c to the respective compiler).
The -c flag must be set to a compiler ("gc", "gccgo") when type-
checking packages containing imports with relative import paths
(import "./mypkg") because the source importer cannot know which
files to include for such packages.
`
func initSizes() {
wordSize := 8
maxAlign := 8
switch build.Default.GOARCH {
case "386", "arm":
wordSize = 4
maxAlign = 4
// add more cases as needed
}
sizes = &types.StdSizes{WordSize: int64(wordSize), MaxAlign: int64(maxAlign)}
}
func usage() {
fmt.Fprintln(os.Stderr, usageString)
fmt.Fprintln(os.Stderr, "usage: gotype [flags] [path ...]")
flag.PrintDefaults()
os.Exit(2)
}
@ -201,49 +102,60 @@ func parseStdin() (*ast.File, error) {
return parse("<standard input>", src)
}
func parseFiles(dir string, filenames []string) ([]*ast.File, error) {
func parseFiles(filenames []string) ([]*ast.File, error) {
files := make([]*ast.File, len(filenames))
errors := make([]error, len(filenames))
var wg sync.WaitGroup
for i, filename := range filenames {
wg.Add(1)
go func(i int, filepath string) {
defer wg.Done()
files[i], errors[i] = parse(filepath, nil)
}(i, filepath.Join(dir, filename))
if sequential {
wg.Wait()
if *sequential {
for i, filename := range filenames {
var err error
files[i], err = parse(filename, nil)
if err != nil {
return nil, err // leave unfinished goroutines hanging
}
}
} else {
type parseResult struct {
file *ast.File
err error
}
}
wg.Wait()
// if there are errors, return the first one for deterministic results
for _, err := range errors {
if err != nil {
return nil, err
out := make(chan parseResult)
for _, filename := range filenames {
go func(filename string) {
file, err := parse(filename, nil)
out <- parseResult{file, err}
}(filename)
}
for i := range filenames {
res := <-out
if res.err != nil {
return nil, res.err // leave unfinished goroutines hanging
}
files[i] = res.file
}
}
return files, nil
}
func parseDir(dir string) ([]*ast.File, error) {
func parseDir(dirname string) ([]*ast.File, error) {
ctxt := build.Default
pkginfo, err := ctxt.ImportDir(dir, 0)
pkginfo, err := ctxt.ImportDir(dirname, 0)
if _, nogo := err.(*build.NoGoError); err != nil && !nogo {
return nil, err
}
if *xtestFiles {
return parseFiles(dir, pkginfo.XTestGoFiles)
}
filenames := append(pkginfo.GoFiles, pkginfo.CgoFiles...)
if *testFiles {
if *allFiles {
filenames = append(filenames, pkginfo.TestGoFiles...)
}
return parseFiles(dir, filenames)
// complete file names
for i, filename := range filenames {
filenames[i] = filepath.Join(dirname, filename)
}
return parseFiles(filenames)
}
func getPkgFiles(args []string) ([]*ast.File, error) {
@ -269,13 +181,15 @@ func getPkgFiles(args []string) ([]*ast.File, error) {
}
// list of files
return parseFiles("", args)
return parseFiles(args)
}
func checkPkgFiles(files []*ast.File) {
compiler := "gc"
if *gccgo {
compiler = "gccgo"
}
type bailout struct{}
// if checkPkgFiles is called multiple times, set up conf only once
conf := types.Config{
FakeImportC: true,
Error: func(err error) {
@ -284,8 +198,8 @@ func checkPkgFiles(files []*ast.File) {
}
report(err)
},
Importer: importer.For(*compiler, nil),
Sizes: SizesFor(build.Default.Compiler, build.Default.GOARCH),
Importer: importer.For(compiler, nil),
Sizes: sizes,
}
defer func() {
@ -320,7 +234,11 @@ func printStats(d time.Duration) {
func main() {
flag.Usage = usage
flag.Parse()
if *printAST || *printTrace {
*sequential = true
}
initParserMode()
initSizes()
start := time.Now()

268
cmd/gotype/gotype14.go Normal file
View File

@ -0,0 +1,268 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !go1.5
// This is a 1:1 copy of gotype.go but for the changes required to build
// against Go1.4 and before.
// TODO(gri) Decide long-term fate of gotype (issue #12303).
package main
import (
"flag"
"fmt"
"go/ast"
"go/build"
"go/parser"
"go/scanner"
"go/token"
"io/ioutil"
"os"
"path/filepath"
"runtime"
"time"
"golang.org/x/tools/go/gccgoimporter"
_ "golang.org/x/tools/go/gcimporter"
"golang.org/x/tools/go/types"
)
var (
// main operation modes
allFiles = flag.Bool("a", false, "use all (incl. _test.go) files when processing a directory")
allErrors = flag.Bool("e", false, "report all errors (not just the first 10)")
verbose = flag.Bool("v", false, "verbose mode")
gccgo = flag.Bool("gccgo", false, "use gccgoimporter instead of gcimporter")
// debugging support
sequential = flag.Bool("seq", false, "parse sequentially, rather than in parallel")
printAST = flag.Bool("ast", false, "print AST (forces -seq)")
printTrace = flag.Bool("trace", false, "print parse trace (forces -seq)")
parseComments = flag.Bool("comments", false, "parse comments (ignored unless -ast or -trace is provided)")
)
var (
fset = token.NewFileSet()
errorCount = 0
parserMode parser.Mode
sizes types.Sizes
)
func initParserMode() {
if *allErrors {
parserMode |= parser.AllErrors
}
if *printTrace {
parserMode |= parser.Trace
}
if *parseComments && (*printAST || *printTrace) {
parserMode |= parser.ParseComments
}
}
func initSizes() {
wordSize := 8
maxAlign := 8
switch build.Default.GOARCH {
case "386", "arm":
wordSize = 4
maxAlign = 4
// add more cases as needed
}
sizes = &types.StdSizes{WordSize: int64(wordSize), MaxAlign: int64(maxAlign)}
}
func usage() {
fmt.Fprintln(os.Stderr, "usage: gotype [flags] [path ...]")
flag.PrintDefaults()
os.Exit(2)
}
func report(err error) {
scanner.PrintError(os.Stderr, err)
if list, ok := err.(scanner.ErrorList); ok {
errorCount += len(list)
return
}
errorCount++
}
// parse may be called concurrently
func parse(filename string, src interface{}) (*ast.File, error) {
if *verbose {
fmt.Println(filename)
}
file, err := parser.ParseFile(fset, filename, src, parserMode) // ok to access fset concurrently
if *printAST {
ast.Print(fset, file)
}
return file, err
}
func parseStdin() (*ast.File, error) {
src, err := ioutil.ReadAll(os.Stdin)
if err != nil {
return nil, err
}
return parse("<standard input>", src)
}
func parseFiles(filenames []string) ([]*ast.File, error) {
files := make([]*ast.File, len(filenames))
if *sequential {
for i, filename := range filenames {
var err error
files[i], err = parse(filename, nil)
if err != nil {
return nil, err // leave unfinished goroutines hanging
}
}
} else {
type parseResult struct {
file *ast.File
err error
}
out := make(chan parseResult)
for _, filename := range filenames {
go func(filename string) {
file, err := parse(filename, nil)
out <- parseResult{file, err}
}(filename)
}
for i := range filenames {
res := <-out
if res.err != nil {
return nil, res.err // leave unfinished goroutines hanging
}
files[i] = res.file
}
}
return files, nil
}
func parseDir(dirname string) ([]*ast.File, error) {
ctxt := build.Default
pkginfo, err := ctxt.ImportDir(dirname, 0)
if _, nogo := err.(*build.NoGoError); err != nil && !nogo {
return nil, err
}
filenames := append(pkginfo.GoFiles, pkginfo.CgoFiles...)
if *allFiles {
filenames = append(filenames, pkginfo.TestGoFiles...)
}
// complete file names
for i, filename := range filenames {
filenames[i] = filepath.Join(dirname, filename)
}
return parseFiles(filenames)
}
func getPkgFiles(args []string) ([]*ast.File, error) {
if len(args) == 0 {
// stdin
file, err := parseStdin()
if err != nil {
return nil, err
}
return []*ast.File{file}, nil
}
if len(args) == 1 {
// possibly a directory
path := args[0]
info, err := os.Stat(path)
if err != nil {
return nil, err
}
if info.IsDir() {
return parseDir(path)
}
}
// list of files
return parseFiles(args)
}
func checkPkgFiles(files []*ast.File) {
type bailout struct{}
conf := types.Config{
FakeImportC: true,
Error: func(err error) {
if !*allErrors && errorCount >= 10 {
panic(bailout{})
}
report(err)
},
Sizes: sizes,
}
if *gccgo {
var inst gccgoimporter.GccgoInstallation
inst.InitFromDriver("gccgo")
conf.Import = inst.GetImporter(nil, nil)
}
defer func() {
switch p := recover().(type) {
case nil, bailout:
// normal return or early exit
default:
// re-panic
panic(p)
}
}()
const path = "pkg" // any non-empty string will do for now
conf.Check(path, fset, files, nil)
}
func printStats(d time.Duration) {
fileCount := 0
lineCount := 0
fset.Iterate(func(f *token.File) bool {
fileCount++
lineCount += f.LineCount()
return true
})
fmt.Printf(
"%s (%d files, %d lines, %d lines/s)\n",
d, fileCount, lineCount, int64(float64(lineCount)/d.Seconds()),
)
}
func main() {
runtime.GOMAXPROCS(runtime.NumCPU()) // not needed for go1.5
flag.Usage = usage
flag.Parse()
if *printAST || *printTrace {
*sequential = true
}
initParserMode()
initSizes()
start := time.Now()
files, err := getPkgFiles(flag.Args())
if err != nil {
report(err)
os.Exit(2)
}
checkPkgFiles(files)
if errorCount > 0 {
os.Exit(2)
}
if *verbose {
printStats(time.Since(start))
}
}

40
cmd/gotype/sizesFor18.go
View File

@ -1,40 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !go1.9
// This file contains a copy of the implementation of types.SizesFor
// since this function is not available in go/types before Go 1.9.
package main
import "go/types"
const defaultCompiler = "gc"
var gcArchSizes = map[string]*types.StdSizes{
"386": {4, 4},
"arm": {4, 4},
"arm64": {8, 8},
"amd64": {8, 8},
"amd64p32": {4, 8},
"mips": {4, 4},
"mipsle": {4, 4},
"mips64": {8, 8},
"mips64le": {8, 8},
"ppc64": {8, 8},
"ppc64le": {8, 8},
"s390x": {8, 8},
}
func SizesFor(compiler, arch string) types.Sizes {
if compiler != "gc" {
return nil
}
s, ok := gcArchSizes[arch]
if !ok {
return nil
}
return s
}

15
cmd/gotype/sizesFor19.go
View File

@ -1,15 +0,0 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build go1.9
package main
import "go/types"
const defaultCompiler = "source"
func SizesFor(compiler, arch string) types.Sizes {
return types.SizesFor(compiler, arch)
}

70
cmd/goyacc/doc.go
View File

@ -1,70 +0,0 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
/*
Goyacc is a version of yacc for Go.
It is written in Go and generates parsers written in Go.
Usage:
goyacc args...
It is largely transliterated from the Inferno version written in Limbo
which in turn was largely transliterated from the Plan 9 version
written in C and documented at
https://9p.io/magic/man2html/1/yacc
Adepts of the original yacc will have no trouble adapting to this
form of the tool.
The directory $GOPATH/src/golang.org/x/tools/cmd/goyacc/testdata/expr
is a yacc program for a very simple expression parser. See expr.y and
main.go in that directory for examples of how to write and build
goyacc programs.
The generated parser is reentrant. The parsing function yyParse expects
to be given an argument that conforms to the following interface:
type yyLexer interface {
Lex(lval *yySymType) int
Error(e string)
}
Lex should return the token identifier, and place other token
information in lval (which replaces the usual yylval).
Error is equivalent to yyerror in the original yacc.
Code inside the grammar actions may refer to the variable yylex,
which holds the yyLexer passed to yyParse.
Clients that need to understand more about the parser state can
create the parser separately from invoking it. The function yyNewParser
returns a yyParser conforming to the following interface:
type yyParser interface {
Parse(yyLex) int
Lookahead() int
}
Parse runs the parser; the top-level call yyParse(yylex) is equivalent
to yyNewParser().Parse(yylex).
Lookahead can be called during grammar actions to read (but not consume)
the value of the current lookahead token, as returned by yylex.Lex.
If there is no current lookahead token (because the parser has not called Lex
or has consumed the token returned by the most recent call to Lex),
Lookahead returns -1. Calling Lookahead is equivalent to reading
yychar from within in a grammar action.
Multiple grammars compiled into a single program should be placed in
distinct packages. If that is impossible, the "-p prefix" flag to
goyacc sets the prefix, by default yy, that begins the names of
symbols, including types, the parser, and the lexer, generated and
referenced by yacc's generated code. Setting it to distinct values
allows multiple grammars to be placed in a single package.
*/
package main

20
cmd/goyacc/testdata/expr/README vendored
View File

@ -1,20 +0,0 @@
This directory contains a simple program demonstrating how to use
the Go version of yacc.
To build it:
$ go generate
$ go build
or
$ go generate
$ go run expr.go
The file main.go contains the "go generate" command to run yacc to
create expr.go from expr.y. It also has the package doc comment,
as godoc will not scan the .y file.
The actual implementation is in expr.y.
The program is not installed in the binary distributions of Go.

Some files were not shown because too many files have changed in this diff Show More