Static Analysis for Fun and Profit: Part 1

A bug in plain sight

A large part of my team’s responsibility at Justworks involves gluing together data from different sources, massaging it into a unified structure, and then returning out the other side. In our code, you’ll see something like this a lot:

type Response struct {
 Status PersonStatus `json:"status"`
}

type PersonStatus string

const (
 Active   PersonStatus = "active"
 Inactive PersonStatus = "inactive"
)

type SourcePerson struct {
 Status string `json:"status"`
}

func mapSourcePerson(sp SourcePerson) Response {
 return Response{
  Status: PersonStatus(sp.Status),
 }
}

We take a SourcePerson, map their values to our unified Response, and call it a day. The only problem is that this is perfectly legal, perfectly compliant, perfectly compiling code hides a nasty bug...perfectly, and is a common issue around Go’s lack of true support for enums(opens in a new tab). To appease the compiler, we can't use SourcePerson's Status as a PersonStatus, so a type conversion is used. After all, they're both strings under the hood. The danger is this: imagine a source had the value Active or even an entirely different value like Waiting. By piping those through, we’ve accidentally exposed a new value not specified by our API's contract, wreaking havoc for any callers of our service.

To check for this sort of scenario is not only difficult for a single reviewer to do by hand, but imagine you want to scale this sort of knowledge. Writing it down is great, but still subject to human error or unaware PR approvers. This is when we looked into writing a “static analyzer” to catch this sort of issue in our PR’s and across the codebase before they become a problem.

Analysis and the Abstract Syntax Tree

If you’ve ever used a linter, go vet or go fmt, rubocop, etc. you've used a static analyzer.Static analysis, in a few words, is the act of looking at your code before it runs, giving you feedback about possible errors before your code is ever executed. The “static” part means “unchanging”, as opposed to “runtime” analysis, where your program is executing, moving memory around and changing state. They’re wonderful for:

• Codifying engineering guidelines for your org

• Surfacing issues before deployment that the compiler doesn’t detect

• Enforcing style or different “tastes” in your codebase

These tools utilize the same few passes that interpreters and compilers take to provide information about the source in the program itself to take action or warn about danger. You could write a regex to scan your source for certain patterns, but to do a proper static analysis, we need to take our Go files and get them into an Abstract Syntax Tree(opens in a new tab), or AST for short. Meaning a snippet such as:

c := a + b

gets turned into something like:

To get there, the tool chain starts with an invocation to the lexer(opens in a new tab). The lexer looks at the characters and words on this line and turns them into tokens or symbols, which are code representations of the text (the following are some pseudo-values of the output):

[IDENTIFIER C, EQUALS_ASSIGNMENT, IDENTIFIER A, PLUS_SIGN, IDENTIFIER B]

You’ll notice they’re close to representations of the source statement itself. To add the meaning of these tokens in that specific order, the Go parser(opens in a new tab) takes over and turns it into types from the ast(opens in a new tab) package:

&ast.AssignStmt{
  Lhs: []ast.Expr{
    &ast.Ident{
      NamePos: 2359391,
      Name: "c",
      Obj: &ast.Object{
        Kind: token.INT,
      },
    },
  },
  TokPos: 2359393,
  Tok: token.DEFINE,
  Rhs: []ast.Expr{
    &ast.BinaryExpr{
      X: &ast.Ident{
        NamePos: 2359396,
        Name: "a",
        Obj: &ast.Object{
          Kind: token.INT,
          Name: "a",
        },
      },
      OpPos: 2359398,
      Op: token.ADD,
      Y: &ast.Ident{
        NamePos: 2359400,
        Name: "b",
        Obj: &ast.Object{
          Kind: token.INT,
          Name: "b",
        },
      },
    },
  },
}

Now the tokens that didn’t mean much before, tell us a lot more, are arranged in a syntax tree, describe positions in the source where the variables come from, and even have type information attached to them.

The root of the tree is the AssignStmt, which is an “assignment statement”, featuring left and right-hand sides, called Lhs and Rhs respectively. It also has a Tok, which refers to the token being used in the assignment: here it’s the DEFINE token because we used := instead of a =.

The Lhs value is a slice of ast.Expr, which is a universal interface that includes all of the different expressions in the ast package (we’ll be casting from this type a lot). In that slice, we see the identifier, ast.Ident, for our variable c.

The Rhs gets more interesting: Rhs is a slice of expressions, but this side also has a single expression, called a “Binary Expression”. This just means it’s an expression that takes two operands and does something to them, here called X and Y. The Op tells us what operation is taking place, token.ADD for addition, and the fields' names and types are given to us.

It’s at this level of detail that our analysis will function: source code types arrange in meaningful ways that can indicate to us not just the statements of our program, but the meaning at each step. We’ll operate at this depth to detect different scenarios to create warnings and hopefully catch dangerous bugs before they make it to production.

The go/analysis package

The Go standard library introduced the golang.org/x/tools/go/analysis(opens in a new tab) package back in 2021, giving a bunch of functionality to folks who want to build their own static analysis. First, let’s get the basics down by making an analysis that warns on every string assignment in our code called noString (we’ll tackle our full problem in our next post). This won't be the most useful analyzer in practice, but introduces the core concepts. The starting point is an analysis.Analyzer, which is the declaration of a new analysis:

// main.go
var Analyzer = &analysis.Analyzer{
 Name: "noString",
 Doc:  "Reports any string assignments",
 Requires: []*analysis.Analyzer{
  inspect.Analyzer,
 },
 Run:       run,
}

func run(pass *analysis.Pass) (any, error) {
 // ...
}

The Analyzer instance needs a name, a description, and a function called Run that will later be invoked by the framework code to perform your logic. The Run function will have passed to it an analysis.Pass variable, which is the main way this analyzer will access the results of other runs as well as reporting its own. In the Requires field you can specify other Analyzers that yours needs to have run first, implying to the framework that other analyses need to be invoked before this one has enough information to do its own work. One required analyzer you'll see quite often is the inspect.Analyzer(opens in a new tab), which tells the framework to wait until we’ve produced an AST (like the one we saw earlier) so we’ll have all that information to operate upon.

Now that we’ve described our Anaylzer instance, we need to figure out what goes into our run function. Remember: we want to look at any place where we assigned a string. First, go a bit broad and ask it for any all assignments:

// main.go
func run(pass *analysis.Pass) (any, error) {
 inspect := pass.ResultOf[inspect.Analyzer].(*inspector.Inspector)

 filter := []ast.Node{
  (*ast.AssignStmt)(nil),
 }
 inspect.Preorder(filter, func(n ast.Node) {
  assigntStmt := n.(*ast.AssignStmt)
 })

 return nil, nil
}

First a line to get the results of the inspection pass out into a variable. This is a type of ast.Inspector(opens in a new tab), a window into exploring the AST of a program:

inspect := pass.ResultOf[inspect.Analyzer].(*inspector.Inspector)

Next, create a filter of different ast.Node(opens in a new tab)’s (these are just like the ast.Expr that we saw earlier) that we want to search the inspector’s previous pass for. This will narrow down which nodes in the AST that we want to look at so we don’t end up looking at the every single bit of the program:

filter := []ast.Node{
 (*ast.AssignStmt)(nil),
}

The result of the inspector pass has a method called Preorder(opens in a new tab) that will receive our filter. This will run a callback function on each node that it finds that passed the filter:

inspect.Preorder(filter, func(n ast.Node) {
 assigntStmt := n.(*ast.AssignStmt)
})

Notice that we’re given an ast.Node again in the callback function, so just convert it back into the concrete type we want to inspect, our assignment statement.

The anonymous function provided will be provided every instance of an assignment, but the linter we want to write only wants string assignments. Our next goal is to check for a series of increasingly more narrow conditions, exiting early until we’re left with something we are confident is that string assignment. First, ignore any assignment that has more than 1 expression on the right hand side:

if len(assignStmt.Rhs) != 1 {
 return
}

Next, use a safe type assertion(opens in a new tab) to check that the right hand side is a “literal”. Literals refer to code where the value is spelled out, like a number, string, slice, or struct:

rhs, ok := assignStmt.Rhs[0].(*ast.BasicLit)
if !ok {
 return
}

Once the right-hand-side is assured to be a literal, make sure it’s a string literal:

if rhs.Kind != token.STRING {
 return
}

At this point in our sieve, the function has gotten pretty confident that a single string is being assigned to something and needs to report this to the overall analyzer pass. To do that, use pass.Report(opens in a new tab) with an analysis.Diagnostic(opens in a new tab) containing a message, positions of where the issue occurs (the expressions have locations of where they start and end in the file), and if you wanted, a suggestion to fix:

pass.Report(analysis.Diagnostic{
 Pos:     rhs.Pos(),
 End:     rhs.End(),
 Message: "assigning a string literal",
})

Putting this together, you should be left with a functioning analyzer:

// main.go
package main

import (
 "go/ast"
 "go/token"

 "github.com/sanity-io/litter"
 "golang.org/x/tools/go/analysis"
 "golang.org/x/tools/go/analysis/passes/inspect"
 "golang.org/x/tools/go/analysis/singlechecker"
 "golang.org/x/tools/go/ast/inspector"
)

var Analyzer = &analysis.Analyzer{
 Name: "noString",
 Doc:  "Reports any string assignments",
 Requires: []*analysis.Analyzer{
  inspect.Analyzer,
 },
 Run:       run,
}

func run(pass *analysis.Pass) (any, error) {
 inspect := pass.ResultOf[inspect.Analyzer].(*inspector.Inspector)

 filter := []ast.Node{
  (*ast.AssignStmt)(nil),
 }
 inspect.Preorder(filter, func(n ast.Node) {
  assignStmt := n.(*ast.AssignStmt)

  // Only want one expression on the right-hand side
  if len(assignStmt.Rhs) != 1 {
   return
  }
  rhs, ok := assignStmt.Rhs[0].(*ast.BasicLit)
  if !ok {
   // Not a literal, can't be a string literal
   return
  }
  if rhs.Kind != token.STRING {
   return
  }

  pass.Report(analysis.Diagnostic{
   Pos:     rhs.Pos(),
   End:     rhs.End(),
   Message: "assigning a string literal",
  })
 })

 return nil, nil
}

To run it, you can build this into a binary, or invoke it with go run . <package path here>.

Testing our Analysis

To test what we wrote, we can write…a test! Once again, the analysis package provides for our needs with the sub-package analysistest:

package main

import (
 "path/filepath"
 "testing"

 "golang.org/x/tools/go/analysis/analysistest"
)

func TestLinter(t *testing.T) {
 testdat, err := filepath.Abs("testdata")
 if err != nil {
  t.Fatal(err)
 }

 analysistest.Run(t, testdat, Analyzer)
}

This boilerplate tells the analysis to run on a given file path of your choice. In the directory testdata we have code that should set off the newly-minted linter:

package testdata

import "fmt"

func f() {
 something := "something"
 fmt.Println(something)
}

➜ go test ./...
--- FAIL: TestLinter (0.44s)
    analysistest.go:630: /Users/jamesholdren/workspace/jw/blog/testdata/testdata.go:6:15: unexpected diagnostic: assigning a string literal
FAIL
FAIL    github.com/jw/blog    0.703s
FAIL

Here the linter produced some output, noting that there was a diagnostic for a string assignment. Great! Except the test is failing because the test didn’t know that it was desired that a diagnostic was reported. The fix is to add a comment of the format: want: "diagnostic messsage here" at the location of where the diagnostic should be reported (this will look familiar if you've used Go's examples before). After denoting in the test code that a diagnostic should happen, try running the test again:

package testdata

import "fmt"

func f() {
 something := "something" // want "assigning a string literal"
 fmt.Println(something)
}

go test ./...
ok      github.com/jw/blog    0.723s

Up next on static analysis…

Ok! Now that we’ve ended up with a working example of a linter, they’re hopefully a bit more approachable. The general strategy as we build analyses is to go wide, hunting for different expressions in the code, then narrow down to a specific goal, report the finding, and iterate. We haven’t yet gotten to the original bug and linter that we’re after, but now we have the basics down in order to build something more complex.

Next time, we’ll walk through that linter, how it was made, delve deeper into the analysis package for Facts, and how it gets integrated into our CI checks.

Want to write code that helps small businesses work fearlessly? We’re hiring across our Technology teams, build with us! Check out our Careers page.