{"id":2669,"date":"2019-10-31T10:06:49","date_gmt":"2019-10-31T18:06:49","guid":{"rendered":"https:\/\/www.pnfsoftware.com\/blog\/?p=2669"},"modified":"2019-10-31T10:14:42","modified_gmt":"2019-10-31T18:14:42","slug":"analyzing-golang-executables","status":"publish","type":"post","link":"https:\/\/www.pnfsoftware.com\/blog\/analyzing-golang-executables\/","title":{"rendered":"Analyzing Golang Executables"},"content":{"rendered":"\n

The Go programming language<\/a> (also known as Golang) has gained popularity during the last few years among malware developers<\/a> . This can certainly be explained by the relative simplicity of the language, and the cross-compilation ability of its compiler, allowing multi-platform malware development without too much effort.<\/p>\n\n\n\n

In this blog post, we dive into Golang executables reverse engineering, and present a Python extension for JEB decompiler to ease Golang analysis; here is the table of content:<\/p>\n\n\n\n

  1. Golang Basics for Reverse Engineers<\/a><\/li>
  2. Making JEB Great for Golang<\/a>
    1. Current Status<\/a><\/li>
    2. Finding (and Naming) Routines<\/a><\/li>
    3. Strings Recovery<\/a><\/li>
    4. Types Recovery<\/a><\/li><\/ol><\/li>
    5. Use-Case: Analysis of StealthWorker Malware<\/a><\/li><\/ol>\n\n\n\n

      The JEB Python script presented in this blog can be found on our GitHub page<\/a>. Make sure to update JEB to version 3.7+ before running it. <\/strong><\/p>\n\n\n\n

      Disclaimer: the analysis in this blog post refers to the current Golang version (1.13) and part of it might become outdated with future releases.<\/em><\/strong><\/p>\n\n\n\n

      Golang Basics for Reverse Engineers<\/h2>\n\n\n\n

      Feel free to skip this part<\/a> if you’re already familiar with Golang reverse engineering.<\/em><\/p>\n\n\n\n

      Let’s start with some facts that reverse engineers might find interesting to know before analyzing their first Golang executable.<\/p>\n\n\n\n

      1. Golang is an open-source<\/a><\/strong> language with a pretty active development community<\/strong>. The language was originally created at Google around 2007, and version 1.0 was released in March 2012<\/a>. Since then, two major versions are released each year<\/a>.<\/p>\n\n\n\n

      2. Golang has a long lineage<\/strong>: in particular many low-level implementation choices — some would say oddities — in Golang can be traced back to Plan9<\/a>, a distributed operating system on which some Golang creators were previously working. <\/p>\n\n\n\n

      3. Golang has been designed for concurrency<\/strong>, in particular by providing so-called “goroutines<\/a>“, which are lightweight<\/em> threads executing concurrently (but not necessarily in parallel<\/a>). <\/p>\n\n\n\n

      Developers can start a new goroutine simply by prefixing a function call by go<\/code>. A new goroutine will then start executing the function, while the caller goroutine returns and continues its execution concurrently with the callee. Let’s illustrate that with the following Golang program:<\/p>\n\n\n

      \nfunc myDummyFunc(){\n\ttime.Sleep(1 * time.Second)\n\tfmt.Println("dummyFunc executed")\n}\n\nfunc main(){\n\t\n\tmyDummyFunc() \/\/ normal call\n\tfmt.Println("1 - back in main")\n\t\n\tgo myDummyFunc() \/\/ !! goroutine call\n\tfmt.Println("2 - back in main")\n\n\ttime.Sleep(3 * time.Second)\n}\n<\/pre><\/div>\n\n\n
      Here, myDummyFunc()<\/code> is called once normally, and then as a goroutine. Compiling and executing this program results in the following output:<\/p>\n\n\n\n
      dummyFunc executed\n1 - back in main\n2 - back in main\ndummyFunc executed<\/code><\/pre>\n\n\n\n
      Notice how the execution was back in main()<\/code> before executing the second call to dummyFunc()<\/code>.<\/p>\n\n\n\n
      Implementation-wise, many<\/em> goroutines can be executed on a single<\/em> operating system thread. Golang runtime takes care of switching goroutines, e.g. whenever one executes a blocking system call. According to the official documentation<\/a> “It is practical to create hundreds of thousands of goroutines<\/strong> in the same address space<\/em>“.<\/p>\n\n\n\n
      What makes goroutines so “cheap” to create is that they start with a very limited stack space (2048 bytes — since Golang 1.4<\/a>), which will be increased when needed. <\/p>\n\n\n\n
      One of the noticeable consequence for reverse engineers is that native routines (almost) all start with the same prologue<\/strong>. Its purpose is to check if the current goroutine’s stack is large enough, as can be seen in the following CFG:<\/p>\n\n\n\n
      \"\"
      Fig. 1: Simplified x86 CFG with Golang prologue for stack growth<\/figcaption><\/figure>\n\n\n\n
      When the stack space is nearly exhausted, more space will be allocated — actually, the stack will be copied somewhere with enough free space. This particular prologue is present only in routines with local variables. <\/p>\n\n\n\n
      How to distinguish a goroutine call from a “normal” call when analyzing a binary?<\/strong> Goroutine calls are implemented by calling runtime.newproc<\/a><\/code>, which takes in input the address of the native routine to call, the size of its arguments, and then the actual routine’s arguments.<\/p>\n\n\n\n
      4. Golang has a concurrent<\/em> garbage collector<\/strong> (GC)<\/strong>: Golang’s GC can free memory while <\/em>other goroutines are modifying it.<\/p>\n\n\n\n
      Roughly speaking, when the GC is freeing memory, goroutines report to it all their memory writes — to prevent concurrent memory modifications to be missed by the current freeing phase. Implementation-wise, when the GC is in the process of marking used memory, all memory writes pass through a “<\/strong>write barrier<\/strong><\/a><\/em>“<\/strong>, which performs the write and informs the GC.<\/p>\n\n\n\n
      For reverse engineers this can result in particularly convoluted control flow graphs (CFG).<\/strong> For example, here is the CFG when a global variable globalString<\/code> is set to newValue<\/code>:<\/p>\n\n\n\n
      \"\"
       Fig. 2: Write to global variable globalString<\/code> (x86 CFG):
      before doing the memory write, the code checks if the write barrier is activated,
       and if yes calls runtime.gcWriteBarrier()<\/code><\/figcaption><\/figure>\n\n\n\n
      Not all memory writes are monitored in that manner; the rules for write barriers’ insertion are described in mbarrier.go<\/a><\/code>.<\/p>\n\n\n\n
      5. Golang comes with a custom compiler tool chain (parser, compiler, assembler, linker), all implemented in Golang<\/strong>. 1<\/a><\/sup> 2<\/a><\/sup><\/p>\n\n\n\n
      From a developer’s perspective, it means that once Go is installed on a machine, one can compiled for any<\/em> supported platform (making Golang a language of choice for IoT malware developers). Examples of supported platforms include Windows x64, Linux ARM and Linux MIPS (see “valid combinations of <\/a>$GOOS<\/a><\/code> and <\/a>$GOARCH<\/a><\/code>“).<\/p>\n\n\n\n
      From a reverse engineer’s perspective, the custom Go compiler toolchain means Golang binaries sometimes come with “exotic” features <\/strong>(which therefore can give a hard time to reverse engineering tools). <\/p>\n\n\n\n
      For example, symbols in Golang Windows executables are implemented using the COFF symbol table<\/a> (while officially<\/a> “COFF debugging information [for executable] is deprecated<\/em>“). The Golang COFF symbol implementation is pretty liberal: symbols’ type is set to a default value<\/a> — i.e. there is no clear distinction between code and data. <\/p>\n\n\n\n
      As another example,  Windows PE read-only data section “.rdata<\/code>” has been defined as executable<\/em> in past Go versions<\/a>. <\/p>\n\n\n\n
      \nInterestingly, Golang compiler internally uses pseudo<\/em> assembly instructions<\/a> (with architecture-specific registers). For example, here is a snippet of pseudo-code for ARM (operands are ordered with source first):

      \n\nMOVW  $go.string.\"hello, world\\n\"(SB), R0
      \nMOVW  R0, 4(R13)
      \nMOVW  $13, R0
      \nMOVW  R0, 8(R13)
      \nCALL  \"\".dummyFunc(SB)
      \nMOVW.P  16(R13), R15<\/code>\n

      \nThese pseudo-instructions could not be understood by a classic ARM assembler (e.g. there is no CALL<\/code> instruction on ARM). Here are the disassembled ARM instructions from the corresponding binary:

      \nLDR       R0, #451404h \/\/ \"hello, world\\n\" address 
      \nSTR       R0, [SP, #4]
      \nMOV       R0, #13
      \nSTR       R0, [SP, #8]
      \nBL        main.dummyFunc
      \nLDR       PC, [SP], #16
      <\/code>\n
      \nNotice how the same pseudo-instruction MOVW<\/code> got converted either as STR<\/code> or<\/b> MOV<\/code> machine instructions. The use of pseudo-assembly comes from Plan9, and allows Golang assembler parser to easily handle all architectures: the only architecture-specific step is the selection of machine instructions (more details       here<\/a>).\n<\/p>\n\n\n\n
      6. Golang uses by default a stack-only<\/em> calling convention<\/strong>. <\/p>\n\n\n\n
      Let’s illustrate that with the following diagram, showing the stack’s state when a routine with two integer parameters a<\/code> and b<\/code>, and two <\/em>return values — declared in Go as “func myRoutine(a int, b int) (int, int)<\/code>” — is called:<\/p>\n\n\n\n
      \"\"
      Fig. 3: Simplified stack view (stack grows downward), when a routine with two parameters and two return values is called . The return values are reserved slots for the callee.<\/figcaption><\/figure><\/div>\n\n\n\n
      It is the caller’s responsibilities to reserve space for the callees’ parameters and returned values, and to free it later on. <\/p>\n\n\n\n
      Note that Golang’s calling convention situation might soon change: since version 1.12, several calling conventions can coexist<\/a> — the stack-only calling convention remaining the default one for backward compatibility reasons.<\/p>\n\n\n\n
      7. Golang executables are usually<\/em> statically-linked, i.e. do not rely on external dependencies<\/strong> 3<\/a><\/sup>. In particular they embed a pretty large runtime environment. Consequently, Golang binaries tend to be large: for example, a “hello world” program compiled with Golang 1.13 is around 1.5MB with its symbols stripped.<\/p>\n\n\n\n
      8. Golang executables embed lots<\/em> of symbolic information<\/strong>:<\/p>\n\n\n\n