File, Operating System, and Related Packages

The standard library provides many packages to support file and directory handling and interaction with the operating system. In many cases, these packages provide operating-system-neutral abstractions that make it simple to create cross-platform Go applications.

The os (operating system) package provides functions for operating-system interactions, such as changing the current working directory, changing file mode and ownership, getting and setting environment variables, and creating and removing files and directories.

In addition, this package provides functions for creating and opening files (os.Create() and os.Open()), and for retrieving file attributes (for example, via the os.FileInfo type), some of which we have seen in earlier installments of this series.

Once a file is opened, especially in the case of text files, it is very common to want to access it via a buffer (to read lines as strings rather than as byte slices). The functionality we need is provided by the bufio package. In addition to using bufio.Readers and bufio.Writers for reading and writing strings, we can also read (and unread) runes, read (and unread) single bytes, read multiple bytes, as well as write runes and single or multiple bytes.

The io (input/output) package provides a large number of functions for working with io.Readers and io.Writers. (Both of these interfaces are satisfied by *os.File values.) For example, we have used the io.Copy() function to copy data from a reader to a writer. This package also contains functions for creating synchronous in-memory pipes.

The io/ioutil package provides a few high-level convenience functions. Among others, the package provides the ioutil.ReadAll() function that reads all of an io.Reader's data and returns it as a []byte; the ioutil.ReadFile() function, which does the same but accepts a string argument (the filename) rather than an io.Reader; the ioutil.TempFile() function, which returns a temporary file (an *os.File); and the ioutil.WriteFile() function, which writes a []byte to a file whose name it is given.

The path package has functions for manipulating UNIX-style paths such as Linux and Mac OS X paths, URL paths, git "references," FTP files, and so on. The path/filepath package provides the same functions as path — and many others — and is designed to provide platform-neutral path handling. This package also provides the filepath.Walk() function for recursively iterating over all the files and directories in a given path.

The runtime package contains many functions and types that give access to Go's runtime system. Most of these are advanced and should not be needed when creating standard maintainable Go programs. However, a couple of the package's constants can be useful — for example, runtime.GOOS, which holds a string (for example, "darwin," "freebsd," "linux," or "windows"), and runtime.GOARCH, which also holds a string (for example, "386," "amd64," or "arm"). The runtime.GOROOT() function returns the GOROOT environment variable's value (or the Go build's root if the environment variable isn't set), and the runtime.Version() function returns the Go version (as a string). The runtime.GOMAXPROCS() and runtime.NumCPU() functions ensure that Go uses all the machine's processors — and are explained in the Go documentation.

File Format-Related Packages

Go's excellent support for file handling applies both to text files (using the 7-bit ASCII encoding or the UTF-8 and UTF-16 Unicode encodings), and to binary files. Go provides specific packages for handling JSON and XML files, as well as its own very fast, compact, and convenient Go binary format. In addition, Go has a csv package for reading CSV (comma-separated values) files. This package treats such files as records (one per line), each of which consists of (comma-separated) fields. The package is quite versatile — for example, it is possible to change the delimiter (from a comma to a tab or other character), as well as other aspects of how it reads and writes records and fields.

The encoding package contains several packages, one of which, encoding/binary, we have already used for reading and writing binary data. The other packages provide encoding and decoding for various other formats — for example, the encoding/base64 package can be used to encode and decode URLs that often use this format.

Graphics-Related Packages

Go's image package provides some high-level functions and types for creating and holding image data. It also has a number of packages that provide encoders and decoders for various standard graphics file formats, such as image/jpeg and image/png.

The image/draw package provides some basic drawing. The third-party freetype package adds more functions for drawing. The freetype package, itself, can draw text using any specified TrueType font, and the freetype/raster package can draw lines and cubic and quadratic curves.

Mathematics Packages

The math/big package provides unlimited (except by memory) size integers (big.Int), and rationals (big.Rat).The math package provides all the standard mathematical functions (based on float64s) and several standard constants. The math/cmplx package provides some standard functions for complex numbers (based on complex128s).

Miscellaneous Packages

In addition to the packages that can be roughly grouped together, the standard library contains a number of packages that stand more or less alone.

The crypto package can provide hashes using the MD5, SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512 algorithms. (Support for each algorithm is supplied by a package, for example, crypto/sha512.) In addition, the crypto package has packages that provide encryption and decryption using a variety of algorithms, such as AES, DES, and so on, each in packages with corresponding names (for example, crypto/aes and crypto/des).

The exec package is used to run external programs. This can also be done using the os.StartProcess() function, but the exec.Cmd type is much more convenient to use.

The flag package provides a command-line parser. It accepts X11-style options (for example, -width, not GNU-style -w, and --width). The package produces a very basic usage message and does not provide any validation beyond a value's type. (So, the package can be used to specify an int option, but not what values are acceptable.) Several alternatives are available from the Go Dashboard.

The log package provides functions for logging information (by default to os.Stdout), and for terminating the program or panicking with a log message. The log package's output destination can be changed to any io.Writer using the log.SetOutput() function. Log messages are output in the form of a timestamp and then the message; the timestamp can be eliminated by calling log.SetFlags(0) before the first log function call. It is also possible to create custom loggers using the log.New() function.

The math/rand package provides many useful pseudo-random number generating functions including rand.Int(), which returns a random int, and rand.Intn(n), which returns a random int in the range [0, n]. The crypto/rand package has a function for producing cryptographically strong pseudo-random numbers. The regexp package provides a very fast and powerful regular expression engine that supports the RE2 engine's syntax.

The sort package provides convenience functions for sorting slices of ints, float64s, and strings, and for performing fast (binary chop) searches on such sorted slices. It also provides generic sort.Sort() and sort.Search() functions that can be used for custom data.

The time package has functions for measuring time and for parsing and formating date, date/time, and time values. The time.After() function can be used to send the current time on the channel it returns after a specified number of nanoseconds have passed. The time.Tick() and time.NewTicker() functions can be used to provide a channel to which a "tick" is sent repeatedly at a specified interval. The time.Time struct has methods for providing the current time, for formatting a date/time as a string, and for parsing date/times.

Networking Packages

The Go standard library has many packages that support networking and related programming. The net package provides functions and types for communicating using UNIX domain and network sockets, TCP/IP, and UDP.

The package also provides functions for domain name resolution. The net/http package makes use of the net package and has functionality for parsing HTTP requests and replies, and provides a basic HTTP client. The net/http package also includes an easy-to-extend HTTP server. The net/url package provides URL parsing and query escaping.

Some other high-level networking packages are included in the standard library. One is the net/rpc (Remote Procedure Call) package, which allows a server to provide objects whose exported methods can be called by clients. Another is the net/smtp (Simple Mail Transport Protocol) package, which can be used to send email.