Monday, October 8, 2018

Detailed walkthrough for a beginner Haskell program

post

This post walks through the development of a small Haskell program for aligning equals symbols in a block of text. This walkthrough targets a beginning programmer by describing several steps and concepts in extra detail.

Note that this post describes how to author, compile, and run single-file Haskell programs for ease of experimentation and learning. For larger Haskell projects you will want to use cabal or stack to scaffold and run the project and to share your work with others. I mainly introduce things this way because this provides a lightweight way to get started and try the language.

Background

I obsess about visual cleanliness of the code I write because I optimize for ease of reading rather than ease of writing. One way I try to improve appearance is aligning equals signs. For example, the code might begin like this:

address = "192.168.0.44"
port = 22
hostname = "wind"

… and then I manually indent the equals signs to all align with each other:

address  = "192.168.0.44"
port     = 22
hostname = "wind"

My editor is vim, and you can install support for that using the Tabular plugin. However, I thought this would be an instructive example to implement from scratch to illustrate how to program in a functional style.

One neat feature of vim is that you can transform text inside your editor using any command line program. For example, I can select text using visual mode and then type:

:!some-command

… and vim will feed the selected text as standard input to the command-line program named some-command and replace the selected text with whatever the command emits to standard output.

This means that I only need to write a program that takes text to align on standard input and then emits the aligned text on standard output. I’ll call this program: align-equals.

Development environment

The command line is my “IDE”,so I will usually keep up to three terminal windows open:

  • One window edits text using vim
  • One window displays type errors using ghcid
  • One window tests the Haskell code I write in a REPL

I also use Nix, specifically a nix-shell to provision my development tools. I prefer Nix to provision development tools is because I don’t want to accumulate globally installed cruft on my system. Using Nix, I can provision any development tool or library transiently using nix-shell.

I will run all of the following examples inside of the following Nix shell (created for each new terminal):

That creates a transient shell that provides ghc, ghci, ghcid, and hoogle with the text and safe Haskell packages available. If I need to change the set of available Haskell packages I edit the command line and recreate the shell.

I turn on live type-checking by running:

… which will then automatically reload align-equals.hs every time the file changes and display any errors or warnings that the Haskell compiler has found.

In another terminal, I open the code I’m editing inside the ghci REPL so that I can interactively test the functions that I am writing:

Then in the third terminal, I actually edit the file:

The program

First, let’s describe what we plan to do using English prose, before we translate that to code:

We want to find the longest prefix preceding an = sign and then add spaces to the end of all other prefixes to match the length of the longest prefix.

Prefix length

In order to do that, we first need a function that computes the number of characters preceding an = sign for a given line. This function should have the following type:

You can read that type signature as saying: “prefixLength is a function whose input is a value of type Text (i.e. a line of input) and whose output is an Int (the number of characters preceding the first = sign)”. We can even add comments to that effect:

I import Data.Text because I prefer not to use the default String type provided by Haskell’s Prelude, which is inefficient. The text package provides a high-performance alternative type named Text along with many useful utilities.

The implementation of the prefixLength function directly follows the description:

As the name suggests, the prefixLength is the length of the prefix. The only non-trivial part is just knowing how to discover the Data.Text.breakOn function created for this purpose.

I usually browse the online documentation for Haskell packages by doing a Google search for “hackage ${package name}” (i.e. “hackage text”). That’s how I found the breakOn function here:

However, some people also prefer to use hoogle, which indexes and searches Haskell functions by name and by type. For example, if I’m looking for a function that splits a Text value into a prefix and suffix, I can run:

I can also verify that my function works as expected by testing my function in the long-running REPL I have open:

I have to enable theOverloadedStrings extension because I’m not using the default String type from the Haskell Prelude. This extension allows other packages to override how string literals behave to work with alternative implementations like Text.

One thing that’s neat about Haskell is how order-insensitive the language is. You can define things out of order and the compiler doesn’t mind. That’s why we can write our code as if it says: “prefixLength is the length of the prefix … and by the way you can compute the prefix and suffix by splitting the string using breakOn”.

This out-of-order coding style also jives well with lazy evaluation. Haskell is a “lazy” language, meaning that the program can evaluate things out-of-order, too, or even not at all if they are never used. For example, our prefixLength function never actually uses the suffix, so the program will never actually bother to compute or allocate that value.

The more you program in Haskell the less you think about your program as a sequence of statements and the more you think about your program as a graph of dependent computations.

Indenting a line

Now we need a function that pads a prefix with spaces up to a desired number of characters:

… or with comments:

This function is slightly longer, but still pretty straightforward:

Note that we can reorder all the lines after the where with no change to the program’s behavior. However, to keep things readable we order them so that the reader can understand them from top to bottom:

  • Split the line into a prefix and suffix
  • Compute the actual length of the prefix
  • Compute the number of spaces to pad by subtracting the actual length from the desired length
  • Create the padding by repeating a space the specified number of times
  • Create a new line by inserting the padding in between the prefix and suffix

Note that this reads a lot like a function defined in an imperative language. For example, the analogous Python code is not much different:

In general, you can port functional programs to imperative programs in this way if the functional program sticks to simple types (i.e. strings, numbers, records, etc.). For these simple programs, the functional program is essentially an imperative program where you forbid reassigning a value (i.e. “mutation”), which is generally good practice anyway for ease of program comprehension.

We can confirm that our function works by saving the complete program so far:

… and then :reloading that program in our REPL:

Indenting multiple lines

Now we can define a function to indent multiple lines:

This function takes advantage of two convenient utilities called lines and unlines.

Data.Text.lines splits a block of Text into a list of lines:

… and Data.Text.unlines combines a list of lines back into a block of Text:

You can use these two utilities to simplify line-oriented Text transformations in Haskell:

  • Split the block of Text into a list of lines
  • Process the list of lines into a new list of lines
  • Join the new list of lines back into a block of Text

The interesting part of our adjustText function is how we process the list of lines:

You can read the above code as saying:

  • Apply (“map”) the prefixLength function over each element of our list of lines to get a list of prefix lengths
  • Find the maximum length
  • If there is no maximum length, return an empty list of lines
  • If there is a maximum length, pad each line using that length

You might wonder: “Why would there not be a maximum length?” Well, consider the case where we begin with 0 lines of input: what is the maximum value of an empty list? The maximumMay function doesn’t throw an exception or return a bogus sentinel value that might be confused for real data. Instead, the maximumMay function returns an optional result:

The a in the type of maximumMay can be any type that is comparable ( i.e. implements Ord), and in the context of our code that type is Int, so we can pretend that the maximumMay function actually has this more specific type:

In other words, given a list of Ints, the maximumMay function will Maybe return an Int (i.e. it might return an Int or it might not). This means that the result will either be a Nothing (i.e. no result) or an Int value wrapped inside of a Just.

We consume the result of maximumMay by “pattern matching” on the result, like this:

The first branch covers the case where the list is empty. Here, the desiredPrefixLength is not in scope, so if we try to use that value we will get a type error. This provides a nice safeguard so that we can’t attempt to access a result that does not exist. In other languages this might be caught at runtime as a java.lang.NullPointerException or AttributeError: 'NoneType' object has no attribute 'x', but with pattern matching the type-checker can detect these sorts of bugs before we even attempt to run the program.

The second branch covers the case where the list is non-empty and does have a sensible maximum length. There we use that length to adjust each line.

The nice thing about pattern matching is that you can’t forget to handle these cases. If you try to use the result of maximumMay directly as an Int then you will get a type error. The maximumMay function wraps its own result in a Maybe to force downstream users to carefully consider how to handle the case where the list may be empty.

Tying it all together

So far all of our functions are “pure” functions, meaning that they are deterministic conversions from their inputs to their outputs that don’t mutate any variables and don’t have any side effects that we care about.

Carefully note the key phrase: “side effects that we care about”. These functions do technically have side effects, including:

  • Allocating memory/registers
  • Taking a non-zero amount of time to compute

Sometimes we do care about these types of side effects in certain contexts, like cryptography (where secure information can leak through these side channels) or embedded programming (where programs require careful attention to time/memory). However, for our simple utility these functions are effectively “pure”.

We can’t use our utility functions from the command line, though, unless we wrap them in a main function which a program can run, like this:

The interact function converts any pure Text transformation into a runnable program that transforms standard input into standard output using the supplied transformation:

This is an example of a “higher-order” function: a function whose input is another function. The input to the interact function is another function of type Text -> Text. Conveniently, our adjustText function has exactly the correct type to be supplied as an argument to the interact function:

… and then any value of type IO () can be assigned to main, which is what our program will run when invoked as a command-line executable.

If we save the following complete example to align-equals.hs:

… then we can compile that using:

… and verify that the executable works using:

That means that I can now use ./align-equals to transform my text buffer by selecting a text block like this:

address = "192.168.0.44"
port = 22
hostname = "wind"

… and running :!./align-equals to align the block:

address  = "192.168.0.44"
port     = 22
hostname = "wind"

… and now I don’t have to tediously align the code manually.

Conclusion

Hopefully this illustrates one way to learn the Haskell language in the course of building small but practical programs. The language introduces many cool features and concepts and this post only touches the tip of the iceberg.

Thursday, August 16, 2018

NixOS in production

nixos.prod

This is a short post summarizing what I wish I had known when I first started using NixOS in production. Hopefully other people will find this helpful to avoid the pitfalls that I ran into.

The main issue with NixOS is that the manual recommends a workflow that is not suitable for deployment to production. Specifically, the manual encourages you to:

  • keep Nix source code on the destination machine
    • i.e. /etc/nixos/{hardware-,}configuration.nix
  • build on the destination machine
  • use Nix’s channel system to obtain nixpkgs code

This post will describe how you can instead:

  • build a source-free binary closure on a build machine
  • transfer and deploy that binary closure to a separate destination machine

This guide overlaps substantially with what the NixOps tool does and you can think of this guide as a way to transplant a limited subset of what NixOps does to work with other provisioning tools (such as Terraform).

You might also find this guide useful even when using NixOS as a desktop operating system for handling more exotic scenarios not covered by the nixos-rebuild command-line tool.

Building the closure

We’ll build up to the final solution by slowly changing the workflow recommended in the NixOS manual.

Suppose that you already have an /etc/nixos/configuration.nix file and you use nixos-rebuild switch to deploy your system. You can wean yourself off of nixos-rebuild by building the binary closure for the system yourself. In other words, you can reimplement the nixos-rebuild build command from scratch.

To do so, create the following file anywhere on your system:

Then run:

Congratulations, you’ve just built a binary closure for your system!

Deploying the closure

nix-build deposits a result symlink in the current directory pointing to the built binary closure. To deploy the current system to match the built closure, run:

Congratulations, you’ve just done the equivalent of nixos-rebuild switch!

As the above command suggests, the closure contains a ./bin/switch-to-configuration which understands a subset of the commands that the nixos-rebuild command does. In particular, the switch-to-configuration script accepts these commands:

$ ./result/bin/switch-to-configuration --help
Usage: ../../result/bin/switch-to-configuration [switch|boot|test]

switch:       make the configuration the boot default and activate now
boot:         make the configuration the boot default
test:         activate the configuration, but don't make it the boot default
dry-activate: show what would be done if this configuration were activated

Adding a profile

The nixos-rebuild command actually does one more thing in addition to buiding the binary closure and deploying the system. The nixos-rebuild command also creates a symlink pointing to the current system configuration so that you can roll back to that configuration later. The symlink also acts like a garbage collection root, preventing the system from being garbage collected until you remove the symlink (either directly using rm or a higher-level utility such as nix-collect-garbage)

You can record the system configuration in the same way as nixos-rebuild using the nix-env command:

Querying system options

You can use the same nixos.nix file to query what options you’ve set for your system, just like the nixos-option utility. For example, if you want to compute the final value of the networking.firewall.allowedTCPPorts option then you run this command:

Pinning nixpkgs

Now that you’ve taken control of the build you can do fancier things like pin nixpkgs to a specific revision of nixpkgs so that you don’t need to use a channels or the NIX_PATH any longer:

In fact, this makes your build completely insensitive to the NIX_PATH, eliminating a potential source of non-determinism from the build.

Building remotely

Now that you’ve removed nixos-rebuild from the equation you can build the binary closure on a separate machine from the one that you deploy to. You can check your nixos.nix, configuration.nix and hardware-configuration.nix files into version control and nix-build the system on any machine that can check out your version controlled Nix configuration. All you have to do is change the import path to be a relative path to the configuration.nix file within the same repository instead of an absolute path:

Then all your build machine has to do is:

$ git clone "git@github.com:${OWNER}/${REPOSITORY}.git"
$ nix-build --attr system "./${REPOSITORY}/path/to/nixos.nix"

To deploy the built binary closure to another machine, use nix copy. If you have SSH access to the destination machine then you can nix copy directly to that machine:

If you do not have SSH access to the destination machine you can instead use nix-copy to create a binary archive file containing the binary closure of your system:

… and upload the binary archive located at /tmp/system to the destination machine using your upload method of choice. Then import the binary archive into the /nix/store on the destination machine using nix copy:

Once the binary closure is on the machine, you install the closure the same way as before:

… replacing /nix/store/... with the /nix/store path of your closure (since there is no result symlink on the destination machine).

Conclusion

That’s it! Now you should be able to store your NixOS configuration in version control, build a binary closure as part of continuous integration, and deploy that binary closure to a separate destination machine. You can also now pin your build to a specific revision of Nixpkgs so that your build is more deterministic.

I wanted to credit my teammate Parnell Springmeyer who taught me the ./result/bin/switch-to-configuration trick for deploying a NixOS system and who codified the trick into the nix-deploy command-line tool. I also wanted to credit Remy Goldschmidt who interned on our team over the previous summer and taught me how to reliably pin Nixpkgs.

Monday, May 21, 2018

How I evaluate Haskell packages

This post summarizes the rough heuristics that I use for evaluating Haskell packages. Usually I use these rules of thumb when:

  • choosing between multiple similar packages
  • deciding whether to depend on a package at all

Some of these guidelines work for other programming languages, too, but some of them are unique to Haskell. Even if you are a veteran programmer in another language you still might find some useful tidbits here when approaching the Haskell ecosystem for the first time.

I've organized the metrics I use into five categories:

  • "Large positive" - Things that impress me
  • "Small positive" - Positive features that make me more likely to adopt
  • "Irrelevant" - Things I never pay attention to when judging package quality
  • "Small negative" - Things that turn me off
  • "Large negative" - Huge red flags

Large positive

  • High quality documentation

    Excellent documentation is one of the strongest signals of maintainer commitment to a package. Extensive documentation is very expensive to keep up to date with a package because documentation is difficult to machine check in general.

    You can definitely have the build check that type signatures or doctests are correct, but anything more elaborate usually requires human intervention to keep up to date.

    Disclaimer: I'm biased here because I'm a huge proponent of documenting things well.

  • Impossible to use incorrectly

    One of the nice features of Haskell is that you can use the type system to make impossible states unrepresentable or enforce invariants to guarantee proper use. I strongly prefer libraries that take advantage of this to rule out errors at compile time so that I can sleep soundly at night knowing that my work built on top of a solid foundation.

  • On Stackage

    All packages on Stackage are guaranteed to build and have a known good build plan (called a Stackage "resolver") that both stack and cabal can reuse. All packages on Stackage "play nice" with each other, which greatly reduces the total cost of ownership for depending on these Haskell packages.

    Being on Stackage is also a good signal that the package is maintained by somebody because packages require some upkeep to ensure that they continue to build against the latest version of other packages.

  • Competitive Benchmarks

    I strongly prefer packages that post benchmarks comparing their package to their competitors. Such a comparison implies that the author went to the trouble of using and understanding alternative packages and using them idiomatically. That perspective often improves the quality and design of their own package.

    Also, competitive benchmarks are rare and require significant upkeep, so they double as strong signal of maintainer commitment.

  • 100+ reverse dependencies

    You can check the packages that depend on a given package (a.k.a. "reverse dependencies") using the following URL:

    https://packdeps.haskellers.com/reverse/${package-name}

    Packages that have a 100 or more reverse dependencies have substantial buy-in from the Haskell ecosystem and are safe to depend on. You're also likely to depend on these packages anyway through your transitive dependencies.

    (Suggested by: Tim Humphries)

Small positive

  • One complete example

    This means that the package has one "end-to-end" example showing how all the pieces fit together. I prefer packages that do this because they don't force me to waste a large amount of time reinventing what the author could have communicated in a much shorter amount of time.

    You would be surprised how many Haskell packages have extensive Haddock coverage but still lack that one overarching example showing how to use the package.

  • 100+ downloads in last 30 days

    Download statistics are not a very strong signal of activity on Hackage since most Haskell build tools (i.e. cabal, stack, Nix) will cache dependencies. These tools also don't require frequent cache invalidations like some other build tools which shall not be named.

    However, having 100+ downloads in the last 30 days is a reasonable signal that enough people were recently able to get the package working. That means that if anything goes wrong then I assume that there must be some solution or workaround.

  • Maintained/used/endorsed by a company

    I think this is a pretty obvious signal. Similar to download metrics, this implies that somebody was able to extract something commercially valuable from this package.

  • Pure or mostly pure API

    Packages that provide pure APIs are easier to test and easier to integrate into a code base. This reduces the total cost of ownership for that dependency.

  • Upper bounds on dependencies that are up to date

    I'm not going to take a strong stance on the "great PVP war" in this post. All I will note is that if a package has upper bounds that are up to date then that is a good sign that the package is actively maintained (whether or not you agree that upper bounds are a good idea).

    In other words, upper bounds are one way that a package author can "virtue signal" that they have the time and resources to maintain a package.

Irrelevant

  • On Hackage

    Many people new to Haskell ecosystem assume that there is some sort of quality bar to be on Hackage just because Hackage is not GitHub. There is no such quality bar; anybody can obtain a Hackage account and upload their packages (even joke packages).

  • Stability field

    The stability field of packages fell out of fashion years ago. Also, in my experience people are poor judges of whether or not their own packages will be stable. I prefer to look at the frequency of releases and how often they are breaking changes as empirical evidence of a package's stability.

  • Version number

    First off, make sure you understand PVP. The key bit is that the first two components of a version number represent the major version of a package. That means that if the package goes from version 1.0 to 1.1 then that signals just as much of a breaking change to the API as if the package went from version 1.0 to 2.0. The usual rule of thumb for interpreting version numbers is:

    X.Y.Z.A
    ↑ ↑ ↑ ↑
    | | | |
    | | | Non-breaking change that is invisible (i.e. documentation or refactor)
    | | | Note: Not all packages use a fourth number in their versions
    | | |
    | | Non-breaking change
    | |
    | Breaking change without fanfare
    |
    Breaking change with great fanfare

    The absolute version number basically does not matter. There are many widely used and mature packages on Hackage that are still version 0.*. This can be jarring coming from another language ecosystem where people expect a mature package to be at least version 1.0.

  • Votes

    Hackage 2 supports voting on packages but I've never consulted a package's votes when deciding whether or not to adopt. I prefer quality signals that cannot be easily faked or gamed.

  • Tests

    I've never checked whether or not I'm using a well-tested package or not. Maybe that's not a good idea, but I'm being honest about how I evaluate packages.

    This probably comes as a surprise to a person coming to Haskell from another language ecosystem, especially a dynamically typed language that rely heavily on tests to ensure quality. Haskell programmers lean much more on the type system to enforce and audit quality. For example, type signatures show me at a glance if a package is overusing side effects or deals with too many states or corner cases that I need to account for.

Small negative

  • Upper bounds that are not up to date

    The other nice thing about adding upper bounds to a package is that they also accurately signal when a package is no longer maintained. They require constant "gardening" to keep up to date so when an author abandons a package they naturally transform from a positive signal into a negative signal simply by virtue of inaction.

  • Large dependency footprint

    This used to be a much bigger issue before the advent of Stackage. People would aggressively trim their dependency trees out of fear that one bad dependency would cause them to waste hours finding a valid build plan.

    I still spend some time these days trying to minimize my dependency footprint, mainly so that I don't need to respond as often to breaking changes in my immediate dependencies. However, this is a relative minor concern in comparison to how bad things used to be.

  • Uses linked lists inappropriately

    The #1 source of Haskell performance degradation is keeping too many objects on the heap and linked lists trash your heap if you materialize the entire list into memory (as opposed to streaming over the list so that you only materialize at most one element at any time).

    I try to avoid dependencies that use linked lists unless they are the optimal data structure because I will be paying the price for those dependencies every time my program runs a garbage collection.

  • Frequent breaking API changes

    Breaking API changes are not a really big deal in Haskell due to the strong type system. Breaking API changes are mainly a minor maintenance burden: if you depend on a library that releases breaking changes all the time you usually only need to spend a few minutes updating your package to build against the latest API.

    This matters more if you maintain a large number of packages or if you have a huge number of dependencies, but if you don't then this is a relatively small concern.

  • Idiosyncratic behavior

    "Idiosyncratic" means that the package author does something weird unique to them. Some examples include using a private utility library or custom Prelude that nobody else uses or naming all of their classes C and all their types T (you know who you are).

    When I see this I assume that the package author has no interest in building consensus or being a "well-behaved" member of the Haskell ecosystem. That sort of package is less likely to respond or adapt to user feedback.

Large negative

  • Uses String inappropriately

    This is a special case of "Uses linked lists inappropriately" since the default String type in Haskell is stored as a linked list of characters.

    Strings tend to be rather large linked lists of characters that are commonly fully materialized. This implies that even modest use of them will add lots of objects to the heap and tax the garbage collector.

  • No documentation

    If I see no documentation on a package I "nope" right out of there. I assume that if a maintainer does not have time to document the package then they do not have time to address user feedback.

  • Not on Hackage

    I assume that if a package is not on Hackage that the maintainer has no interest in supporting or maintaining the package. The bar for adding a package to Hackage is so low that the absence of a package on Hackage is a red flag.

    Note that this does not imply that packages on Hackage will be supported. There are packages on Hackage where the author does not intend to support the package.

    Note: a reader pointed out that this is confusing given that I treat "on Hackage" as "Irrelevant". Perhaps another way to phrase this is that being on Hackage is a necessary but not sufficient signal of maintainer support.

  • Maintainer ignoring pull requests

    This is obviously the clearest sign that a maintainer does not respond to user feedback. There are fewer things that upset me more than a maintainer that completely ignores volunteers who submit pull requests to fix bugs.

Conclusion

Hopefully that gives you some quick and dirty heuristics you can use when evaluating Haskell packages for the first time. If you think that I missed something important just let me know and I'll update this post.

Monday, February 5, 2018

The wizard monoid

Recent versions of GHC 8.0 provides a Monoid instance for IO and this post gives a motivating example for why this instance is useful by building combinable "wizard"s.

Wizards

I'll define a "wizard" as a program that prompts a user "up front" for multiple inputs and then performs several actions after all input has been collected.

Here is an example of a simple wizard:

main :: IO ()
main = do
    -- First, we request all inputs:
    putStrLn "What is your name?"
    name <- getLine

    putStrLn "What is your age?"
    age <- getLine

    -- Then, we perform all actions:
    putStrLn ("Your name is: " ++ name)
    putStrLn ("Your age is: " ++ age)

... which produces the following interaction:

What is your name?
Gabriel<Enter>
What is your age?
31<Enter>
Your name is: Gabriel
Your age is: 31

... and here is an example of a slightly more complex wizard:

import qualified System.Directory

main :: IO ()
main = do
    -- First, we request all inputs:
    files <- System.Directory.listDirectory "."
    let askFile file = do
            putStrLn ("Would you like to delete " ++ file ++ "?")
            response <- getLine
            case response of
                "y" -> return [file]
                _   -> return []

    listOfListOfFilesToRemove <- mapM askFile files
    let listOfFilesToRemove = concat listOfListOfFilesToRemove

    -- Then, we perform all actions:
    let removeFile file = do
            putStrLn ("Removing " ++ file)
            System.Directory.removeFile file
    mapM_ removeFile listOfFilesToRemove

... which produces the following interaction:

Would you like to delete file1.txt?
y<Enter>
Would you like to delete file2.txt?
n<Enter>
Would you like to delete file3.txt?
y<Enter>
Removing file1.txt
Removing file3.txt

In each example, we want to avoid performing any irreversible action before the user has completed entering all requested input.

Modularity

Let's revisit our first example:

main :: IO ()
main = do
    -- First, we request all inputs:
    putStrLn "What is your name?"
    name <- getLine

    putStrLn "What is your age?"
    age <- getLine

    -- Then, we perform all actions:
    putStrLn ("Your name is: " ++ name)
    putStrLn ("Your age is: " ++ age)

This example is really combining two separate wizards:

  • The first wizard requests and displays the user's name
  • The second wizard requests and displays the user's age

However, we had to interleave the logic for these two wizards because we needed to request all inputs before performing any action.

What if there were a way to define these two wizards separately and then combine them into a larger wizard? We can do so by taking advantage of the Monoid instance for IO, like this:

import Data.Monoid ((<>))

name :: IO (IO ())
name = do
    putStrLn "What is your name?"
    x <- getLine
    return (putStrLn ("Your name is: " ++ x))

age :: IO (IO ())
age = do
    putStrLn "What is your age?"
    x <- getLine
    return (putStrLn ("Your age is: " ++ x))

runWizard :: IO (IO a) -> IO a
runWizard request = do
    respond <- request
    respond

main :: IO ()
main = runWizard (name <> age)

This program produces the exact same behavior as before, but now all the logic for dealing with the user's name is totally separate from the logic for dealing with the user's age.

The way this works is that we split each wizard into two parts:

  • the "request" (i.e. prompting the user for input)
  • the "response" (i.e. performing an action based on that input)

... and we do so at the type-level by giving each wizard the type IO (IO ()):

name :: IO (IO ())

age :: IO (IO ())

The outer IO action is the "request". When the request is done the outer IO action returns an inner IO action which is the "response". For example:

--      ↓ The request
name :: IO (IO ())
--          ↑ The response
name = do
    putStrLn "What is your name?"
    x <- getLine
    -- ↑ Everything above is part of the outer `IO` action (i.e. the "request")

    --      ↓ This return value is the inner `IO` action (i.e. the "response")
    return (putStrLn ("Your name is: " ++ x))

We combine wizards using the (<>) operator, which has the following behavior when specialized to IO actions:

ioLeft <> ioRight

= do resultLeft  <- ioLeft
     resultRight <- ioRight
     return (resultLeft <> resultRight)

In other words, if you combine two IO actions you just run each IO action and then combine their results. This in turn implies that if we nest two IO actions then we repeat this process twice:

requestLeft <> requestRight

= do respondLeft  <- requestLeft
     respondRight <- requestRight
     return (respondLeft <> respondRight)

= do respondLeft  <- requestLeft
     respondRight <- requestRight
     return (do
         unitLeft  <- respondLeft
         unitRight <- respondRight
         return (unitLeft <> unitRight) )

-- Both `unitLeft` and `unitRight` are `()` and `() <> () = ()`, so we can
-- simplify this further to:
= do respondLeft  <- requestLeft
     respondRight <- requestRight
     return (do
         respondLeft
         respondRight )

In other words, when we combine two wizards we combine their requests and then combine their responses.

This works for more than two wizards. For example:

request0 <> request1 <> request2 <> request3

= do respond0 <- request0
     respond1 <- request1
     respond2 <- request2
     respond3 <- request3
     return (do
         respond0
         respond1
         respond2
         respond3 )

To show this in action, let's revisit our original example once again:

import Data.Monoid ((<>))

name :: IO (IO ())
name = do
    putStrLn "What is your name?"
    x <- getLine
    return (putStrLn ("Your name is: " ++ x))

age :: IO (IO ())
age = do
    putStrLn "What is your age?"
    x <- getLine
    return (putStrLn ("Your age is: " ++ x))

runWizard :: IO (IO a) -> IO a
runWizard request = do
    respond <- request
    respond

main :: IO ()
main = runWizard (name <> age)

... and this time note that name and age are awfully similar, so we can factor them out into a shared function:

import Data.Monoid ((<>))

prompt :: String -> IO (IO ())
prompt attribute = do
    putStrLn ("What is your " ++ attribute ++ "?")
    x <- getLine
    return (putStrLn ("Your " ++ attribute ++ " is: " ++ x))

runWizard :: IO (IO a) -> IO a
runWizard request = do
    respond <- request
    respond

main :: IO ()
main = runWizard (prompt "name" <> prompt "age")

We were not able to factor out this shared logic back when the logic for the two wizards were manually interleaved. Once we split them into separate logical wizards then we can begin to exploit shared structure to compress our program.

This program compression lets us easily add new wizards:

import Data.Monoid ((<>))

prompt :: String -> IO (IO ())
prompt attribute = do
    putStrLn ("What is your " ++ attribute ++ "?")
    x <- getLine
    return (putStrLn ("Your " ++ attribute ++ " is: " ++ x))

runWizard :: IO (IO a) -> IO a
runWizard request = do
    respond <- request
    respond

main :: IO ()
main = runWizard (prompt "name" <> prompt "age" <> prompt "favorite color")

... and take advantage of standard library functions that work on Monoids, like foldMap so that we can mass-produce wizards:

import Data.Monoid ((<>))

prompt :: String -> IO (IO ())
prompt attribute = do
    putStrLn ("What is your " ++ attribute ++ "?")
    x <- getLine
    return (putStrLn ("Your " ++ attribute ++ " is: " ++ x))

runWizard :: IO (IO a) -> IO a
runWizard request = do
    respond <- request
    respond

main :: IO ()
main = runWizard (foldMap prompt [ "name", "age", "favorite color", "sign" ])

More importantly, we can now easily see at a glance what our program does and ease of reading is a greater virtue than ease of writing.

Final example

Now let's revisit the file removal example through the same lens:

import qualified System.Directory

main :: IO ()
main = do
    -- First, we request all inputs:
    files <- System.Directory.listDirectory "."
    let askFile file = do
            putStrLn ("Would you like to delete " ++ file ++ "?")
            response <- getLine
            case response of
                "y" -> return [file]
                _   -> return []

    listOfListOfFilesToRemove <- mapM askFile files
    let listOfFilesToRemove = concat listOfListOfFilesToRemove

    -- Then, we perform all actions:
    let removeFile file = do
            putStrLn ("Removing " ++ file)
            System.Directory.removeFile file
    mapM_ removeFile listOfFilesToRemove

We can simplify this using the same pattern:

import qualified System.Directory

main :: IO ()
main = do
    files <- System.Directory.listDirectory "."
    runWizard (foldMap prompt files)

prompt :: FilePath -> IO (IO ())
prompt file = do
    putStrLn ("Would you like to delete " ++ file ++ "?")
    response <- getLine
    case response of
        "y" -> return (do
            putStrLn ("Removing " ++ file)
            System.Directory.removeFile file )
        _   -> return (return ())

runWizard :: IO (IO a) -> IO a
runWizard request = do
    respond <- request
    respond

All we have to do is define a wizard for processing a single file, mass-produce the wizard using foldMap and the Monoid instance for IO takes care of bundling all the requests up front and threading the selected files to be removed afterwards.

Conclusion

This pattern does not subsume all possible wizards that users might want to write. For example, if the wizards depend on one another then this pattern breaks down pretty quickly. However, hopefully this provides an example of you can chain the Monoid instance for IO with other Monoid instance (even itself!) to generate emergent behavior.