pipes-http-1.0: Streaming HTTP/HTTPS clients

The pipes-http package now provides an HTTP client for pipes. This was made possible by Michael Snoyman, who released http-client and http-client-tls, which are a conduit-independent subset of http-conduit. I wanted to thank Michael for releasing those packages, which greatly simplified my job. I even get TLS support for free thanks to him, which is incredible.

I've chosen to write the thinnest layer possible, only providing functions to stream in request bodies and stream out response bodies. Everything else is re-exported from http-client and http-client-tls. Hopefully this will encourage shared contributions to those libraries from both pipes and conduit users.

I'll show the library in action with an example of how you would query and format the titles of the top 10 posts on /r/haskell:

-- haskell-reddit.hs

{-# LANGUAGE OverloadedStrings #-}

import Control.Lens (_Right, (^..))
import Control.Lens.Aeson (key, values, _String)
import Control.Monad.Trans.State.Strict (evalStateT)
import Data.Aeson.Parser (json')
import qualified Data.Text    as T
import qualified Data.Text.IO as T
import Pipes.Attoparsec (parse)
import Pipes.HTTP

main = do
    req <- parseUrl "http://www.reddit.com/r/haskell.json"
    withManager defaultManagerSettings $ \m ->
        withHTTP req m $ \resp -> do
            json <- evalStateT (parse json') (responseBody resp)

            let titles :: [T.Text]
                titles = json  ^.. _Right
                                 . key "data"
                                 . key "children"
                                 . values
                                 . key "data"
                                 . key "title"
                                 . _String

            mapM_ (T.putStrLn . format) (take 10 titles)

format :: T.Text -> T.Text
format txt =
    if   T.length txt <= columns
    then T.concat [bullet,                txt          ]
    else T.concat [bullet, T.take columns txt, ellipsis]
  where
    bullet   = "[*] "
    ellipsis = "..."
    columns = 60 - (T.length bullet + T.length ellipsis)

This example uses pipes-attoparsec to parse the response body as JSON and lens-aeson to query the JSON value for titles:

$ ./haskell-reddit
[*] My Haskell will
[*] What are some common "design patterns" when it comes ...
[*] The Resumption and Reactive Resumption Monads
[*] `cabal install` failing in Vagrant
[*] A Lazy Evaluation - Why I Find Learning Haskell Hard
[*] The Essence Of Reynolds
[*] A test driven haskell course
[*] Wheb -- A WAI framework.
[*] I wrote a simple <200 line Forth interpreter. Does...
[*] GHC iOS 7.8 RC1
$

You can find pipes-http on Hackage or on Github.