tag:blogger.com,1999:blog-1777990983847811806.post815102329499356906..comments2024-03-16T16:29:29.582-07:00Comments on Haskell for all: Perfect streaming using `pipes-bytestring`Gabriella Gonzalezhttp://www.blogger.com/profile/01917800488530923694noreply@blogger.comBlogger10125tag:blogger.com,1999:blog-1777990983847811806.post-74328796659083973012013-09-23T12:12:24.189-07:002013-09-23T12:12:24.189-07:00Yes, you would. For more sophisticated cases like...Yes, you would. For more sophisticated cases like this you can use `pipes-attoparsec` which turns any `attoparsec` parser into the equivalent `pipes-parse` parser (using the `parse` function). So in practice you could actually do the entire example within `pipes-attoparsec` instead of the way I just suggested.Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-59395802596900247132013-09-23T10:23:48.501-07:002013-09-23T10:23:48.501-07:00Don't you need to parse the ":" sepa...Don't you need to parse the ":" separately ? (and throw it away ?)Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-14455262477749353332013-09-23T08:50:21.668-07:002013-09-23T08:50:21.668-07:00So I generally provide a two-tiered approach. For...So I generally provide a two-tiered approach. For every function of producers I usually provide a lower-level parser that you can use to build it up step by step. The exception is `pipes-bytestring` where I left out most of the parsers because I wanted to get it out, but I plan on adding them later.<br /><br />I can use the example you gave: the way I would break down the problem is to first provide two parsers, one for keys and one for values, with these types:<br /><br />parseKey :: (Monad m) => StateT (Producer ByteString m r) m Key<br /><br />parseValue :: (Monad m) => StateT (Producer ByteString m r) m Value<br /><br />Then you could combine those into a parser for a key-value pair:<br /><br />parseKeyVal :: (Monad m) => StateT (Producer ByteString m r) (Key, Val)<br />parseKeyVal = (,) <$> parseKey <*> parseVal<br /><br />Then I would provide a high-level function that applied that parser repeatedly for user convenience:<br /><br />keyVals :: (Monad m) => Producer ByteString m r -> Producer (Key, Val) m r<br /><br />The idea is that the parsers are more reusable but the functions are more convenient, so the pipes ecosystem usually provides both. For an example of this, see the `pipes-binary` package, which provides the `decode` parser (to decode one element from the stream) and the `decodeMany` function (to decode a stream of elements).Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-74415684006132730982013-09-23T03:45:13.045-07:002013-09-23T03:45:13.045-07:00"Perfect streaming" is inseparable from ..."Perfect streaming" is inseparable from "Perfect parsing", it seems. Or at least from "Perfect regex-matching": splitting input at fixed separators/at fixed lengths/at encountering different sort of input symbols.<br /><br />By the way, imagine I have a string-producer which produces a potentially infinite string. I want to turn it into "Key-Value pairs" producer, by treating each line as a key-value pair in "key: value" format. So how would be "Producer KeyString" and "Producer ValueString" combined to express this?<br /><br />Joker_vDhttps://www.blogger.com/profile/15515260210737317416noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-44215590810868492452013-09-22T08:11:00.789-07:002013-09-22T08:11:00.789-07:00I'm generally reluctant to add type synonyms o...I'm generally reluctant to add type synonyms or type families unless they are "sort of opaque" (i.e. the user can meaningfully interact with without understanding what the type synonym expands to).<br /><br />In some trivial use cases this works (i.e. when consuming simple functions already written for you this way, such as `lines`, `takeFree`, and `unlines`), but in other cases if you don't know that it is using `FreeT` or how that works then you are missing out on a lot of functionality, such as traversing the data type by hand if there is not a `FreeT` recursion scheme for what you have in mind.<br /><br />Also, I would like people to grow more comfortable with using `FreeT` directly as I think it is a rather fundamental type, right up there with lists. For me, I would prefer to hide things that are ugly, but there is nothing ugly about `FreeT` in my mind.Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-74315096119829763612013-09-22T02:46:44.908-07:002013-09-22T02:46:44.908-07:00Yes, and perhaps to give simplified type signature...Yes, and perhaps to give simplified type signatures to functions.Sjoerd Visscherhttps://www.blogger.com/profile/10698430967044536619noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-30210731785041017112013-09-21T09:09:10.858-07:002013-09-21T09:09:10.858-07:00You mean like an automated way that new users can ...You mean like an automated way that new users can use to find out what the equivalent pipe type would be?Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-35292169718349149642013-09-21T08:52:44.600-07:002013-09-21T08:52:44.600-07:00Could you make a type family that turns the type s...Could you make a type family that turns the type signature of a lazy IO function into the type signature of a pipes function?Sjoerd Visscherhttps://www.blogger.com/profile/10698430967044536619noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-27536584847413840832013-09-20T18:09:30.190-07:002013-09-20T18:09:30.190-07:00For `conduit`, yes. For, `io-streams` you would n...For `conduit`, yes. For, `io-streams` you would need to modify input streams to return `Either r a` (where `r` is analogous to the `conduit`\`pipes` return value) instead of `Maybe a`, and make them functors over the `r` so that they would work with `FreeT`.Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-47709577369058700302013-09-20T18:05:18.377-07:002013-09-20T18:05:18.377-07:00Gabriel, wouldn't the `FreeT` trick (which is ...Gabriel, wouldn't the `FreeT` trick (which is great, by the way, I love the idea) work just as well for any other streaming framework, like Conduit or io-streams?John Wiegleyhttps://www.blogger.com/profile/04225185479676611787noreply@blogger.com