tag:blogger.com,1999:blog-1777990983847811806.post2489355138117852496..comments2024-03-16T16:29:29.582-07:00Comments on Haskell for all: pipes-2.3 - Bidirectional pipesGabriella Gonzalezhttp://www.blogger.com/profile/01917800488530923694noreply@blogger.comBlogger15125tag:blogger.com,1999:blog-1777990983847811806.post-58700305325339999022012-09-10T20:52:11.267-07:002012-09-10T20:52:11.267-07:00This comment has been removed by the author.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-23354996667834504362012-09-10T20:33:33.937-07:002012-09-10T20:33:33.937-07:00I'm not sure I share your viewpoint, but I wil...I'm not sure I share your viewpoint, but I will keep an open mind and I look forward to seeing the future release.Paul Chiusanohttps://www.blogger.com/profile/04844651950877109501noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-30290162979077955192012-09-10T18:45:18.702-07:002012-09-10T18:45:18.702-07:00Check Control.Proxy.Tutorial for a concrete exampl...Check Control.Proxy.Tutorial for a concrete example. (the mixedClient function) of how you would want the result of composition to still be embeddable within a larger monad.<br /><br />The other reason I keep the monad is that I consider it inseparable from the core of why composition works in the first place. As I describe in the post immediately following this one, composition just weaves lists of Kleisli arrows together into a new list of Kleisli arrows and demonstrates a lot of elegant and non-trivial interactions between Kleisli composition and Proxy composition. This leads me to believe that the Kleisli category is not playing a supporting role to composition but rather is significantly intertwined with the very meaning of composition.<br /><br />I'm going to release several standard library functions in this next release and you will see a LOT of really elegant interactions between Kleisli composition ane Proxy composition when you see the source code. Proxies lend themselves to very elegant code, much more so than even Pipes.Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-8721348870482249422012-09-10T18:11:37.650-07:002012-09-10T18:11:37.650-07:00I was actually thinking of Stop as being nullary, ...I was actually thinking of Stop as being nullary, but I think I see now why you settled on the solution that you did. This might be a longer discussion, but my feeling is that it isn't necessary for Pipes to have a Return constructor and it somewhat confuses the model. You can have a separate type to give you monad syntax sugar for building up Pipes, but the Return isn't actually needed and any Pipe defined using monad sugar could in principle have been written directly - the monad for Pipe adds no expressive power. Ed calls this monad used for building up transducers a 'Plan', and it exists purely for convenience.<br /><br />With that removed, your transducers become _only_ a category rather than a category and a monad at the same time, which I think is cleaner and avoids tricky questions.<br /><br />By the way, this has been a great discussion, I'm really enjoying it! :)Paul Chiusanohttps://www.blogger.com/profile/04844651950877109501noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-58345652062144565252012-09-10T17:24:18.405-07:002012-09-10T17:24:18.405-07:00A Stop constructor version is fine so long as it i...A Stop constructor version is fine so long as it is isomorphic to the Maybe version I just described. However, a Stop constructor with no continuation doesn't work (I think... unless I misunderstand what you are saying).<br /><br />The corner case you have to consider is how to write a proper downstream identity pipe. If you use a Stop with no continuation to handle awaiting a terminated pipe, then there is no correct identity for:<br /><br />idP <+< return r = return r<br /><br />Practically speaking this means that the only pipe that can return is the most downstream one.<br /><br />However, the notion of most downstream is context-dependent, since it depends on where a pipe is in the final assembled pipeline, namely whether or not it is in the most downstream position. The category laws guarantee that you can reason about each pipe's behavior independent of its context so usually law violations like these indicate to me that there is something context-dependent about an implementation. A downstream identity law violation indicates that the most downstream position is being given special treatment, as in the above example.Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-70515832954862387712012-09-10T16:41:26.602-07:002012-09-10T16:41:26.602-07:00I see, you are just using the fact that (a -> b...I see, you are just using the fact that (a -> b, b) is isomorphic to Maybe a -> b. Though then you need a new type to handle the special rules for composing these pipes. Is there a good reason not to just add a Stop constructor and the extra fallback argument to the Await constructor?Paul Chiusanohttps://www.blogger.com/profile/04844651950877109501noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-39487933642993806522012-09-10T14:50:17.780-07:002012-09-10T14:50:17.780-07:00Yes, this is possible to write as an extension to ...Yes, this is possible to write as an extension to the Pipe type. I found the only way to correctly intercept upstream termination and preserve the Category is the following semantics:<br /><br />*) Wrap ordinary yielded values in a Just. When a pipe terminates, yield a final Nothing (which functions like your "Stop")<br />*) If a pipe receives a Nothing, it can still await again, but the next time it awaits it yields a Nothing first (so that downstream still has a chance to handle termination).<br /><br />The default "await" unwraps all Justs and just ignores Nothings and reawaits again. The more sophisticated await returns the raw Maybe value so you can handle upstream termination appropriately.<br /><br />This solution works with ordinary monads and the FreeT type and doesn't require indexed monads. Also, you can define a functor from the ordinary Pipe type to the extended Pipe type so all classic Pipe code is automatically compatible with this extended version.<br /><br />The problem is that it does not mix well with finalization, at least not for the Pipe type, and so the only way I was able to mix the two in the Frame type was to use indexed monads. I still haven't revisited it for the Proxy type to see if I can get it to mesh correctly now, so it's an open question I still have to address. However, finalization is the higher of the two priorities at the moment, after which I'm going to try my hand again at guarding against upstream termination.Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-17720549092955492292012-09-10T14:03:04.708-07:002012-09-10T14:03:04.708-07:00Thanks for your reply. The code you posted makes t...Thanks for your reply. The code you posted makes total sense, and I can see how it could be extended to send requests (containing actual values) to multiple servers and merging their results somehow.<br /><br />Have you considered making Pipe/Proxy into a MonadPlus, so the Await constructor takes a Pipe/Proxy to run if the awaiting fails? You then add a Stop constructor to PipeF, and the usual behavior is to Stop if an Await fails, but operations like merge can do something more intelligent. You can even make the fallback case a Producer, rather than a Pipe, to prevent it from awaiting on a nontrivial value (you'd probably have to use regular recursion rather than going through FreeT, though I'm not sure FreeT is buying you much anyway).Paul Chiusanohttps://www.blogger.com/profile/04844651950877109501noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-61615180100304895732012-09-10T10:01:57.798-07:002012-09-10T10:01:57.798-07:00Actually, as I explained in a comment to Paul belo...Actually, as I explained in a comment to Paul below, you can interact with multiple clients by nesting the Proxy monad transformer within itself.<br /><br />I really like the reactive demand programming idea. I toyed around with using pipes/proxies for reactive programming, but I've never really developed the idea significantly. I'd really like to see what you do with it.Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-47409063507625939292012-09-10T07:45:59.281-07:002012-09-10T07:45:59.281-07:00I've hpasted the code here:
http://hpaste.org...I've hpasted the code here:<br /><br />http://hpaste.org/74530<br /><br />The code has inline commentary in the comments.<br /><br />Resource safety is not a problem. Although I haven't released it yet, the general gist of it is that I write an extended composition that lets pipe register monoids associated with code segments and when the pipeline terminates it collects all the currently registered monoids and includes them in the return value.<br /><br />For the special case of the monoid being a finalizer (i.e. the "(Monad m) => m ()" monoid), then you register finalizers and the result contains all the remaining finalizers to run. Then you just extend runPipe to auto-run this last collected finalizer.<br /><br />This plays nice with nesting since the finalizer that is returned only belongs to the outermost pipe layer, so you don't accidentally get multiple frees.Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-70533160768987286392012-09-10T05:15:06.044-07:002012-09-10T05:15:06.044-07:00Very interesting, thanks for the response. I'm...Very interesting, thanks for the response. I'm a little hazy on how nesting proxies lets you combine values from multiple servers. Could you give a concrete example -- let's say I have two external sources of integers, and I'd like to merge these two into a single stream by always emitting the smaller of the two values from each source.<br /><br />One thing I like about Ed's library is that it lets you write pure (non-monadic) stream transducers that operate on multiple streams. Ed has a cap combinator that lets you partially apply a two-input machine, but that results in a transducer that is no longer pure. It sounds like nesting of proxies would have to do something similar. Are there any difficulties with ensuring resource-safety with this approach?Paul Chiusanohttps://www.blogger.com/profile/04844651950877109501noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-62466264037064049822012-09-09T20:09:03.771-07:002012-09-09T20:09:03.771-07:00Oops, I meant to reply directly to this but my res...Oops, I meant to reply directly to this but my response ended up as a separate top-level comment:<br /><br />http://www.haskellforall.com/2012/09/pipes-23-bidirectional-pipes.html?showComment=1347246470907#c5026677331628820503Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-50266773316288205032012-09-09T20:07:50.907-07:002012-09-09T20:07:50.907-07:00Actually, you can interact with multiple servers (...Actually, you can interact with multiple servers (or multiple clients, or multiple proxies). You just nest a Proxy within a Proxy. The outer proxy interacts with one session while the inner proxy interacts with an entirely different session. This isn't just hypothetical as I do this in my own code all the time and it works really well for me. This is also the exact same approach that Oleg and John Millikin advocate for enumerator and I'll explain why I agree with them.<br /><br />First off, this approach automatically works for interacting with multiple servers, clients, or proxies, whereas Edward's machines currently only support multiple servers. I assume that he'll write machines to handle the other two cases, but right off the bat you can see that even if he does then his approach leads to API complexity and overhead. Now you have three concepts instead of one, similar to the situation before pipes where people had three APIs for sources/sinks/transducers and three sets of semantics that the users had to keep track of. It would seem strange to liberate ourselves from that distinction only to reinvent it again.<br /><br />Ok, but say that I ignore that and let's concern ourselves with just servers for now. I will draw an analogy between the two alternative approaches and simple functions. You can imagine that the proxy approach of layering two proxies to interact with multiple inputs is analogous to curried functions:<br /><br />a -> b -> c<br /><br />Why curried functions? Because I can partially apply a layered proxy by composing and running just the outermost session, leaving behind a proxy that interacts with just a single session. Edward's machines are analogous to uncurrying functions:<br /><br />(a, b) -> c<br /><br />... except that he only really sweetens the special case of two inputs. For more inputs you end up with something analogous to:<br /><br />(((a, b), c), d) -> e<br /><br />... in which case you aren't sweetening that much and just trading one type of layering for another one.<br /><br />However, while I wouldn't use Edward's approach in my code, I still think it has a lot of potential for other purposes (such as possibly efficiency) and I can see how other people might like it for reasons of personal preference.<br /><br />In principle, anybody can write machines that compile to proxies (just like Edward's machines currently compile to pipes). However, I wouldn't write it myself, only because I can't actively maintain and advocate for something I don't strongly believe in. Somebody who believes in machines strongly should be the one to implement that.Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-46178846445077770912012-09-09T17:30:22.761-07:002012-09-09T17:30:22.761-07:00One limitation of this API is the server must alwa...One limitation of this API is the server must always respond with a value of the same type, which loses out on some really obvious use cases that come up when a client conceptually needs to pull from multiple servers. Ed Kmett's (very recently released) machines package has a solution to this problem. What do you think of it, and could you see pipes going in a similar direction in the future?Paul Chiusanohttps://www.blogger.com/profile/04844651950877109501noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-76821715424982735152012-09-06T16:18:31.336-07:002012-09-06T16:18:31.336-07:00Excellent work. Bidirectional will make the packag...Excellent work. Bidirectional will make the package usable in many more domains, and the client-server metaphor will help people understand the explanation (so long as they aren't thinking of multi-client concurrency).<br /><br />Regarding your comment about bidirectional with category laws: a paradigm I am developing, Reactive Demand Programming, achieves that (and generalized arrow laws). RDP is not message passing; it involves bidirectional data synchronization ('reactive' paradigm) and I elide events (such as messages) because they encourage a great deal of non-essential state. However, it is easy to model events with short-lived data.<br /><br />dmbarbourhttps://www.blogger.com/profile/12370605342201490009noreply@blogger.com