tag:blogger.com,1999:blog-1777990983847811806.post4685900715446546582..comments2024-03-16T16:29:29.582-07:00Comments on Haskell for all: Use Haskell for shell scriptingGabriella Gonzalezhttp://www.blogger.com/profile/01917800488530923694noreply@blogger.comBlogger22125tag:blogger.com,1999:blog-1777990983847811806.post-53572201392858548822015-09-10T10:33:20.372-07:002015-09-10T10:33:20.372-07:00Hi Gabriel,
thanks a lot!!!Hi Gabriel,<br /><br />thanks a lot!!!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-60747255097625483902015-09-10T07:34:04.842-07:002015-09-10T07:34:04.842-07:00`fork` automatically cancels the thread once the `...`fork` automatically cancels the thread once the `Managed` computation is done, so what your code was doing was creating the thread and then immediately canceling it afterwards.<br /><br />The most direct way to do what you want is to use the `async` library that `turtle` is based on top of. It provides the `mapConcurrently` function:<br /><br />http://hackage.haskell.org/package/async-2.0.2/docs/Control-Concurrent-Async.html#v:mapConcurrently<br /><br />... which you can use like this:<br /><br /> mapConcurrently (\vid -> shellStrict ...) (videos vids)<br /><br />... and that will run all the commands in parallel.<br /><br />You can do this with `turtle`'s `fork` but it would be trickier. You would have to do something like this:<br /><br /> loop :: [Video] -> Managed ()<br /> loop (vid:vids) = do<br /> async <- using (fork (shell ... vid ...))<br /> loop vids<br /> liftIO (wait async)<br /> loop [] = return ()<br /><br />... and then that would make sure that they all get fired off in parallel and the computation waits for the result.<br /><br />I should also update the documentation for `fork` to explain the auto-cancellation policy when the current thread is done. It's internally implemented in terms of `withAsync` from the `async` library which does the same thing.Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-21339042110296969622015-09-10T04:11:57.099-07:002015-09-10T04:11:57.099-07:00Dear Gabriel,
I would like to download some files...Dear Gabriel,<br /><br />I would like to download some files in parallel but can't get it working. Code works nice<br />without the "T.sh . T.using . T.fork" part. No separate processes are spawn. I don't know <br />how to fix the problem. Can you help me out?<br /><br />Thanks.<br /><br />Source code is here:<br />https://gist.github.com/schmidh/fa6112719e08c626db44Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-55253336018668205162015-09-07T09:50:11.444-07:002015-09-07T09:50:11.444-07:00You're welcome!You're welcome!Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-1353695774208993582015-09-07T03:38:58.698-07:002015-09-07T03:38:58.698-07:00Gabriel, you just made my day. I was using shelly ...Gabriel, you just made my day. I was using shelly for some scripting tasks, but I like your library much more. I've been playing with it for the last few hours and I already love it :-) Thank you!Anonymoushttps://www.blogger.com/profile/04991630866467543263noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-42110351670505771282015-07-25T16:28:33.744-07:002015-07-25T16:28:33.744-07:00But then you would need that Haskell-to-shell comp...But then you would need that Haskell-to-shell compiler to be installed on your system. Why not just install GHC instead of that Haskell-to-shell compiler?<br /><br />Also, keep in mind that you can compile an executable that you can distribute to users if they don't have GHC installed.<br /><br />The more long-term solution to this problem is to petition your distro maintainers to include GHC (or some by default with the distro.Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-35372451183287874312015-07-25T16:24:18.872-07:002015-07-25T16:24:18.872-07:00One of the main reasons I don't use Haskell fo...One of the main reasons I don't use Haskell for shell scripting is that I like having standalone scripts that work out-of-the-box. I often write shell scripts for things like configuration and installation where you can't assume anything but a minimal POSIX system. Using Haskell for this is troublesome because GHC needs to be installed first, whereas Bourne shell is available on all Unix flavors. This would be fine for doing Haskell-related tasks, but asking a user to install GHC just to do something entirely unrelated is a bit too much.<br /><br />I think there is a way to get the best of both worlds though, if there is some sort of Haskell-to-shell compiler. It doesn't even have to be fancy … an EDSL would probably work.Anonymoushttps://www.blogger.com/profile/13972277093409170484noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-41400673482204534772015-02-03T07:38:18.474-08:002015-02-03T07:38:18.474-08:00List monad is not counter-intuitive for pythonists...List monad is not counter-intuitive for pythonists. Implicit loops are weird for bash scripters.Yuriy Syrovetskiyhttps://www.blogger.com/profile/10564785392508591420noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-15010231513859655402015-02-03T07:37:56.340-08:002015-02-03T07:37:56.340-08:00This comment has been removed by the author.Yuriy Syrovetskiyhttps://www.blogger.com/profile/10564785392508591420noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-46317934621123658732015-02-03T07:27:30.791-08:002015-02-03T07:27:30.791-08:00I don't think the list monad is counter-intuit...I don't think the list monad is counter-intuitive to beginners. Most of these people will be familiar with for loops or Python's list comprehensions, so they can easily make the connection. I haven't had anybody complain that they actually found it counter-intuitive. Quite the opposite: Python programmers have complimented that it's easier to use than Python's API where you have to pick between easy and non-streaming or complex and streaming.Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-44223453476394820632015-02-02T23:57:52.511-08:002015-02-02T23:57:52.511-08:00Ok. So how about to get rid of Shell monad? When s...Ok. So how about to get rid of Shell monad? When something like `ls` multiplies continuation, it is counter-intuitive. Yes, I know about list monad, but newcomers don't, and `ls` in bash doesn't multiply ther rest of a script. Maybe `IO (Stream a)` fits best?Yuriy Syrovetskiyhttps://www.blogger.com/profile/10564785392508591420noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-81847577460693499612015-02-02T19:06:42.054-08:002015-02-02T19:06:42.054-08:00I actually did consider wrapping all `IO` actions ...I actually did consider wrapping all `IO` actions in `liftIO`. There are actually two ways to do this:<br /><br />A) Use the inferred type of:<br /><br />> MonadIO m => m r<br /><br />B) Specialize the type to `Shell`:<br /><br />> Shell r<br /><br />There were two problems with approach `A`:<br /><br />* I wanted concrete types to teach users with instead of type classes. One of the design constraints of the library was that it had to be teachable to absolute Haskell beginners.<br />* It's a leaky abstraction. Not everything `IO` action in the Haskell ecosystem is wrapped in `liftIO`, so the moment they deviate from the turtle library they are suddenly hit with `liftIO` anyway.<br /><br />There were a few problems with approach `B`:<br /><br />* There's no function of type `Shell a -> IO a`. The best you can do is `Shell a -> IO (Maybe a)` or `Shell a -> IO [a]`. That means that once you wrap an `IO` action in `Shell` the result can no longer be reused within a naked `IO` block.<br />* This actually makes the teaching process rougher on new Haskell programmers because I don't have any useful `IO` actions within the library to use as examples to explain how `IO` works.<br /><br />Generally there is a tension between the "principle of least permission" (giving things the narrowest types possible) and the proliferation of too many types. I decided that I could live with teaching new users the distinction between `IO` and `Shell` (since they'll have to learn it anyway if they want to use other Haskell libraries).<br /><br />I have a lot more to say on this subject, but a good starting point is my post on the Functor Design Pattern (http://www.haskellforall.com/2012/09/the-functor-design-pattern.html), which talks about how I prefer to specialize components as much as possible and delay unifying their types until the very last moment. Vice versa, I discourage pre-unifying everything into a monolithic component framework.Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-8276798614992985762015-02-02T08:01:33.200-08:002015-02-02T08:01:33.200-08:00Why some of turtle functions are IO actions, and s...Why some of turtle functions are IO actions, and some are Shell actions? How to unify it? How about everything in Shell and `main = sh $ do ...`?Yuriy Syrovetskiyhttps://www.blogger.com/profile/10564785392508591420noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-77099644349212309072015-01-31T10:15:08.795-08:002015-01-31T10:15:08.795-08:00`inshell` and `inproc` are how you embed external ...`inshell` and `inproc` are how you embed external commands as streams within your program. Here's a contrived example:<br /><br />> stdout (inproc "cat" [] stdin)<br /><br />That will take `stdin`, divert its stream through the external `cat` command, and then return the stream back to Haskell-land where it can be consumed by `stdout`. You can chain as many of these commands as you want in a row this way:<br /><br />> stdout (inproc "cmd2" ["a", "b"] (inproc "cmd1" ["x", "y"] stdin))<br /><br />That would be analogous to:<br /><br />> cmd1 x y | cmd2 a b<br /><br />So the precise answer to "What is `|`?" is "function application", since you are just chaining functions that transform streams. These functions all have type:<br /><br /> Shell Text -> Shell Text<br /><br />The answer to "What is `&`?" is the `fork` command from `Turtle.Prelude`. That lets you fork a background process and it returns a reference to the process (i.e. an `Async`) that you can use to manage it:<br /><br /> fork :: IO a -> Managed (Async a)<br /><br />The answer to "What is <()`" is just "function application" again. In turtle you don't pass streams by reference, but rather by value. That means there is no need for process substitution in the first place. I can just pass the lazy stream directly as an argument since it's a first-class value in Haskell.Gabriella Gonzalezhttps://www.blogger.com/profile/01917800488530923694noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-47125586269987705982015-01-31T04:14:52.316-08:002015-01-31T04:14:52.316-08:00I'm sure the author of Haskell's famed pip...I'm sure the author of Haskell's famed pipes library could provide an implementation of `|` :) As for `&` I would do that with the async library in a way that, in my opinion, if very readable:<br /><br />main = do<br />a <- async procA<br />b <- async procB<br />wait a<br />print "At least A is done"<br />wait b<br />print "Both A and B are done"<br /><br />There's probably a way to abstract things so`&` works as expected.<br /><br />I can't comment on <(), don't know Bash well enough.Gareth Charnockhttps://www.blogger.com/profile/03586753421018055751noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-42693638050149079582015-01-31T03:55:43.935-08:002015-01-31T03:55:43.935-08:00This comment has been removed by the author.Gareth Charnockhttps://www.blogger.com/profile/03586753421018055751noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-77848215223973468892015-01-30T23:53:04.245-08:002015-01-30T23:53:04.245-08:00I'm beginning to think this isn't a bad id...I'm beginning to think this isn't a bad idea. Christopher Donehttps://www.blogger.com/profile/11058249597696527532noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-65295878508028287232015-01-30T12:20:59.509-08:002015-01-30T12:20:59.509-08:00<() is basically function application.
| and &...<() is basically function application.<br /><br />| and & are provided by lazyness.Anonymoushttps://www.blogger.com/profile/18431008965885346468noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-46388881364640816102015-01-30T11:16:05.768-08:002015-01-30T11:16:05.768-08:00What if we had a quasi-quoter from bash syntax to ...What if we had a quasi-quoter from bash syntax to turtle? It wouldn't help for the interpreted case, but it would make writing compiled scripts dead easy, since your target audience presumably already knows that syntax. Then they'd have the power of Haskell within reach when they were ready for it.John Wiegleyhttps://www.blogger.com/profile/04225185479676611787noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-56742286399840835922015-01-30T10:51:52.797-08:002015-01-30T10:51:52.797-08:00Great idea! I applaud the effort. I as well would ...Great idea! I applaud the effort. I as well would request pipe as it's a mandatory feature.ESTraderhttps://www.blogger.com/profile/15865815442255396133noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-25691522088471017892015-01-30T02:08:58.001-08:002015-01-30T02:08:58.001-08:00This isn't a meaningful alternative to bash un...This isn't a meaningful alternative to bash until you show equivalents to at least these bash operators:<br /><br />| (pipe)<br />& (background job) and wait<br /><() (create fifo pulling input from subshell)<br /><br />Most shell scripts are gluing together other commands written in a variety of different languages. It's an orchestration language with simple job control and trivial parallelization via pipes.<br /><br />At best, this currently seems to be a replacement for DOS's batch interpreter, which is scarcely even a scripting language.Barry Kellyhttps://www.blogger.com/profile/10559947643606684495noreply@blogger.comtag:blogger.com,1999:blog-1777990983847811806.post-38029772580171182762015-01-30T00:25:23.178-08:002015-01-30T00:25:23.178-08:00Cool. :-) Looks very attractive, newbies and scrip...Cool. :-) Looks very attractive, newbies and scripters alike should like it. Is there a story for piping like in shell-conduit? <br /><br />I made a major version bump to formatting (6.2.0) to include this simplification, I'd been meaning to drop the Holey type for a while after it became clear abstracting over the particular monoid wasn't useful. Oh, I discovered this nifty Monoid instance recently, check it out: https://github.com/chrisdone/formatting#using-more-than-one-formatter-on-the-same-argument Christopher Donehttps://www.blogger.com/profile/11058249597696527532noreply@blogger.com