Sunday, January 28, 2018

Dhall Survey Results (2017-2018)

I advertised a survey in my prior post to collect feedback on the Dhall configuration language. You can find the raw results here:

18 people completed the survey at the time of this writing and this post discusses their feedback.

This post assumes that you are familiar with the Dhall configuration language. If you are not familiar, then you might want to check out the project website to learn more:

"Which option describes how often you use Dhall"

  • "Never used it" - 7 (38.9%)
  • "Briefly tried it out" - 7 (38.9%)
  • "Use it for my personal projects" - 1 (5.6%)
  • "Use it at work" - 3 (16.7%)

The survey size was small and the selection process is biased towards people who follow me on Twitter or read my blog and are interested enough in Dhall to answer the survey. So I don't read too much into the percentages but I am interested in the absolute count of people who benefit from using Dhall (i.e. the last two categories).

That said, I was pretty happy that four people who took the survey benefit from the use of Dhall. These users provided great insight into how Dhall currently provides value in the wild.

I was also surprised by how many people took the survey that had never used Dhall. These people still gave valuable feedback on their first impressions of the language and use cases that still need to be addressed.

"Would anything encourage you to use Dhall more often?"

This was a freeform question designed to guide future improvements and to assess if there were technical issues holding people back from adopting the language.

Since there were only 18 responses, I can directly respond to most of them:


Mainly, a formal spec (also an ATS parsing implementation)

Backends for more languages would be great.

More compilation outputs, e.g. Java

The main thing is more time on my part (adoption as JSON-preprocessor and via Haskell high on my TODO), but now that you ask, a Python binding

Stable spec, ...

Julia bindings

Scala integration, ...

I totally agree with this feedback! Standardizing the language and providing more backends are the highest priorities for this year.

No language was mentioned more than once, but I will most likely target Java and Scala first for new backends.


Some way to find functions would be immensely helpful. The way to currently find functions is by word of mouth.

This is a great idea, so I created an issue on the issue tracker to record this suggestion:

I probably won't have time myself to implement this (at least over the next year), but if somebody is looking for a way to contribute to the Dhall ecosystem this would be an excellent way to do so.


I would use Dhall more often if it had better type synonym support.

...; ability to describe the whole config in one file

Fortunately, I recently standardized and implemented type synonym in the latest release of the language. For example, this is legal now:

    let Age = Natural

in  let Name = Text

in  let Person = { age : Age, name : Name }

in  let John : Person = { age = +23, name = "John" }

in  let Mary : Person = { age = +30, name = "Mary" }

in  [ John, Mary ]

I assume the feedback on "ability to describe the whole config in one file" is referring a common work-around of importing repeated types from another file. The new type synonym support means you should now be able to describe the whole config in one file without any issues.


Verbosity for writing values in sum types was a no-go for our use case

...; more concise sum types.

The most recent release added support for the constructors keyword to simplify sum types. Here is an example of this new keyword in action:

    let Project =
          constructors
          < GitHub  : { repository : Text, revision : Text }
          | Hackage : { package : Text, version : Text }
          | Local   : { relativePath : Text }
          >

in  [ Project.GitHub
      { repository = "https://github.com/Gabriel439/Haskell-Turtle-Library.git"
      , revision   = "ae5edf227b515b34c1cb6c89d9c58ea0eece12d5"
      }
    , Project.Local { relativePath = "~/proj/optparse-applicative" }
    , Project.Local { relativePath = "~/proj/discrimination" }
    , Project.Hackage { package = "lens", version = "4.15.4" }
    , Project.GitHub
      { repository = "https://github.com/haskell/text.git"
      , revision   = "ccbfabedea1cf5b38ff19f37549feaf01225e537"
      }
    , Project.Local { relativePath = "~/proj/servant-swagger" }
    , Project.Hackage { package = "aeson", version = "1.2.3.0" }
    ]

Missing a linter usable in editors for in-buffer error highlighting.

..., editor support, ...

Dhall does provide a Haskell API including support for structured error messages. These store source spans which can be used for highlighting errors, so this should be straightforward to implement. These source spans power Dhall's error messages, such as:

$ dhall <<< 'let x = 1 in Natural/even x'

Use "dhall --explain" for detailed errors

Error: Wrong type of function argument

Natural/even x

(stdin):1:14

... the source span is how Dhall knows to highlight the Natural/even x subexpression at the bottom of the error message and that information can be programmatically retrieved from the Haskell API.


Sometimes have to work around record or Turing completeness limitations to generate desired json or yaml: often need record keys that are referenced from elsewhere as strings.

I will probably never allow Dhall to be Turing complete. Similarly, I will probably never permit treating text as anything other than an opaque value.

Usually the solution here is to upstream the fix (i.e. don't rely on weakly typed inputs). For example, instead of retrieving the field's name as text, retrieve a Dhall function to access the desired field, since functions are first-class values in Dhall.

If you have questions about how to integrate Dhall into your workflow, the current best place to ask is the issue tracker for the language. I'm not sure if this is the best place to host such questions and I'm open to suggestions for other forums for support.


Would like to be able to compute the sha1 or similar of a file or expression.

The Dhall package does provide a dhall-hash utility exactly for this purpose, which computes the semantic hash of any file or expression. For example:

$ dhall-hash <<< 'λ(x : Integer) → [1, x, 3]'
sha256:2ea7142f6bd0a1e70a5843174379a72e4066835fc035a5983a263681c728c5ae

Better syntax for optional values. It is abysymmal now. But perhaps I'm using dhall wrong. My colleague suggests merging into default values.

Dropping the lets

simpler syntax; ...

There were several complaints about the syntax. The best place to discuss and/or propose syntax improvements is the issue tracker for the language standard.

For example:


Better error messages; ...

The issue tracker for the Haskell implementation is the appropriate place to request improvements to error messages. Here are some recently examples of reported issues in error messages:


I worry that contributors/friends who'd like to work on a project with me would be intimidated because they don't know the lambda calculus

Dhall will always support anonymous functions, so I doubt the language will be able to serve those who shy away from lambda calculus. This is probably where something like JSON would be a more appropriate choice.


..., type inference

..., full type inference, row polymorphism for records

This is probably referring to Haskell/PureScript/Elm-style inference where the types of anonymous functions can be inferred (i.e. using unification). I won't rule out supporting type inference in the future but I doubt it will happen this year. However, I welcome any formal proposal to add type inference to the language now that the type-checking semantics have been standardized.


It's cool, but I'm yet to be convinced it will make my life easier

No problem! Dhall does not have a mission to "take over the world" or displace other configuration file formats.

Dhall's only mission is to serve people who want programmable configuration files without sacrificing safety or ease of maintenance.

"What do you use Dhall for?"

"Why do you use Dhall?"

This was the part of the survey that I look forward to the most: seeing how people benefit from Dhall. These were two freeform questions that I'm combining in one section:


WHAT: HTML templating, providing better manifest files (elm-package.json)

WHY: Dhall as a template language allows us to express all of the inputs to a template in one place. We don't have to hunt-and-peck to find out what a template is parameterized by. And if we want to understand what some variable is in the template, we can just look at the top of the file where its type is declared. We want exact dependencies for our elm code. There is no specific syntax for exact versions in elm-package.json. There is a hack around that limitation which requires enforcement by convention (or CI scripts or some such). We use Dhall to generate exact versions by construction. The JSON that's generated uses the hack around the limitation, but we don't have to think about that accidental complexity.


WHAT: Ad hoc command-line tools


WHAT: Generating docker swarm and other configs from common templates with specific variables inserted safely.

WHY: JSON and yaml suck, want to know about errors at compile time.


WHAT: Kubernetes

WHY: Schema validation. Kubernetes is known for blowing up in your face at runtime by doing evaluation and parsing at the same time


WHAT: Config

WHY: Strong typing & schema


WHY: Rich functinality and concise syntax


WHAT: JSON preprocessor

WHY: JSON is universal but a bit dumb


WHAT: configuration - use haskell integration and spit out json/yaml on some projects (to kubernetes configs for instance)

WHY: I love pure fp and dealing with configs wastes a lot of my time and is often error prone


Interestingly, ops-related applications like Docker Swarm and Kubernetes were mentioned in three separate responses. I've also received feedback (outside the survey) from people interested in using Dhall to generate Terraform and Cloudformation templates, too.

This makes sense because configuration files for operational management tend to be large and repetitive, which benefits from Dhall's support for functions and let expressions to reduce repetition.

Also, reliability matters for operations engineer, so being able to validate the correctness of a configuration file ahead of time using a strong type system can help reduce outages. As Dan Luu notes:

Configuration bugs, not code bugs, are the most common cause I’ve seen of really bad outages. When I looked at publicly available postmortems, searching for “global outage postmortem” returned about 50% outages caused by configuration changes. Publicly available postmortems aren’t a representative sample of all outages, but a random sampling of postmortem databases also reveals that config changes are responsible for a disproportionate fraction of extremely bad outages. As with error handling, I’m often told that it’s obvious that config changes are scary, but it’s not so obvious that most companies test and stage config changes like they do code changes.

Also, type-checking a Dhall configuration file is a faster and more convenient to valid correctness ahead-of-time than an integration test.

Functions and types were the same two justifications for my company (Awake Security) using Dhall internally. We had large and repetitive configurations for managing a cluster of appliances that we wanted to simplify and validate ahead of time to reduce failures in the field.

Another thing that stood out was how many users rely on Dhall's JSON integration. I already suspected this but this survey solidified that.

I also learned that one user was using Dhall not just to reduce repetition, but to enforce internal policy (i.e. a version pinning convention for elm-package.json files). This was a use case I hadn't thought of marketing before.

Other feedback

We tried it as an option at work for defining some APIs but describing values in sum types was way too verbose (though I think I understand why it might be needed). We really like the promise of a strongly typed config language but it was pretty much unusable for us without building other tooling to potentially improve ergonomics.

As mentioned above, this is fixed in the latest release with the addition of the constructors keyword.


Last time I checked, I found syntax a bit awkward. Hey, Wadler's law :) Anyway, great job! I'm not using Dhall, but I try to follow its development, because it's different from everything else I've seen.

Syntax concerns strike again! This makes me suspect that this will be a recurring complaint (similar to Haskell and Rust).

I also appreciate that people are following the language and hopefully getting ideas for their own languages and tools. The one thing I'd really like more tools to steal is Dhall's import system.


You rock. Thanks for all your work for the community.

Dhall is awesome!

KOTGW

I love the project and am grateful for all the work that's been done to make it accessible and well documented. Keep up the good work!

Thank you all for the support, and don't forget to thank other people who have contributed to the project by reporting issues and implementing new features.

2 comments:

  1. I think json-to-dhall would be very valuable. Especially if it could take highly duplicated JSON and generate less duplicated dhall configuration code.

    Additionally, having the example transformations from json to dhall and allowing users to generate dhall based on json configurations they're familiar with would let them gain an intuition for how it works.

    They could immediately see potential benefits for them and how much/how little they would have to learn.

    ReplyDelete
    Replies
    1. I think the main issue for that use case is what to do if the user provides weakly typed JSON, such as a list with elements of different types. See also: https://github.com/dhall-lang/dhall-haskell/issues/326

      Delete