Monday, February 10, 2020

Dhall Survey Results (2019-2020)

dhall-2020

The results are in for this year’s Dhall survey, which you can find here:

You might also want to compare to the summary of last year’s survey:

Note that I will not include all write-in responses, but you can consult the above links if you want to read them all. I will only highlight responses that are representative of clear themes in survey feedback. I am also omitting praise from this summary (which I greatly appreciate, though!), because I would like to draw attention to what needs improvement.

Adoption

This year a greater number of survey respondents reported using Dhall at work:

Which option best describes your usage of Dhall:

  • 9 (14.3%) - Never used Dhall
  • 13 (20.6%) - Briefly tried Dhall
  • 11 (17.5%) - Use Dhall (but only for personal projects)
  • 11 (17.5%) - Use Dhall at work (but only me)
  • 19 (30.2%) - Use Dhall at work along with coworkers

… compared to last year:

  • 7 (11.7%) - Never used it
  • 22 (36.7%) - Briefly tried it out
  • 11 (18.3%) - Use it for my personal projects
  • 19 (31.7%) - Use it at work
  • 1 ( 1.7%) - Write in: Trying to convince work people to use it

… even though fewer people completed the survey this year (down from 73 responses to 64).

The most likely reason for the smaller number of responses was the greater length of the survey. This will also likely influence the distribution of responses since people more interested in Dhall will be more motivated to complete this year’s longer survey. Next year I will probably trim the survey down again in order to gather a larger sample size.

Even taking that into account, the number of respondents using Dhall at work grew in absolute terms.

This year’s survey added a new category to distinguish whether people using Dhall at work were doing so alone or alongside their coworkers. I was pleased to see that Dhall users do not appear to have difficulty persuading their coworkers to use Dhall.

Reasons to adopt

What do you use Dhall for?

CI / CD / Ops (especially Kubernetes) continue to be the most prominent use cases for using Dhall:

Writing environment variables for configurations of containerized application

deployment configs and secrets management

Higher-level common configuration from which tool configuration is derived.

project configuration (CI, K8s, etc) & templating

Generating ECS task definitions

SRE/DevOps related configuration

Kubernetes mostly + some glue config with AWS

Kubernetes config

Mostly for kubernetes cluster configuration.

Configuration of my custom build system

dhall-kubernetes

Concourse Pipelines

publish interfaces for Ansible roles to make their usage easier through Dhall based config

Simple configuration for Haskell app, prototype kubernetes cluster config

Generate the yaml passed to –values for helm. Some gitlab ci configuration.

Application server configuration, database replacement

Setting up Docker Compose files for parts of our product to be used for automatic testing.

Configuration of application in kubernetes

Kubernetes management and shared config (application and infrastructure)

Generating yaml and build files

CI setup (generate ansible yml files)

Built bitbucket pipeline typing. Also internal configuration for a pdf-filler application.

Ansible config

For configuration and build utility (combinded with Shake)

Package definitions for a Linux ports tree builder I was building

Customer-specific configuration files

My favorite response was:

Dhall is the configuration format for my company’s product

However, there were still a few responses that didn’t fall into one of the DevOps use cases:

Defining a schema for metadata

Test data generation

Configuration of a personal chat bot, dhall-cabal,

Game data, templating LaTeX, canonical source of truth (upstream of JSON) for Elm apps.

dzen-dhall

Configuration for personal projects and generating DBC files at work

One surprising result was that only person wrote in Spago (PureScript’s package manager) as how they used Dhall:

PureScript through spago

… even though 7 survey respondents reported using Spago in a subsequent section! In fact, 5 of them wrote in completely different use cases for Dhall. This suggests a potential issue with the survey design: people might have chosen not to write in answers that were already covered by subsequent questions.

Other responses simply noted that Dhall is a programmable configuration language:

Adding a small amount of automation to config generation

Move non-turing-complete data out of config/programs, declarative programs that get interpreted as data

Configuration

Configuration

…configuration

configuration

Configuration generation

Generate yaml configuration files

configuration of programs written in haskell

The pitch

One of the things we do periodically is refine our “pitch”, based on feedback from users (including, but not limited to, these surveys). This section covers reasons to use Dhall in users’ own words when they answered:

Why do you use Dhall?

One of the most interesting answers to me was from the same respondent who said that Dhall was the configuration format for their company’s product. In their own words:

Because YAML is horrible. Because having multiple configuration files for a single product is horrible and dhall is the best “big config file” format.

The above pitch must be pretty compelling for a company to embrace Dhall to that extent. In fact, the latter half of the pitch is fairly similar to the one currently used by dhall-lang.org, focusing on Dhall’s suitability for configuration “in the large”, although we’ve recently de-emphasized replacing YAML specifically. Several commercial users provided input into that branding (and one of them could have been the same person who left the above feedback).

Being a “big config file” format can imply many different virtues, but in my experience users usually mean the the following programming language features:

  • Types (to reduce errors at scale)
  • Functions (to avoid repeating one’s self)
  • Imports (to have a single source of truth that one can import)
  • Integrations (to unify disparate configuration formats)

… and several responses were related to these “scale”-related reasons to adopt:

Consistent typing across deployment configs and environments

strong types, functions and remote (decentralised) importing

Type safety, avoid writing yaml by hand

It allows us to provide a function via environment variable

Higher-level common configuration from which tool configuration is derived.

It replaced a lot of redundant JSON configs; adding new services is a lot quicker now, and cross-cutting changes less painful

To avoid copy&paste and the maintanance problems caused by it

Type safety, safe imports, no YAML indentation madness.

To help reduce code duplication and avoid errors in the values and keys of configuration filez. Mainly used it so far to help setup a more maintainable set of Kinect clusters and users for myself, which also made it easier to add.

It has a strong type system

Type-safety, the ability to move a lot of logic/formatting (e.g. in templating) into Dhall proper, the ability to represent ADTs/union types.

Abstraction!

Because i can abstract boilerplate

It provides a sane language for configuring systems (with types and abstraction)

Type checking, configuration typo safety, factorising yaml

Abstract useful config + defaults

Doesn’t have the weird gotchas of yaml. Possible to make DRY config files.

The main reason has been to decrease duplication.

type safety

I love the idea of strongly typed config, and also the programming aspect means there is less chance for error since you can reuse a component instead of copy/paste. I have been pushing for dhall at work as there have been very severe incidents at work from bad config files

Strong type safety

I like the ability to evaluate expressions, value substitution

Strong types and non-repetitive abstractions. Ease of publishing modules

our CI setup is now big and has lots of commonalities between projects. dhall helps us avoid preventable mistakes thanks to type checks and allows us to share common config with functions

it’s typed and avoids repetition

Not repeating common parts of configuration (like Kubernetes YAML hell)

Normally a general-purpose programming language can also do these things, and several respondents noted why they preferred Dhall over a general-purpose language:

It’s non-Turing-complete, but allows imports and static typing

It’s convenient and provides a good power-to-weight ratio

Only non-turing-complete config language that is based on a lambda calculus (and is modern/not a hobby project)

Because it won’t crash at runtime (totality)

It is by far the most well-wrought typed configuration language

I want to use something with proper type theory behind it and dhall is actually almost useful for some problems we have at work

It’s a nice config language

It’s quick, compact and type-safe

simple but effektive system f, i like that

Others explained the motivation in terms of the current solution they were struggling with and trying to replace (commonly YAML):

To make sense of this pile of configuration mess

Verifying and reading through our combo of custom scripts, chef, and other solutions was a nightmare

so I don’t need to create my own configuration language and other existing ones suck more (json, yaml, that weird python thing…)

Because YAML and database were not the choices

For all the benefits over YAML

better helm-templates

Documentation

This year we asked people what needed better documentation to see if there were any blind spots in our current coverage:

What needs better documentation?

Like last year, people most commonly requested documentation on best practices and design patterns:

Recursive things. (This has been discussed on Slack somewhat.)

Packaging

I’d love a manual on best practices and a contributing guide to the Haskell implementation

Cookbooks

The prelude/standard library, or rather, surfacing documentation centrally in browsable form

How to create defaults, for records with optional values. How to design types with backward compatibility in mind.

Nested record updating, though the blog post shows some improvement here with the default types :: syntax.

How to make use of dhall at the peripheries of a large project.

How to deal with generated files (!), e.g. CI yaml config; how to include dhall projects into nix configs (dhallToNix); best practices for pre-caching imports

Imports section could be a bit more extensive like import prelude, import private repos (GitHub, BitBucket), multi imports. currently the information is scattered around various pages.

Perhaps patterns. For example, we have { a :: ty1, b:: Maybe ty2} and want users to be able to write { a = val } without b = None or default \ { a = val }

(this is probably because I didn’t google it properly), how to properly deal with freezing and updating hashes

Best practices and real-world examples. I’d love to use something like Dhall for managing configuration at work, but it’s very hard to tell if it will handle my usecases well, and it’s hard to dive in and try it out because I don’t know if I’m doing it right

We have made progress on that front in two forms:

  • The Dhall Configuration Language Manual

    This manual was created as a series of how-to guides for common tasks and idioms related to the language. Feedback from these surveys helps inform what topics I choose for each chapter.

  • Some design patterns became language features

    The most notable example this year is standardizing support for the record completion operator for better handling of defaults

In fact, several improvements year (and some currently in progress) are directly inspired by my work on the book. Any time I describe a workflow that seems too painful I make changes to the language, tooling, or ecosystem to smooth things over.

Besides the book, the thing most likely to improve over the coming year is packaging, documenting, and discovering new Dhall packages. For example, I just created a Google Summer of Code project proposal for a student work on a documentation generator for Dhall packages:

The second most common request was to improve documentation for the core language features:

Still not sure how the merge keyword works.

i somehow have trouble finding the doc page about record operations and iirc the record projection thing where you take the a subset of a record’s field is not included in the page listing records operations

Importing of other files

The introduction to FP. Lots of devs work in Go, Ruby, etc and need help thinking about polymorphism with sum types. Also more clarification on the let...in syntax in early guides and an explanation on why you need :let in the repl.

I tend to have a hard time finding comprehensive info on syntactic features and end up hearing about them on guthub issues

Common errors like “-/+” syntax, “not a function”

the type system

This is understandable because the closest thing Dhall has to a complete resource on this is the Haskell implementation’s tutorial:

One of my short-term goals is to translate this into a language-independent tutorial.

There is an existing language-independent tutorial for translating Dhall to JSON:

… but that doesn’t cover all of the language features like the Haskell tutorial does.

Language bindings

Which language bindings do you currently use?

  • 37 (84.1%) - Haskell
  • 6 (13.6%) - Bash
  • 5 (11.4%) - Nix
  • 3 ( 6.8%) - Ruby
  • 2 ( 4.5%) - Rust
  • 1 ( 2.3%) - Golang
  • 1 ( 2.3%) - Swift
  • 0 ( 0.0%) - Clojure
  • 0 ( 0.0%) - Eta
  • 0 ( 0.0%) - Java (via Eta)

The number of Haskell users is not surprising given that the Haskell implementation also powers the shared command-line tools, like dhall/dhall-to-{json,yaml}/dhall-lsp-server. Many Dhall users do not use Haskell the language and instead use the Haskell implementation to generate JSON or YAML from Dhall while waiting for a language binding for their preferred language.

I think the one response for Swift might have been a mistaken answer intended for the next section. As far as I know there currently are not Dhall bindings to Swift (not even ones in progress).

One of the interesting take-aways from the above question is that the JVM is one of the areas where Dhall is not being used as a native language binding despite existing bindings. I get the impression that most JVM users are waiting for Java/Scala bindings.

Desired language bindings

Which language bindings would you like to see get more attention?

  • 17 (39.5%) - Python
  • 12 (27.9%) - Scala
  • 11 (25.6%) - PureScript
  • 9 (20.9%) - JavaScript
  • 7 (16.4%) - Go
  • 7 (16.3%) - Java
  • 3 ( 7.0%) - C++
  • 3 ( 7.0%) - Elm
  • 2 ( 4.7%) - C#
  • 2 ( 4.7%) - Kotlin
  • 2 ( 4.7%) - Rust
  • 1 ( 2.3%) - Swift
  • 1 ( 2.3%) - TypeScript
  • 1 ( 2.3%) - PHP
  • 1 ( 2.3%) - Perl
  • 1 ( 2.3%) - C
  • 1 ( 2.3%) - A C/Rust library so all the other langs can bind to

Python is an interesting response because at one point there was progress on a Python binding to Dhall, but that stalled out.

The demand for Python makes sense because Python is used heavily in Dhall’s primary use case (CI / CD / Ops), alongside Go. In fact, Go was listed as well, although possibly not mentioned as often due to the Go binding to Dhall being far closer to completion.

In fact, the Go binding just announced the first release candidate for version 1.0.0:

Note that survey respondents preferred bindings in functional languages over their more widely used imperative counterparts. For example, there was greater demand for Scala compared to Java and greater demand for PureScript compared to JavaScript. This might owe to Dhall’s functional programming heritage.

A few survey respondents appear to not be aware that there is a complete Rust binding to Dhall now available. This is understandable, though, given that the Rust binding only officially announced recently.

Integrations

Which of the following integrations do you use?

  • 26 (63.4%) - JSON (via dhall-to-json)
  • 25 (61.0%) - YAML (via dhall-to-yaml)
  • 10 (24.4%) - Kubernetes (via dhall-to-kubernetes)
  • 7 (17.1%) - JSON (via Prelude.JSON.render)
  • 7 (17.1%) - purescript-packages (via spago)
  • 5 (12.2%) - YAML (via Prelude.JSON.renderYAML)
  • 3 ( 7.3%) - Cabal (via dhall-to-cabal)
  • 1 ( 2.4%) - Write-in: Nix (via dhall-to-nix)
  • 0 ( 0.0%) - XML (via dhall-to-xml)
  • 0 ( 0.0%) - XML (via Prelude.XML.render)
  • 0 ( 0.0%) - TOML (via JSON)

The thing I take away from the above numbers is that a large number of people would still benefit from language bindings and they currently work around the absence of a language binding by generating JSON/YAML.

Desired integrations

Which integrations would you like to see get more attention?

  • 22 (68.8%) - Terraform
  • 11 (34.4%) - Docker Compose
  • 9 (28.1%) - HCL
  • 8 (25.0%) - Prometheus
  • 4 (12.5%) - Packer
  • 2 ( 6.3%) - INI
  • 2 ( 6.3%) - Concourse
  • 2 ( 3.1%) - Write-in: Ansible
  • 1 ( 3.1%) - GoCD
  • 1 ( 3.1%) - Write-in: Grafana
  • 1 ( 3.1%) - Write-in: Nix, Nixops
  • 1 ( 3.1%) - Write-in: Dockerfile
  • 1 ( 3.1%) - Write-in: Google Cloud Builder
  • 1 ( 3.1%) - Write-in: Travis
  • 1 ( 3.1%) - Write-in: Vault
  • 1 ( 3.1%) - Write-in: GitHub Actions
  • 1 ( 3.1%) - Write-in: Drone CI
  • 1 ( 3.1%) - Write-in: CloudFormation
  • 1 ( 3.1%) - Write-in: Jenkins
  • 1 ( 3.1%) - Write-in: TOML
  • 1 ( 3.1%) - Write-in: Bitbucket pipelines

Terraform was far and away the most requested integration. One of the interesting challenges about this potential integration is figuring out what is the right way to integrate Dhall with Terraform because Terraform has its own programming features (like a DSL for defining function-like modules).

Dhall packages

Which of the following Dhall packages do you use?

  • 32 (100.0%) - Prelude
  • 4 ( 12.5%) - dhall-packages (Dhall monorepo)
  • 1 ( 3.1%) - hpack-dhall (hpack bindings)
  • 1 ( 3.1%) - github-actions-dhall (GitHub Actions bindings)
  • 1 ( 3.1%) - dhall-terraform (Terraform bindings)
  • 1 ( 3.1%) - dhall-semver (Semantic versions)
  • 1 ( 3.1%) - dhall-concourse (Concourse bindings)
  • 1 ( 3.1%) - dhall-bhat (Haskell type classes in Dhall)
  • 1 ( 3.1%) - dada (Recursion schemes)
  • 0 ( 0.0%) - dho (CircleCI bindings)
  • 0 ( 0.0%) - dhallql (Query language)
  • 0 ( 0.0%) - dhallia (Dhall as an IDL)
  • 0 ( 0.0%) - cpkg (C package manager)
  • 0 ( 0.0%) - caterwaul (Category theory)

Unsurprisingly, most people use the Prelude. The thing that did catch my eye was how many respondents used dhall-packages (mainly because I hadn’t realized how fast it had grown since the last time I checked it out).

dhall-packages appears to have the potential to grow into the Dhall analog of Helm, meaning a repository containing useful types and predefined recipes for deploying Kubernetes services. I can see this repository easily giving Dhall a competitive edge in the Kubernetes space since I know quite a few people are looking for a Helm alternative without the headaches associated with templating YAML.

ASCII vs Unicode

dhall format tries to be as opinionated as possible, but currently permits one way to customize behavior: the tool can either emit Unicode symbols (e.g. λ, →, ∀) or ASCII symbols (e.g. \, ->, forall).

I asked several questions about ASCII versus Unicode to see if there was a possibility of standardizing on one or the other. Unfortunately, people were split in this regard. The only thing they agreed upon was that they preferred not to input Unicode symbols directly:

Do you prefer to input ASCII or Unicode symbols in your editor (before formatting the code)?

  • 58 (93.5%) - ASCII
  • 4 ( 6.5%) - Unicode

… but on the other two questions people split pretty evenly, with a slight preference for Unicode:

How do you format your code?

  • 24 (42.9%) - Unicode
  • 22 (39.3%) - ASCII
  • 6 (10.7%) - I don’t format my code
  • 4 ( 7.3%) - Other

Do you prefer to read Dhall code that uses ASCII or Unicode symbols?

  • 28 (50.9%) - Unicode
  • 25 (45.5%) - ASCII
  • 2 ( 3.6%) - Other

What’s interesting is that you get clearer preferences when you slice the data by how much people use Dhall.

For example, people who have never used Dhall or briefly tried Dhall prefer ASCII by roughly a 3-to-1 margin:

How do you format your code?

  • 10 (66.7%) - ASCII
  • 3 (20.0%) - I don’t format my code
  • 2 (13.3%) - Unicode

Do you prefer to read Dhall code that uses ASCII or Unicode symbols?

  • 12 (75.0%) - ASCII
  • 4 (25.0%) - Unicode

… whereas other categories (e.g. personal projects or work) prefer Unicode by roughly a 2-to-1 margin:

How do you format your code?

  • 22 (55.0%) - Unicode
  • 11 (27.5%) - ASCII
  • 4 (10.0%) - Other
  • 3 ( 7.5%) - I don’t format my code

Do you prefer to read Dhall code that uses ASCII or Unicode symbols?

  • 23 (60.5%) - Unicode
  • 13 (34.2%) - ASCII
  • 2 ( 5.3%) - Other

There are several possible ways to interpret that evidence:

  • Perhaps Dhall could expand its potential audience by formatting ASCII

  • Perhaps people prefer the Unicode syntax the more they use Dhall

  • Perhaps there is a “founder effect” since originally dhall format only supported Unicode

Either way, I don’t plan on making any changes to dhall format immediately, but I will use this data to inform future formatting discussions on Discourse.

One person also added Unicode-related feedback in the “Other feedback” section:

I feel strongly that unicode symbols are not worth supporting indefinitely, as they typically can’t be typed and add mental overhead when reading. The symbols themselves also have a mathematical bent, which can be intimidating for those not well versed in math / logic programming.

Growth

Would anything encourage you to use Dhall more often?

Language bindings:

I wish there was a good way to do nix through dhall, but I don’t have any good suggestions.

something like the dhall Haskell library for purescript

Js/purescript bindings in release state. …

Scala/JVM bindings (Eta is suboptimal & unmaintained) …

python bindings

Successfully getting my work on board with it (which means having a perfect golang integration)

Getting more bindings for things I use

better docs; bindings for the JVM languages

Bindings on other languages (hard to get people to contribute on a project using the Haskell impl)

Better Python support, so I can sell it to colleagues who use Python.

Better (documented) language bindings.

Packages/Integrations:

I’d love to use it in more places, eg to replace all of our terraform code or CI configuration; currently those integrations just aren’t there and I don’t have time to bridge the gap

More libraries / packages. I think dhall needs a richer ecosystem. In particular i’d love complete terraform bindings and more kubernetes packages (a la helm)

First class Kubernetes, Terraform, Vault, Ansible integration.

When I looked at Dhall most recently, it wasn’t obvious to me that an ecosystem of packages was springing up around it. Might want to add a link (or a more prominent one if there is already one and I missed it).

More packages (like dhall-kubernetes)

… Also, better Terraform/HCL bindings, Ansible integration, or integrations for common CI systems (Jenkins/jenkins-job-builder, CircleCI, GoCD, etc)

… more integrations (I would like to be able to configure EVERYTHING using Dhall ^^)

Tooling:

Better performance

better (especially more concise) Error messages

structural editor with auto-completion based on symbols in scope and with automatic let-floating

Speed and ergonomics of the Emacs integration. Currently it’s terribly slow to type check.

Language features:

Ability to import YAML without yaml-to-dhall

nested record updates

Usage unicode chars in imported filenames without quotes

… Also Text/Double operations and possibly comparisons. I understand and agree with the reasons this hasn’t been done, but finding a way to do this without compromising the goal of the language would add so much potential

Formatting improvements:

Vonderhaar-style dhall format. Long multi-line literals with interpolations become really illegible currently. More built-ins. Text becoming non-opaque. The ability to expose multiple things from a record (this is currently being discussed).

Ultimately it was the autoformatter/idomatic formatting of Dhall that turned me off. I wanted my package format to be clear and readable to people new to my project but the idomatic was very messy in my eyes. Here’s a comparison of Dhall and the TCL inspired pkg definition format I came up with: https://gist.github.com/wezm/dfdce829964c410e2c521aa3ca132ddd

Social proof:

Popularity

Popularity is the big one so I could get away with it more at work. …

Documentation:

… Better docs to educate team members with.

Better starter documentation. maybe also how to put in place Dhall build in CI?

Other unassorted responses:

Hard to describe in one line, but an easier way to get values in and out of larger dhall projects (github issue 676 being a symptom of this)

For work maybe a convincing reason to use it in place of yaml. For personal, making it easier to support backward compatiblity in types.

The different type hack for multiple resources in a single Kubernetes file made me drop it, I could’ve never recommended it to my coworkers over worse (as in, worse is better) templating solutions.

Issue #1521

Generation of Dhall types from Haskell types; Easier extension with own functions

A nice bidirectional typechecker for less type annotation burden

Contributions

One thing I checked is if there were any barriers to adoption that we were not aware of:

Would anything encourage you to contribute to the Dhall ecosystem more often?

There were not many common themes that stood out, so I’ll include the responses verbatim:

Maybe some links to resources on how to get started with programming theory (eg. info on type notation for the standard, parser, “compiler”, etc). Basically I feel I probably just need to learn more, but I’m not entirely sure what.

Nope, as a new contributor this year, the Dhall community has been an absolute delight to start contributing to.

better reporting for missing env vars (not one by one)

better personal usecases

No - I already want to do a lot more!

A contributing guide to the haskell implementation

Linked Haskell tutorials and perhaps partial (or early) implementations of dhall features

Perhaps simplified core language? I sometimes think that the language is large and difficult, especially around “safety guarantees” features, and it might make it harder to develop a new language binding.

Using it at work

I found it difficult to generate dhall-kubernetes bindings

The ecosystem seems to be very contributor-friendly, but I don’t have enough time at the moment.

Getting standards for repo layout, documentation, discovery

… and my favorite response was:

There’s a ‘good first issue’ label that’s attached to no issues.

Conclusion

The main thing I concluded from the survey feedback is that people want us to focus on ease of integration, especially language bindings, and especially Python language bindings.

A major change from last year’s feedback was the dramatic drop in requests for better documentation / examples / use cases. People appear to understand the motivation for the language and how to use Dhall to solve their problems and now they care more about streamlining the integration process as much as possible.

If this is your first time hearing about the survey you can still complete the survey:

I receive e-mail notifications when people complete the survey so I will be aware of any feedback you provide this way. Also, completing the survey will let you browse the survey results more easily.

Before concluding this post, I would like to highlight that this year’s survey used approval voting to let people select their preferred language bindings. If you’re a person frustrated with a political system based on first-past-the-post voting, I encourage you to research approval voting as an alternative voting method with a higher power-to-weight ratio (simpler than ranked choice voting and produces better outcomes).

Posted by at No comments:
Subscribe to: Posts (Atom)