Idea for "dhall text" tool: accept arguments, including shell commands as (Text -> Text)

#1

Some context for this idea – I’m currently intending to use dhall as a templating language with multiple outputs – latex and HTML. I already have a system in place, but I’d like to migrate things to dhall fully.
The actual raw copy I intend on writing in markdown, and in my current non-dhall system they are being translated into html or latex appropriately.

This markdown-to-html or markdown-to-latex process is the current thing that is makes things a little painful. markdown-to-html I can sort of fix by loading a client-side js markdown library, which works but is unideal. markdown-to-latex my current solution is to use dhall to generate a yaml file, and then use pandoc’s templating on that yaml file, but I’d really prefer to not have to require any templating system outside of dhall.

Ideally I’d like something that only needs to leverage the dhall shell tool and other shell scripts, and not have to spin up an entire programming language to write something, otherwise there wouldn’t be much gain from what I am currently doing.

So, here’s my idea! Currently, dhall text only renders Text as far as I know. But, we could maybe extend it to rendering Text -> Text, as well, by calling as dhall --textarg "hello", which feeds in a string at runtime, applies it to the function, and renders the resulting final Text. And, what if it could also render (Text -> Text) -> Text functions, as well, by calling as dhall --shellarg "pandoc -f markdown -t html". Then in the dhall expression \(f : Text -> Text) -> .., it would invoke the shell function in stdin-stdout mode to get that Text -> Text, so I could write a dhall expression like \(renderMarkdown : Text -> Text) -> .. and have a markdown rendering function. It would be assumed (or gently suggested) that the shell command given must be pure (return the same stdout for every stdin, and change nothing major about its state on every invocation) for this to make sense.

I could probably dive in to implementing this myself. Dhall evaluation resolution is already in IO iirc, so I think this could work.

Let me know if there is a better way to achieve this, though! Or if this feature is too heavy or convoluted to be worth including into the dhall tool, and I can maybe try to figure out other workarounds or stick with the ones I already mentioned :slight_smile:

1 Like
#2

whoops, looks like I’ve opened a whole can of worms that has been heavily discussed already :sweat_smile:
ie Proposal: "forward declarations" for custom built-ins · Issue #800 · dhall-lang/dhall-lang · GitHub and Permit text parsing at import resolution time? · Issue #989 · dhall-lang/dhall-lang · GitHub . I guess this is just one idea out of many for solving the same problem. Hopefully I am not adding too much to the noise :slight_smile:

1 Like
#3

Yeah, I was just about to comment that I think this would be better to address within the language by defining a template as a function and then supplying function arguments to instantiate template parameters

#4

By “the language” do you mean dhall, or a host language that would be loading dhall as a library?

#5

I mean Dhall, that way you could do something like:

$ yourTool --expr './template.dhall { param0 = …, param1 = … }'
#6

I was playing around with adding this to the dhall tool and it seems to be easier to implement than I thought! It only required adding a dependency on process, and very minor changes to the executable code itself! Is it “simple”/universal enough to merit bringing in? PR: Comparing dhall-lang:master...mstksg:text-with-args · dhall-lang/dhall-haskell · GitHub

-- testfile.dhall
let text = https://raw.githubusercontent.com/dhall-lang/dhall-lang/v21.1.0/Prelude/Text/package.dhall
in  \(f : Text -> Text) -> text.concatMapSep "," Text f [ "hello", "world" ]

And running with the dhall above:

$ dhall text --file testfile.dhall --argCmd cat
hello,world
$ dhall text --file testfile.dhall --argCmd "tr '[:lower:]' '[:upper:]'"
HELLO,WORLD
$ dhall text --file testfile.dhall --argCmd "pandoc -f markdown -t html"
<p>hello</p>
,<p>world</p>
$ dhall text --file testfile.dhall --argCmd "md5sum -z"
5d41402abc4b2a76b9719d911017c592  -,7d793037a0760186574b0282f2f435e7  -

Which works exactly how I was expecting :slight_smile:

Is this maybe too heavyweight of a thing to bring into the main cli tool?

edit: the current implementation has some bugs when taking more than one argument, but they seem fixable :slight_smile:

#7

I think maybe it doesn’t belong in the main binary, but i made a separate standalone tool :slight_smile: