Help on recursion needed (nested schema records)

#1

Hi,

in our data lake application we use dhall to describe “data objects” (basically like a table known from relational database consisting of a name and column/field specifications).

We already use dhall for it but the current implementation has several drawbacks. One was the way we implemented recursion to describe nested columns/fields (fields whose type allows to store fields again).

We implemented that by replicating the type defintion of “Field” to “Field2”, “Field3” (to support at least nesting to level 3.).

I read the description on how to implement recursion in dhall but I honestly do not get it. Maybe you can help me with the following example.

Note that I want to use schema records (to allow defaults and also to simply add type annotations for deserialzation) and “compile” the definition to json at the end (in case that is relevant for the solution).

Let’s assume our definition can only have fields/columns of type bool or struct. struct allows arbritarily deep nesting. Since I use “other” names to emulate the nesting, I wrote it for only one level of nesting. I can add more but that should be fine for this example

-- a field can have a a bool type
let BoolType = { Type = { __typeInfo : Text }, default.__typeInfo = "BoolType" }

-- this is the 2nd level for the nesting
-- a field on level 2 can only be a bool, here we omit the Struct since we do not have defined level3
let FieldDataType2 = < Bool : BoolType.Type  >

-- thats a field on level 2
let Field2 =
      { Type = { name : Text, type : FieldDataType2, __typeInfo : Text }
      , default.__typeInfo = "Field2"
      }

-- this is dfinition of fields for the root/first level
-- herw we have fields of type struct that can contain a list of fields for level2
let StructType =
      { Type = { fields : List Field2.Type, __typeInfo : Text }
      , default.__typeInfo = "StructType"
      }
-- the field type can be bool or struct
let FieldDataType = < Bool : BoolType.Type | Struct : StructType.Type >
-- thats a field of the root/first level
let Field =
      { Type = { name : Text, type : FieldDataType, __typeInfo : Text }
      , default.__typeInfo = "Field"
      }

-- an example: level1 consists of col1 and col2, col2 contains two columns col2.1 and col2.2
let schemaNested1 =
      [ Field::{ name = "col1", type = FieldDataType.Bool BoolType::{=} }
      , Field::{
        , name = "col2"
        , type = FieldDataType.Struct StructType::{
                fields = [
                      Field2::{ name = "col2.1", type = FieldDataType2.Bool BoolType::{=} }
                    , Field2::{ name = "col2.2", type = FieldDataType2.Bool BoolType::{=} }
                ] }
        }
      ]

in  schemaNested1

So how would I have to change this example to support the nested columns and do not have to add Field2 etc. ?

Many thanks in advance, Christian