$foreach and $exists functions

[lauwers]

Calin has previously suggested adding $foreach and $exists functions that would apply boolean expressions to lists of nodes or relationships. $foreach evaluates to true if the boolean expression evaluates to true for all elements in the list (boolean and) and $exists evaluates to true if the boolean expression evaluates to true for at least one of the elements in the list (boolean or). I believe Calin's suggested syntax was as follows:

$foreach: [boolean_comp_name, list1, list2]
$foreach: [boolean_comp_name, list1, value]
$exists: [boolean_comp_name, list1, list2]
$exists: [boolean_comp_name, list1, value]

Meaning that for any combination of list1 and list2 elements the applied boolean_comp is true.

I believe this syntax can be simplified as follows:

$foreach: [ <list of nodes>, <boolean expression> ]
$exists: [ [ <list of nodes>, <boolean expression> ]

I believe there is no need to specify two list arguments, since the boolean expression can decide which arguments to take. Of course, if one of the arguments of the boolean expression is a list, then we may need a second $foreach function that iterates over the second list. In that case, we need to be able to identify which element in which list is being used, which may require a variable name. Using a variable name, the syntax would look as follows:

$foreach: [ <list of nodes>, <variable name>, <boolean expression> ]
$exists: [ [ <list of nodes>, <variable name>, <boolean expression> ]

The boolean expression could then use that variable name as appropriate.

With this syntax, we could then support formal definitions of declarative policies. For example, the following would express a colocation policy for all nodes in a colocation group named colo-group:

$foreach: 
- colo-group
- X
- $foreach: 
  - colo-group
  - Y
  - $equal:
    - [ X, RELATIONSHIP, host, TARGET, uuid ]
    - [ Y, RELATIONSHIP, host, TARGET, uuid ]

[pmbruun]

I would prefer $all or $forall, since $foreach would normally be a distribution function.

This amounts, in the general case, to TOSCA supporting lambdas, and there are two different functionalities that you may want to achieve: distribution and aggregation.

• Both $foreach and $exists are examples of aggregation, using AND and OR respectively. You could just as well want an aggregating SUM.
• Alternatively, you would could want distribution - a function that applies the expression to each element in a list and returns the results in a new list.

Consider now that functions $and, $or, $sum, etc. applied to a list of booleans/numbers automatically aggregate. So @and: <list-expression>* returns the conjunction of the elements of the lists (potentially skipping any null elements?), and @or: <list-expression>* returns the disjunction of the list elements.

Now if we define $each: [ <list of nodes>, <variable name>, <expression> ] as evaluating the expression for each element of the list expanded with the variable name into the expression, then you can express $forall as:

$and:
- $each: 
  - colo-group
  - X
  - $foreach: 
    - colo-group
    - Y
    - $equal:
      - [ X, RELATIONSHIP, host, TARGET, uuid ]
      - [ Y, RELATIONSHIP, host, TARGET, uuid ]

You may of course still have shorthand functions $forall and $exists for this.

[lauwers]

Yes, I believe we want aggregation semantics, not distribution.
By the way, Calin raised a concern about the fact that the proposal above uses the same function syntax for defining function values (i.e. values that are obtained by evaluating a function) and for function arguments ("function pointers") to the $forall and $exists functions. Any opinions on that?

[koppor]

Back in the days of OASIS TOSCA 1.0, one could use one's own expression language:

<Precondition expressionLanguage="xs:anyURI">
  condition
</Precondition>

(That came from WS-BPEL)

Back in those days, one could use XPath or Java. For XPath, the concrete expression was (more or less) clear; for Java, one needed another specification). So, why not offer a Python / go / ... expression language plug in possibility?

In another project, we made good experiences with jq, although the syntax is not as nice as Python.