Update to Tutorial D, July 2020
Quote from dandl on July 16, 2020, 12:52 amQuote from Dave Voorhis on July 10, 2020, 6:19 pmQuote from dandl on July 10, 2020, 3:20 pmWhat purpose does WITH serve, exactly? It looks like it just declares and initialises a variable local to a block, so how is it different from VAR?
It declares a set of constants in a well-defined and limited scope. I think it's the only way to declare constants in Tutorial D.
Thank you, that makes perfect sense.
In addition to the "expression" form, there used to be a "statement" form of WITH that allowed things like this:
I don't recall why the statement form of WITH was considered "ill-advised" and deprecated/removed. Hugh?
I have to say I balk at introducing WITH into UPDATE/INSERT/EXTEND but not as the general statement construct that it used to be (which I liked!)
But if there's a good justification for it not being a general statement construct and only putting it in UPDATE and INSERT (and EXTEND, I guess) then I'm happy to implement it in Rel.
So the current WITH establishes a new scope in which to evaluate an expression, allowing the expression to be decomposed into a series of steps, mainly (I assume) for improved readability, but also allowing one to avoid evaluating the same sub-expression multiple times.
Yes, and improved writability, to avoid having write the same sub-expression multiple times.
PL/I and Pascal both had somewhat similar constructs (syntax now long forgotten and I can't be bothered to look) and it's distantly similar to C#'s using construct but without the IDisposable semantics, obviously.
Each WITH can define multiple constant values separated by commas. And each inner expression could itself contain further WITH constructs, allowing arbitrarily nested local scopes. And potentially the reuse of names in inner scopes conflicting with those in outer scopes ('shadowing' as discussed previously). All left to the implementer to decide.
Yes, unless specified away, of course. "No existing identifier, including those introduced by an outer WITH or an earlier term in the same WITH, may be redefined by WITH," for example.
Extending WITH to a general statement construct seems consistent with the general philosophy of block-structured languages. It seems to do no harm, and it allows expression decomposition and de-duplication to be extended across more than one assignment and/or update operator. Seems a good thing to me.
Yes, to me as well.
Extending WITH to updates (which means UPDATE, INSERT and DELETE, for completeness) has the same effect: allowing decomposition and de-duplication across the parts of an update (aka assignment). Possibly the same ideas extend to SUMMARIZE, EXTEND and other shorthands, but these are just expressions anyway, so weren't they already covered?
There seem to be 3 cases to consider:
- Allow WITH for any expression, as part of the computation of a single value
- Allow WITH for any single statement, as part of a single update/assignment
- Allow WITH for any statement or block, as a general capability of the language.
I would find it hard to stop short of No 3. You have other extensions in your implementation, why not this one? As long as you provide full syntactic compliance with what is defined, extras are yours to choose.
Yes. And there's a fourth case, which is the one suggested in this thread: 4. Allow WITH inside certain single statements, but only as an extension within their syntax in a manner that suggests it could have been generalised to any statement or block, but for reasons unspecified, wasn't.
Quote from dandl on July 16, 2020, 1:00 amLooking back through my lengthy correspondence with Chris on this subject, I find it hard to pin down exactly where WITH on statements is flawed. I know that we were worried about its use on multiple assignments where two or more assigns have the same target. I asked Chris if he could give a good answer to Dave's question but all he could come up with was:
"WITH (...) : WITH (...) : ... WITH (...) : nonwith stmt body ;
"I'm not sure how to define the semantics of such a construct. (Note that the WITH clauses aren't nested but are all at the same level.) "
I can add that in my case I was encouraged by Rel's current failure to support WITH on statements, guessing that Dave must have run nto implementation problems.
No, it was removed only because the Tutorial D specification removed it. I'll have to check where/when, as I don't recall.
When it was in Rel, it worked fine. There were no implementation problems that I recall. I simply treated a statement like
WITH (A :=x) : WITH (B := A + 2) : WITH (C :=z) : nonwith stmt body ;as being equivalent toWITH (A :=x,B := A + 2, C :=z) : nonwith stmt body ;which (if I recall correctly) was precisely equivalent to a hypothetical...
CONST A :=x;
CONST B := A + 2;
CONST C := z;
nonwith stmt body;...for which it should be straightforward to either retain those semantics -- the constant definitions occur sequentially -- or treat it (the same) as multiple assignment.
There is a difference.
- WITH(a,b,c) looks like a single scope
- WITH(a):WITH(b):WITH(c): looks like nested scopes but might not be
- WITH(a): x+WITH(b): y has to be nested, surely?
If WITH (a,b,c) is defined as simple left-to-right precedence, then whether the scopes are nested or not is equivalent. Indeed, WITH(a,b,c) can simply be defined to be shorthand for WITH(a): WITH(b): WITH(c): ...
Quote from AntC on July 16, 2020, 5:10 amAlso I've not bothered with Dave's
THE_QTY(QTY)business -- another bit of excessive circumlocution IMO.
It's an intentional one for illustrating how types work; no numeric or aggregate operators have been defined for type QTY, though they can be. Doing so is an exercise for the computer science student. Discussing whether or not they should be is an exercise for the information systems student.
But not relevant here.
Quote from AntC on July 16, 2020, 6:17 amQuote from Hugh on July 9, 2020, 11:22 amHere's a response from Chris Date:
Here's a trivial example of the kind of thing I had in mind:
EXTEND S :
{ WITH ( temp := ( SP MATCHING S ) WHERE QTY > 500 AND NOT ( PNO = 'P1' ) ) :
A := SUM ( temp , QTY ) ,
B := MAX ( temp , QTY ) ,
C := MIN ( temp , QTY ) ,
D := COUNT ( temp ) }
Obviously the definition of temp could be arbitrarily complex in practice.
Obviously, temp is in scope for each of the assigns, overriding any existing definition under normal block-scoping rules.
Ah, here's a maybe better example/reworking of Chris's:
EXTEND S :{ WITH ( temp := ( SP MATCHING REL{ TUP{ S# S# } } ) WHERE QTY > 500 AND NOT ( PNO = 'P1' ) ) :A := SUM ( temp , QTY ) ,B := MAX ( temp , QTY ) ,C := MIN ( temp , QTY ) ,D := COUNT ( temp ) }EXTEND S : { WITH ( temp := ( SP MATCHING REL{ TUP{ S# S# } } ) WHERE QTY > 500 AND NOT ( PNO = 'P1' ) ) : A := SUM ( temp , QTY ) , B := MAX ( temp , QTY ) , C := MIN ( temp , QTY ) , D := COUNT ( temp ) }[snip]
( SP MATCHING REL{ TUP{ S# S# } } ) isn't a valid expression. The second appearance of S# is undefined (unless a variable of that name is in scope). You seem to be imagining that the second operand of a dyadic relational operator invocation can reference attributes of the first (a mistake I have made myself before now). Try putting those operands the other way around if you don't see what I mean.
Hugh
Somewhere in the plethora of recent posts in this thread Dave Voorhis said that he removed WITH support on statements in Rel when we deleted it from Tutorial D. But that deletion took place the day I started this thread and I had previously found that Rel lacked it. That's why I said I had been encouraged by Rel's lack of support for it.
Hugh
Quote from Hugh on July 16, 2020, 11:33 amQuote from AntC on July 16, 2020, 6:17 amQuote from Hugh on July 9, 2020, 11:22 amHere's a response from Chris Date:
Here's a trivial example of the kind of thing I had in mind:
EXTEND S :
{ WITH ( temp := ( SP MATCHING S ) WHERE QTY > 500 AND NOT ( PNO = 'P1' ) ) :
A := SUM ( temp , QTY ) ,
B := MAX ( temp , QTY ) ,
C := MIN ( temp , QTY ) ,
D := COUNT ( temp ) }
Obviously the definition of temp could be arbitrarily complex in practice.
Obviously, temp is in scope for each of the assigns, overriding any existing definition under normal block-scoping rules.
Ah, here's a maybe better example/reworking of Chris's:
EXTEND S :{ WITH ( temp := ( SP MATCHING REL{ TUP{ S# S# } } ) WHERE QTY > 500 AND NOT ( PNO = 'P1' ) ) :A := SUM ( temp , QTY ) ,B := MAX ( temp , QTY ) ,C := MIN ( temp , QTY ) ,D := COUNT ( temp ) }EXTEND S : { WITH ( temp := ( SP MATCHING REL{ TUP{ S# S# } } ) WHERE QTY > 500 AND NOT ( PNO = 'P1' ) ) : A := SUM ( temp , QTY ) , B := MAX ( temp , QTY ) , C := MIN ( temp , QTY ) , D := COUNT ( temp ) }[snip]( SP MATCHING REL{ TUP{ S# S# } } ) isn't a valid expression. The second appearance of S# is undefined (unless a variable of that name is in scope). You seem to be imagining that the second operand of a dyadic relational operator invocation can reference attributes of the first (a mistake I have made myself before now). Try putting those operands the other way around if you don't see what I mean.
> You seem to be imagining that the second operand of a dyadic relational operator invocation can reference attributes of the first
Thanks Hugh but no, that wouldn't have achieved the logic I was trying for. (What seems to you would have been an identity operation on SP.) I want/expect the free S# to denote the attribute name in the outer S. I expected (didn't imagine) that in an 'open expression' within the body of EXTEND S, free names (i.e. S#) would get bound to attribute names introduced by the target of the EXTEND. That is, even if there were a variable of that name in scope, it would get shadowed by the attribute names introduced via S. Similarly in SUM( temp, QTY), apparently, Tutorial D expects that free names in the exp (second argument to SUM, and note that it can be an exp, not just an <attribute ref>) bind to attribute names introduced by the relexp (first argument).
The r.h. operand of MATCHING (or other dyadic relops) isn't bound by attribute names from the l.h. operand. If that MATCHING were a JOIN, I'd expect I could happily put those operands the other way round.
OK If what I showed doesn't work, how do I achieve the intent of 'recalculating' temp for each tuple in S that's getting EXTENDed? What Dave shows is perhaps the only way? That is
EXTEND S :
{ temp := ( SP MATCHING REL{ TUP{ S# S# } } ) WHERE QTY > 500 AND NOT ( PNO = 'P1' ) ,
A := SUM ( temp , QTY ) ,
B := MAX ( temp , QTY ) ,
C := MIN ( temp , QTY ) ,
D := COUNT ( temp )
} {ALL BUT temp}
Or will that not work either? Why not?
Where does the Tutorial D spec tell us which syntactic constructs introduce attribute names into scope; and what is the textual extent of that scope in the 'body' of the construct?
Quote from Hugh on July 16, 2020, 11:33 amQuote from AntC on July 16, 2020, 6:17 amQuote from Hugh on July 9, 2020, 11:22 amHere's a response from Chris Date:
Here's a trivial example of the kind of thing I had in mind:
EXTEND S :
{ WITH ( temp := ( SP MATCHING S ) WHERE QTY > 500 AND NOT ( PNO = 'P1' ) ) :
A := SUM ( temp , QTY ) ,
B := MAX ( temp , QTY ) ,
C := MIN ( temp , QTY ) ,
D := COUNT ( temp ) }
Obviously the definition of temp could be arbitrarily complex in practice.
Obviously, temp is in scope for each of the assigns, overriding any existing definition under normal block-scoping rules.
Ah, here's a maybe better example/reworking of Chris's:
EXTEND S :{ WITH ( temp := ( SP MATCHING REL{ TUP{ S# S# } } ) WHERE QTY > 500 AND NOT ( PNO = 'P1' ) ) :A := SUM ( temp , QTY ) ,B := MAX ( temp , QTY ) ,C := MIN ( temp , QTY ) ,D := COUNT ( temp ) }EXTEND S : { WITH ( temp := ( SP MATCHING REL{ TUP{ S# S# } } ) WHERE QTY > 500 AND NOT ( PNO = 'P1' ) ) : A := SUM ( temp , QTY ) , B := MAX ( temp , QTY ) , C := MIN ( temp , QTY ) , D := COUNT ( temp ) }[snip]( SP MATCHING REL{ TUP{ S# S# } } ) isn't a valid expression.
> ... isn't a valid expression
Isn't it? Then please explain an example which looks very similar, in the doco 'Tutorial D 2016-09-22' page 27. It's given as the expansion of SUMMARIZE r PER ( p ) : { X := SUM ( XX ) }:
EXTEND ( p ) : { X := SUM ( ( ( r ) MATCHING
RELATION { TUPLE { B1 B1, B2 B2, ..., Bm Bm } } )
{ ALL BUT B1, B2, ... Bm }, XX }
[sic I've copied that expression as faithfully as my old eyes can tell, but it seems to have unbalanced parens. I suspect it needs a closing ) between XX and the closing }.]
B1, B2, ..., Bm are attribute names from relexp p. That's exactly like my example taking attribute name S# from relvar S, the one getting EXTENDed.
Where does the Tutorial D spec tell us which syntactic constructs introduce attribute names into scope; and what is the textual extent of that scope in the 'body' of the construct?
It doesn't, per quote from the spec:
For example, details of the following are all omitted:
Language characters, identifiers, scope of names, etc.
There are many subtle issues around scopes (block, operator and WITH) and the interaction with (possibly nested) open expressions, which would be documented by the implementer. In the case of Rel, they are not.
Quote from Hugh on July 16, 2020, 11:56 amSomewhere in the plethora of recent posts in this thread Dave Voorhis said that he removed WITH support on statements in Rel when we deleted it from Tutorial D. But that deletion took place the day I started this thread and I had previously found that Rel lacked it. That's why I said I had been encouraged by Rel's lack of support for it.
I beg to differ. It's at the top of page 122 of my paper copy of DTATRM, and roughly the middle of page 127 in DTATRM.pdf:
- The <with> statement has been dropped and WITH expressions have been clarified.
I remember being a tad disappointed, and there might have been discussion about it on the forum at the time, but I removed it in accordance with DTATRM.
Quote from dandl on July 16, 2020, 1:19 pmWhere does the Tutorial D spec tell us which syntactic constructs introduce attribute names into scope; and what is the textual extent of that scope in the 'body' of the construct?
It doesn't, per quote from the spec:
For example, details of the following are all omitted:
Language characters, identifiers, scope of names, etc.There are many subtle issues around scopes (block, operator and WITH) and the interaction with (possibly nested) open expressions, which would be documented by the implementer. In the case of Rel, they are not.
It's open source. Feel free to document them. :-)