Quote from Erwin on June 12, 2019, 9:42 pm

Quote from johnwcowan on June 12, 2019, 3:19 pm

Quote from Erwin on June 12, 2019, 1:50 pm

The flattening to URL syntax creates trouble where we would use parentheses to control nesting in more common syntax versions :

JOIN ( MINUS (A,B) , MINUS(C,D) )   <--->   join/minus/a/b/minus/c/d            ??????

(Note that join being associative we would surely like to have join/bar/baz/baq/bat !!!)

 

Yes, that's the translation I am proposing.  Since you know that (in this syntax) join and minus take exactly two arguments, you can reconstruct precisely where the parens go.  This is Polish notation, which requires all operators to have a fixed number of arguments.

The consequence is that your second example comes out join/bar/join/baz/join/baq/bat (or any number of other ways, like join/join/join/bar/baz/baq/bat).  Of course this is not a syntax you'd normally want to construct by hand, but it's easy to do mechanically given Tutorial D syntax or a tree representation.

RESTRICT and similar (involving scalar expressions which could in some sense be regarded as a free lambda-type parameter) are also going to be funny ones.

RESTRICT ( R , EQ ( MYFOOBAR ( A1, A2, A5 ) , STRING(%) ) )

or in fact

RESTRICT ( R , <really_just_any_technically_valid_wff_no_matter_how_crazy> )

Quote from Erwin on June 12, 2019, 9:58 pm

Quote from Erwin on June 12, 2019, 9:42 pm

Quote from johnwcowan on June 12, 2019, 3:19 pm

Quote from Erwin on June 12, 2019, 1:50 pm

The flattening to URL syntax creates trouble where we would use parentheses to control nesting in more common syntax versions :

JOIN ( MINUS (A,B) , MINUS(C,D) )   <--->   join/minus/a/b/minus/c/d            ??????

(Note that join being associative we would surely like to have join/bar/baz/baq/bat !!!)

 

Yes, that's the translation I am proposing.  Since you know that (in this syntax) join and minus take exactly two arguments, you can reconstruct precisely where the parens go.  This is Polish notation, which requires all operators to have a fixed number of arguments.

The consequence is that your second example comes out join/bar/join/baz/join/baq/bat (or any number of other ways, like join/join/join/bar/baz/baq/bat).  Of course this is not a syntax you'd normally want to construct by hand, but it's easy to do mechanically given Tutorial D syntax or a tree representation.

RESTRICT and similar (involving scalar expressions which could in some sense be regarded as a free lambda-type parameter) are also going to be funny ones.

RESTRICT ( R , EQ ( MYFOOBAR ( A1, A2, A5 ) , STRING(%) ) )

or in fact

RESTRICT ( R , <really_just_any_technically_valid_wff_no_matter_how_crazy> )

Also note that modern languages usually allow varargs so MYFOOBAR (A1, A2, A5) could be SUM(A1, A2, A5) just as well as SUM (JAN,FEB,MAR,APR,MAY,JUN) which has a nice resemblance to associative joins and unions which you apparently want to rule out.  Rule out scalar vararg functions too ?  If not, then why do so for the nonscalar vararg functions ?  Because the scenario doesn't fit the chosen solution ?

(PS I vaguely remember doing my bit of work devising my own syntax for these things.  Pls regard my comments as challenges not criticisms.)

Quote from johnwcowan on June 12, 2019, 10:13 pm

Quote from Erwin on June 12, 2019, 9:58 pm

Also note that modern languages usually allow varargs so MYFOOBAR (A1, A2, A5) could be SUM(A1, A2, A5) just as well as SUM (JAN,FEB,MAR,APR,MAY,JUN) which has a nice resemblance to associative joins and unions which you apparently want to rule out.  Rule out scalar vararg functions too ?  If not, then why do so for the nonscalar vararg functions ?  Because the scenario doesn't fit the chosen solution ?

Yes, I'd say you'd have to rule out all vararg functions, scalar and non-scalar.  Alternatively you could simply know which operators are vararg and do something like /myfoobar/a1/a2/a5/!, where ! indicates the end of a vararg construction.

In a broader sense, however, the limiting factor is not processing power on the client, but bandwidth (and to some degree processing power on the server).  There is little efficiency gain (and may be loss) in doing unions on the server, for example.  What you really need is project/remove, a modest amount of restrict, and possibly the various joins.

Quote from Erwin on June 13, 2019, 7:31 am

Quote from johnwcowan on June 12, 2019, 10:13 pm

There is little efficiency gain (and may be loss) in doing unions on the server, for example.  What you really need is project/remove, a modest amount of restrict, and possibly the various joins.

That is pretty shortsighted.  Any expression can reasonably appear as the second arg of a MINUS or NOT MATCHING.  E.g. you can have

R1 NOT MATCHING (R2 UNION R3)

There often probably isn't even any "doing the union" involved and if you were to claim "there is little efficiency gain doing the NOTMATCHING on the server", euhm well, trying to stay polite, not the brightest idea you ever had.

Quote from dandl on June 13, 2019, 8:52 am

Quote from johnwcowan on June 12, 2019, 10:13 pm

In a broader sense, however, the limiting factor is not processing power on the client, but bandwidth (and to some degree processing power on the server).  There is little efficiency gain (and may be loss) in doing unions on the server, for example.  What you really need is project/remove, a modest amount of restrict, and possibly the various joins.

The limiting factor is typically network bandwidth, I/O bandwidth, memory, processing power in that order.There is also a modest benefit for in-order row processing rather than indexed.

The typical evaluation structure for a relational expression is a pipeline, which aims to minimise resource usage in that order. There are enormous benefits if the pipeline can terminate early, which is the case for EXISTS(), SKIP/TAKE paging and some restricts. On really big datasets, memory usage is often the limiting factor.

Most queries tend to produce less output rows than input, so network bandwidth dominates and it would be crazy to ship raw data to the client. COUNT(UNION(r1,r2)) would be an extreme example.

Your proposition can only be true in the reverse situation, where there are as many or more output rows than input. The prime example of that is a cross join. For everything else, keeping it on the server is a good bet.