Quote from dandl on May 26, 2020, 1:47 pm

Quote from Dave Voorhis on May 26, 2020, 8:58 am

Quote from dandl on May 26, 2020, 4:05 am

Quote from Dave Voorhis on May 25, 2020, 3:39 pm

Quote from dandl on May 25, 2020, 3:06 pm

Quote from Dave Voorhis on May 25, 2020, 10:25 am

Quote from dandl on May 25, 2020, 6:37 am

Quote from Darren Duncan on May 25, 2020, 6:23 am

Quote from dandl on May 23, 2020, 9:38 am

A function relation is a relation that can appear in a relational expression as an argument to a relational operator. The declaration of an FR requires only a heading. The definition is supplied by an implementation written in some other language.

Can you give some example code for this proposal?

I'm still working on it, but something like this at present:

var si = RelationNode.Import(SourceKind.Csv, ".", "S", "SNo:text,SName:text,Status:integer,City:text");

var ss = si.Select("City", new TupSelect(t => (string)t[0] == "Paris"));

var si = RelationNode.Import(SourceKind.Csv, ".", "S", "SNo:text,SName:text,Status:integer,City:text"); var ss = si.Select("City", new TupSelect(t => (string)t[0] == "Paris"));

var si = RelationNode.Import(SourceKind.Csv, ".", "S", "SNo:text,SName:text,Status:integer,City:text");
var ss = si.Select("City", new TupSelect(t => (string)t[0] == "Paris"));

The key point is that a function relation defines a heading and a conforming function. From the implementation language point of view any object implementing the required interface will do. The function code can be inline, user-defined or system, of arbitrary complexity, and is the only part that has access to the scalar type system.

How does your "function relation" differ from a (read-only?) relvar?

It's never a relvar. It does look like a relation value when used as an argument to a suitable function, but there are restrictions -- it doesn't work backwards.

Rel reads (and in some cases, writes) CSV files, Excel spreadsheets, MS Access databases, and tables in SQL DBMSs. From the Tutorial D point of view, they're all relvars; their physical implementation is irrelevant and (largely -- it's specified in the relvar declaration) external to Tutorial D.

In short, I'm not clear what a "function relation" adds to the canon.

Those should certainly look like relation constants. I have those too. But you cannot use PLUS that way. You have to join in such a way as to provide the arguments from a which a result can be calculated. It's a mapping from arguments to return value. It's a function, not a constant. [algorithmic if it makes you happier, but not a constant.]

I'm not sure what you mean by "it doesn't work backwards", but are you sure RelationNode.Import(...) is the example you meant to use?

I'm surprised. These things look so obvious.

They look obvious because you wrote them. What is obvious to the writer is rarely as obvious to the reader.

A function maps from one set to another, from arguments to return value. It doesn't work backwards, from return value to arguments. A constant works both ways: a SINE constant could be looked up on the return value to get the arcsine.

The Import() node was included so you could see where the data came from, including the heading. The next line select is the relation function.

In the absence of explanation, I thought the Import() (node?) was the relation function.

Am I right that what you mean by a "function relation" is a function that returns a relation that is parametrised by subsequent terms in the expression?

For example, a RelationNode.Import with a SQL source might use Select to generate appropriate SQL for execution by an external DBMS, rather than restricting in-memory?

A relation function:

• is invoked by name and heading
• takes a relation as an argument, heading must conform
• returns a relation, heading is inferred.

An import relation function might have a heading like "sql,rva" and would import one table as an RVA per tuple with attribute SQL in its argument.

Is a "relation function" what TTM calls a "(read-only) relational operator"?

That term is defined by TTM RM Pre 3 and 20. While there are similarities, that definition is insufficiently powerful to allow generic operators, either those of the RA or those envisaged by RM VSS 6. The point of relation functions is to build on the insight of App-A and HHT to present a function (an algorithm) in a form similar to a relcon. For that to work all the relation operators have to be generic and configurable where needed by heading (exactly as per project and rename in TD). Each operator infers a result heading from its inputs (just like App-A), and that approach works across the board. Implementation code is open for enhancement or replacement.

So instead of the different mechanisms in RM Pre 3/18/20, OO Pre 6, VSS 4/5/6 and all the shorthands hard-coded into TD, there is a single mechanism that spans them all. I've done 16 operators, just a couple to go.

If I understood Ant's last post, he had already come up with a similar strategy, but with strong ties to Haskell. This approach will work just fine with any modern GP language.

Quote from Dave Voorhis on May 26, 2020, 2:09 pm

Quote from dandl on May 26, 2020, 1:47 pm

Quote from Dave Voorhis on May 26, 2020, 8:58 am

Quote from dandl on May 26, 2020, 4:05 am

Quote from Dave Voorhis on May 25, 2020, 3:39 pm

Quote from dandl on May 25, 2020, 3:06 pm

Quote from Dave Voorhis on May 25, 2020, 10:25 am

Quote from dandl on May 25, 2020, 6:37 am

Quote from Darren Duncan on May 25, 2020, 6:23 am

Quote from dandl on May 23, 2020, 9:38 am

A function relation is a relation that can appear in a relational expression as an argument to a relational operator. The declaration of an FR requires only a heading. The definition is supplied by an implementation written in some other language.

Can you give some example code for this proposal?

I'm still working on it, but something like this at present:

var si = RelationNode.Import(SourceKind.Csv, ".", "S", "SNo:text,SName:text,Status:integer,City:text");

var ss = si.Select("City", new TupSelect(t => (string)t[0] == "Paris"));

var si = RelationNode.Import(SourceKind.Csv, ".", "S", "SNo:text,SName:text,Status:integer,City:text"); var ss = si.Select("City", new TupSelect(t => (string)t[0] == "Paris"));

var si = RelationNode.Import(SourceKind.Csv, ".", "S", "SNo:text,SName:text,Status:integer,City:text");
var ss = si.Select("City", new TupSelect(t => (string)t[0] == "Paris"));

The key point is that a function relation defines a heading and a conforming function. From the implementation language point of view any object implementing the required interface will do. The function code can be inline, user-defined or system, of arbitrary complexity, and is the only part that has access to the scalar type system.

How does your "function relation" differ from a (read-only?) relvar?

It's never a relvar. It does look like a relation value when used as an argument to a suitable function, but there are restrictions -- it doesn't work backwards.

Rel reads (and in some cases, writes) CSV files, Excel spreadsheets, MS Access databases, and tables in SQL DBMSs. From the Tutorial D point of view, they're all relvars; their physical implementation is irrelevant and (largely -- it's specified in the relvar declaration) external to Tutorial D.

In short, I'm not clear what a "function relation" adds to the canon.

Those should certainly look like relation constants. I have those too. But you cannot use PLUS that way. You have to join in such a way as to provide the arguments from a which a result can be calculated. It's a mapping from arguments to return value. It's a function, not a constant. [algorithmic if it makes you happier, but not a constant.]

I'm not sure what you mean by "it doesn't work backwards", but are you sure RelationNode.Import(...) is the example you meant to use?

I'm surprised. These things look so obvious.

They look obvious because you wrote them. What is obvious to the writer is rarely as obvious to the reader.

A function maps from one set to another, from arguments to return value. It doesn't work backwards, from return value to arguments. A constant works both ways: a SINE constant could be looked up on the return value to get the arcsine.

The Import() node was included so you could see where the data came from, including the heading. The next line select is the relation function.

In the absence of explanation, I thought the Import() (node?) was the relation function.

Am I right that what you mean by a "function relation" is a function that returns a relation that is parametrised by subsequent terms in the expression?

For example, a RelationNode.Import with a SQL source might use Select to generate appropriate SQL for execution by an external DBMS, rather than restricting in-memory?

A relation function:

• is invoked by name and heading
• takes a relation as an argument, heading must conform
• returns a relation, heading is inferred.

An import relation function might have a heading like "sql,rva" and would import one table as an RVA per tuple with attribute SQL in its argument.

Is a "relation function" what TTM calls a "(read-only) relational operator"?

That term is defined by TTM RM Pre 3 and 20. While there are similarities, that definition is insufficiently powerful to allow generic operators, either those of the RA or those envisaged by RM VSS 6. The point of relation functions is to build on the insight of App-A and HHT to present a function (an algorithm) in a form similar to a relcon. For that to work all the relation operators have to be generic and configurable where needed by heading (exactly as per project and rename in TD). Each operator infers a result heading from its inputs (just like App-A), and that approach works across the board. Implementation code is open for enhancement or replacement.

So instead of the different mechanisms in RM Pre 3/18/20, OO Pre 6, VSS 4/5/6 and all the shorthands hard-coded into TD, there is a single mechanism that spans them all. I've done 16 operators, just a couple to go.

If I understood Ant's last post, he had already come up with a similar strategy, but with strong ties to Haskell. This approach will work just fine with any modern GP language.

Again, this must be obvious to you but I find it completely baffling. Sorry.

Quote from AntC on May 27, 2020, 1:30 am

Quote from Dave Voorhis on May 26, 2020, 2:09 pm

Quote from dandl on May 26, 2020, 1:47 pm

Quote from Dave Voorhis on May 26, 2020, 8:58 am

Quote from dandl on May 26, 2020, 4:05 am

Quote from Dave Voorhis on May 25, 2020, 3:39 pm

Quote from dandl on May 25, 2020, 3:06 pm

Quote from Dave Voorhis on May 25, 2020, 10:25 am

Quote from dandl on May 25, 2020, 6:37 am

Quote from Darren Duncan on May 25, 2020, 6:23 am

Quote from dandl on May 23, 2020, 9:38 am

A function relation is a relation that can appear in a relational expression as an argument to a relational operator. The declaration of an FR requires only a heading. The definition is supplied by an implementation written in some other language.

Can you give some example code for this proposal?

I'm still working on it, but something like this at present:

var si = RelationNode.Import(SourceKind.Csv, ".", "S", "SNo:text,SName:text,Status:integer,City:text");

var ss = si.Select("City", new TupSelect(t => (string)t[0] == "Paris"));

var si = RelationNode.Import(SourceKind.Csv, ".", "S", "SNo:text,SName:text,Status:integer,City:text"); var ss = si.Select("City", new TupSelect(t => (string)t[0] == "Paris"));

var si = RelationNode.Import(SourceKind.Csv, ".", "S", "SNo:text,SName:text,Status:integer,City:text");
var ss = si.Select("City", new TupSelect(t => (string)t[0] == "Paris"));

The key point is that a function relation defines a heading and a conforming function. From the implementation language point of view any object implementing the required interface will do. The function code can be inline, user-defined or system, of arbitrary complexity, and is the only part that has access to the scalar type system.

How does your "function relation" differ from a (read-only?) relvar?

It's never a relvar. It does look like a relation value when used as an argument to a suitable function, but there are restrictions -- it doesn't work backwards.

Rel reads (and in some cases, writes) CSV files, Excel spreadsheets, MS Access databases, and tables in SQL DBMSs. From the Tutorial D point of view, they're all relvars; their physical implementation is irrelevant and (largely -- it's specified in the relvar declaration) external to Tutorial D.

In short, I'm not clear what a "function relation" adds to the canon.

Those should certainly look like relation constants. I have those too. But you cannot use PLUS that way. You have to join in such a way as to provide the arguments from a which a result can be calculated. It's a mapping from arguments to return value. It's a function, not a constant. [algorithmic if it makes you happier, but not a constant.]

I'm not sure what you mean by "it doesn't work backwards", but are you sure RelationNode.Import(...) is the example you meant to use?

I'm surprised. These things look so obvious.

They look obvious because you wrote them. What is obvious to the writer is rarely as obvious to the reader.

A function maps from one set to another, from arguments to return value. It doesn't work backwards, from return value to arguments. A constant works both ways: a SINE constant could be looked up on the return value to get the arcsine.

The Import() node was included so you could see where the data came from, including the heading. The next line select is the relation function.

In the absence of explanation, I thought the Import() (node?) was the relation function.

Am I right that what you mean by a "function relation" is a function that returns a relation that is parametrised by subsequent terms in the expression?

For example, a RelationNode.Import with a SQL source might use Select to generate appropriate SQL for execution by an external DBMS, rather than restricting in-memory?

A relation function:

• is invoked by name and heading
• takes a relation as an argument, heading must conform
• returns a relation, heading is inferred.

An import relation function might have a heading like "sql,rva" and would import one table as an RVA per tuple with attribute SQL in its argument.

Is a "relation function" what TTM calls a "(read-only) relational operator"?

That term is defined by TTM RM Pre 3 and 20. While there are similarities, that definition is insufficiently powerful to allow generic operators, either those of the RA or those envisaged by RM VSS 6. The point of relation functions is to build on the insight of App-A and HHT to present a function (an algorithm) in a form similar to a relcon. For that to work all the relation operators have to be generic and configurable where needed by heading (exactly as per project and rename in TD). Each operator infers a result heading from its inputs (just like App-A), and that approach works across the board. Implementation code is open for enhancement or replacement.

So instead of the different mechanisms in RM Pre 3/18/20, OO Pre 6, VSS 4/5/6 and all the shorthands hard-coded into TD, there is a single mechanism that spans them all. I've done 16 operators, just a couple to go.

If I understood Ant's last post, he had already come up with a similar strategy, but with strong ties to Haskell. This approach will work just fine with any modern GP language.

Again, this must be obvious to you but I find it completely baffling. Sorry.

Me neither. I'm still baffled. I'm particularly alarmed at the allegation I [AntC] came up with "a similar strategy". I maybe worked to the same objective; which is "similar" in some vague sense to Appendix A PLUS or HHT 1975. I just don't believe "will work just fine with any modern GP language". Because what I did relied heavily on Haskell's higher-order parametric types, phantom types, function types, lambda expressions. It could be adapted to other equally sophisticated functional languages; but I don't see how it would go with a GP language, unless you make heavy use of generics -- and in a way that is not statically provably type-safe.

I also relied heavily on RM Pre 15 (keys) and VSS 2 (key inference) for my 'algorithmic relations' [HHT's term]. Because keys are a consequence of Functional Dependencies. Because if your <whatever-these-are> relations represent functions, they'd darn better have FDs.

David has repeatedly blanked my every mention of keys. So these allegedly 'function relations' are not relations in the RM Pre 15 sense.

My POC -- which I immediately qualify as not industrial strength -- arose from

• Appendix A PLUS and the mechanisms to implement (say) EXTEND using it;
• HHT 1975 'algorithmic relations' -- especially the tantalising mention of LAMBDA expressions; and
• A discussion on the forum in which Philip Kelly critiqued the A operators as not being relationally complete -- contra the claim in Appendix A.

Let's tease out that last point: given a relation R with attributes A, B both INT but not C, X, Y, Z; and Appx A PLUS with heading {X INT, Y INT, Z INT} and FDs {X, Y} -> Z; {Z, X} -> Y; {Z, Y} -> X (because PLUS represents subtraction just as much as addition); and an expression R EXTEND {C := A + B}; how do we follow Appx A to join or compose PLUS with R? We need a RENAME; Appendix A says that can be expressed as an EXTEND and project-away -- i.e. a COMPOSE. OK how do we achieve the RENAME in (say) R JOIN (PLUS RENAME {A := X, B := Y, C := Z}). philipxy said you can't/you're drawn into an infinite regress. Either EXTEND / RENAME must be a primitive (you can define RENAME in terms of EXTEND), or you implement RENAME as COMPOSE and there must be some primitive construct that represents REL{A := X, B := Y, C := Z} -- which looks to me like a (trivial) algorithmic relation; and indeed Tropashko's QBQL has a shorthand to achieve that. To spell it out:

• relation (I'll call it) REN_XYZ2ABC;
• Heading {X INT, Y INT, Z INT, A INT, B INT, C INT};
• FDs X <-> A; Y <-> B; Z <-> C (using <-> to denote a both-ways FD);
• Note those are embedded/non-full/non-normalised FDs, so there's more constraints than just a key.
• We could express that as WITH RENXYZ2ABC := REL{A := X} TIMES REL{B := Y} TIMES REL{C := Z} ... but we still need that primitive to denote equi-relations.

Hugh's reaction at the time was 'that's still a relation'. Which is true; but doesn't answer the point that it's a relation required to have a particular structure, with no primitive in Appendix A to declare that structure. Whereas for HHT 1975 an equi-relation is run-of-the-mill, not worth making a special case.

Note that RENXYZ2ABC can be used to equally to rename A, B, C to X, Y, Z via COMPOSE -- just as PLUS is Janus-faced; or to EXTEND a relation from A, B, C to X, Y Z by 'copying' attribute values, via JOIN.

What I built captured the FDs as phantom types; and did type-level inference of FD's over nestings of algorithmic relations within operator invocations; as well as inferring HHT's 'effectiveness' rules for whether the result of an expression with algrels was also an algrel, or a 'stored set of tuples'. [HHT's term -- it should really be 'storable' because the result from your expression might just be displayed, not stored].