The Third Manifesto ForumThe Third Manifesto: TTM General discussionMeaning of a relation

Please or Register to create posts and topics.

Meaning of a relation

#31 · January 15, 2021, 10:30 am

Quote from dandl on January 15, 2021, 10:30 am

Quote from tobega on January 15, 2021, 9:13 am

A tuple with named attributes is exactly what a Tailspin "structure" is, so creating a relation value is very similar to creating a list of structures.

Be careful: they look the same but a tuple in TTM is more like a map than a traditional struct. There is no order to the names.

Right, there can certainly be dragons here. In Tailspin a structure is also a map from keys to values. More concerning is that the instinctive interpretation of a structure would always be a "has-A"-like relationship between the attributes, regardless of which you pick as the subject of the statement, so that would be like an "and" between attributes, which may conflict with the predicate.

Be careful. These are not RA concepts. The RA deals only in relations: it has no concept of keys and values and picking an attribute. Relations in, relations out. The power comes mainly from JOIN, which looks nothing like indexing.

The idea of a "projection" extends to array indexing, albeit with different projection types for the array parts, and the same syntax for relations can be applied to structures embedded in multi-dimensional arrays. So, e.g., project onto the "foo" attribute of every fifth element of every column of a permutation of the rows.

The RA cannot in general do permutations, and there is no concept of indexing.

Certainly, the example was to show how the same/similar concept and syntax works the same/similarly in a non-RA context.

I don't see it.

All operations in Tailspin can produce a stream of zero or more values which is perfect for ungrouping, but could also make any projection produce more than one tuple, if desired.

Newer languages have built-in Maps and Sets, but they are really just a poor man's relations (tada!) so in Tailspin I want to provide more powerful operations, like the possibility to do a "join" between maps, for example. Since Tailspin has very strict rules about where mutable variables can exist, I'll probably still provide Map and Set processors (i.e. mutable objects), and a more general Relation/Table to enable insertion/deletion semantics and then do relational operations on immutable views.

When I have the relation syntax, I realize that a relation works very well as a source of basic facts for Datalog, so then I suppose I might be getting closer to integrating it with the relational selection syntax.

Yes, a relation is 'just' a set of tuples. No, a tuple is not quite like anything found in any other programming language: it's part struct, part map, part novel.

My view is that the best answer to the 'D' question set by TTM is to implement the features of TTM in an existing language. I can get close in C# but would need a compiler modification; Ant has a solution in Haskell (with a customised version of records), but maybe it could be done in Shen? I haven't looked.

I am fairly certain that it can be done in Shen, or indeed any Lisp, because Lisp gives you access to the Read part of the REPL. If you want to go that route, I think your work might be greatly appreciated and heavily used if you chose to do it in Julia. Since you already have the heavy work of the ERA done, it's "only" implementation "details" left ;-).

Implementing a D to satisfy TTM requires static typing and a particular kind of type inference. REPL is no help, you need to be able to hook into the type system. Lisp does not come with the right kind of type system operators, but Shen just might. It wasn't around when I last surveyed the possibilities, but I admit, it does look promising. I've got another project going right now, but in due course...

The basic relation is bagrules over {container:, contained:, amount:} which states that a container has directly inside it amount of the contained.

Then we set up the datalog rule that we can say that X is a Container of Y either it directly contains it, or it directly contains Z that is a Container of Y.

Last we ask for all pairs (X, 'shiny gold') that satisfy the Container rule, capture them in a list and take the length of the list to find the requested answer. In Tailspin, the inspiration from XSLT is that we don't say in words what we create, but draw a picture, so '[' and ']' create a list containing what the stream producer between them produces.

The syntax $bagrules(<{container: <=X>, contained: <=Y>}>) is how to access the value named bagrules, the () were previously for array indexing but now for general projection-like things. In this case we do a selection on the relation value called bagrules (not yet implemented). The thing within angles <> is how I in Tailspin "draw a picture" of a condition, so a tuple/structure, by {}, containing an attribute "container" that is equal to X and an attribute "contained" that is equal to Y.

Sorry, don't get it. Maybe something more formal?

Here again we have a happy coincidence that the selection syntax is exactly like a condition in a "when"-statement, and another serendipity that the same syntax could also be applied for selection/filtering on lists/arrays (not implemented there either, yet, but I thought about it long ago)

You may not have noticed, but Algebra-A eliminates 'when' in favour of Join with a Relcon. Join is all powerful.

The thing I want to make more Tailspinny is then how to specify the datalog rules and how to specify free variables. On reflection it does come quite close to the "draw a picture" ideal.

Again, I don't get it. But keep trying.

Quote from tobega on January 15, 2021, 9:13 am

A tuple with named attributes is exactly what a Tailspin "structure" is, so creating a relation value is very similar to creating a list of structures.

Be careful: they look the same but a tuple in TTM is more like a map than a traditional struct. There is no order to the names.

Right, there can certainly be dragons here. In Tailspin a structure is also a map from keys to values. More concerning is that the instinctive interpretation of a structure would always be a "has-A"-like relationship between the attributes, regardless of which you pick as the subject of the statement, so that would be like an "and" between attributes, which may conflict with the predicate.

Be careful. These are not RA concepts. The RA deals only in relations: it has no concept of keys and values and picking an attribute. Relations in, relations out. The power comes mainly from JOIN, which looks nothing like indexing.

The idea of a "projection" extends to array indexing, albeit with different projection types for the array parts, and the same syntax for relations can be applied to structures embedded in multi-dimensional arrays. So, e.g., project onto the "foo" attribute of every fifth element of every column of a permutation of the rows.

The RA cannot in general do permutations, and there is no concept of indexing.

Certainly, the example was to show how the same/similar concept and syntax works the same/similarly in a non-RA context.

I don't see it.

All operations in Tailspin can produce a stream of zero or more values which is perfect for ungrouping, but could also make any projection produce more than one tuple, if desired.

Newer languages have built-in Maps and Sets, but they are really just a poor man's relations (tada!) so in Tailspin I want to provide more powerful operations, like the possibility to do a "join" between maps, for example. Since Tailspin has very strict rules about where mutable variables can exist, I'll probably still provide Map and Set processors (i.e. mutable objects), and a more general Relation/Table to enable insertion/deletion semantics and then do relational operations on immutable views.

When I have the relation syntax, I realize that a relation works very well as a source of basic facts for Datalog, so then I suppose I might be getting closer to integrating it with the relational selection syntax.

Yes, a relation is 'just' a set of tuples. No, a tuple is not quite like anything found in any other programming language: it's part struct, part map, part novel.

My view is that the best answer to the 'D' question set by TTM is to implement the features of TTM in an existing language. I can get close in C# but would need a compiler modification; Ant has a solution in Haskell (with a customised version of records), but maybe it could be done in Shen? I haven't looked.

I am fairly certain that it can be done in Shen, or indeed any Lisp, because Lisp gives you access to the Read part of the REPL. If you want to go that route, I think your work might be greatly appreciated and heavily used if you chose to do it in Julia. Since you already have the heavy work of the ERA done, it's "only" implementation "details" left ;-).

Implementing a D to satisfy TTM requires static typing and a particular kind of type inference. REPL is no help, you need to be able to hook into the type system. Lisp does not come with the right kind of type system operators, but Shen just might. It wasn't around when I last surveyed the possibilities, but I admit, it does look promising. I've got another project going right now, but in due course...

The basic relation is bagrules over {container:, contained:, amount:} which states that a container has directly inside it amount of the contained.

Then we set up the datalog rule that we can say that X is a Container of Y either it directly contains it, or it directly contains Z that is a Container of Y.

Last we ask for all pairs (X, 'shiny gold') that satisfy the Container rule, capture them in a list and take the length of the list to find the requested answer. In Tailspin, the inspiration from XSLT is that we don't say in words what we create, but draw a picture, so '[' and ']' create a list containing what the stream producer between them produces.

The syntax $bagrules(<{container: <=X>, contained: <=Y>}>) is how to access the value named bagrules, the () were previously for array indexing but now for general projection-like things. In this case we do a selection on the relation value called bagrules (not yet implemented). The thing within angles <> is how I in Tailspin "draw a picture" of a condition, so a tuple/structure, by {}, containing an attribute "container" that is equal to X and an attribute "contained" that is equal to Y.

Sorry, don't get it. Maybe something more formal?

Here again we have a happy coincidence that the selection syntax is exactly like a condition in a "when"-statement, and another serendipity that the same syntax could also be applied for selection/filtering on lists/arrays (not implemented there either, yet, but I thought about it long ago)

You may not have noticed, but Algebra-A eliminates 'when' in favour of Join with a Relcon. Join is all powerful.

The thing I want to make more Tailspinny is then how to specify the datalog rules and how to specify free variables. On reflection it does come quite close to the "draw a picture" ideal.

Again, I don't get it. But keep trying.

Andl - A New Database Language - andl.org

#32 · January 15, 2021, 1:13 pm

Quote from tobega on January 15, 2021, 1:13 pm

Quote from dandl on January 15, 2021, 10:30 am

Quote from tobega on January 15, 2021, 9:13 am

A tuple with named attributes is exactly what a Tailspin "structure" is, so creating a relation value is very similar to creating a list of structures.

Be careful: they look the same but a tuple in TTM is more like a map than a traditional struct. There is no order to the names.

Right, there can certainly be dragons here. In Tailspin a structure is also a map from keys to values. More concerning is that the instinctive interpretation of a structure would always be a "has-A"-like relationship between the attributes, regardless of which you pick as the subject of the statement, so that would be like an "and" between attributes, which may conflict with the predicate.

Be careful. These are not RA concepts. The RA deals only in relations: it has no concept of keys and values and picking an attribute. Relations in, relations out. The power comes mainly from JOIN, which looks nothing like indexing.

The idea of a "projection" extends to array indexing, albeit with different projection types for the array parts, and the same syntax for relations can be applied to structures embedded in multi-dimensional arrays. So, e.g., project onto the "foo" attribute of every fifth element of every column of a permutation of the rows.

The RA cannot in general do permutations, and there is no concept of indexing.

Certainly, the example was to show how the same/similar concept and syntax works the same/similarly in a non-RA context.

I don't see it.

OK, let's say we have a list/array of the first five primes [2,3,5,7,11]. In an array these have indices, I choose to start from 1, so indices 1,2,3,4,5. Let's say it is instead a tuple where the index is the name of the attribute, so I have { 1: 2, 2: 3, 3: 5, 4: 7, 5:11 } now I think projecting onto the attribute named 3 is rather similar to projecting the list onto its third element (or, as we tend to think of it, selecting the element at position 3). In many languages, Tailspin included, you can select ranges or even pick a list of indices to "project" the list onto a smaller list.

All operations in Tailspin can produce a stream of zero or more values which is perfect for ungrouping, but could also make any projection produce more than one tuple, if desired.

Newer languages have built-in Maps and Sets, but they are really just a poor man's relations (tada!) so in Tailspin I want to provide more powerful operations, like the possibility to do a "join" between maps, for example. Since Tailspin has very strict rules about where mutable variables can exist, I'll probably still provide Map and Set processors (i.e. mutable objects), and a more general Relation/Table to enable insertion/deletion semantics and then do relational operations on immutable views.

When I have the relation syntax, I realize that a relation works very well as a source of basic facts for Datalog, so then I suppose I might be getting closer to integrating it with the relational selection syntax.

Yes, a relation is 'just' a set of tuples. No, a tuple is not quite like anything found in any other programming language: it's part struct, part map, part novel.

My view is that the best answer to the 'D' question set by TTM is to implement the features of TTM in an existing language. I can get close in C# but would need a compiler modification; Ant has a solution in Haskell (with a customised version of records), but maybe it could be done in Shen? I haven't looked.

I am fairly certain that it can be done in Shen, or indeed any Lisp, because Lisp gives you access to the Read part of the REPL. If you want to go that route, I think your work might be greatly appreciated and heavily used if you chose to do it in Julia. Since you already have the heavy work of the ERA done, it's "only" implementation "details" left ;-).

Implementing a D to satisfy TTM requires static typing and a particular kind of type inference. REPL is no help, you need to be able to hook into the type system. Lisp does not come with the right kind of type system operators, but Shen just might. It wasn't around when I last surveyed the possibilities, but I admit, it does look promising. I've got another project going right now, but in due course...

Sure, whatever gives you joy. Shen's type system is of course optionally on or off, but that may be ok, and afaik the type system is turing complete. Julia is probably slightly stricter with its nominative parametric types although dynamic types (type Any) are the default and the structure of the system may not match what you require. Just saying that Shen has a shrinking user base that peaked at 500 people while Julia is fast becoming the language of choice for research scientists who manage lots of data and appreciate mathematical rigour.

The basic relation is bagrules over {container:, contained:, amount:} which states that a container has directly inside it amount of the contained.

Then we set up the datalog rule that we can say that X is a Container of Y either it directly contains it, or it directly contains Z that is a Container of Y.

Last we ask for all pairs (X, 'shiny gold') that satisfy the Container rule, capture them in a list and take the length of the list to find the requested answer. In Tailspin, the inspiration from XSLT is that we don't say in words what we create, but draw a picture, so '[' and ']' create a list containing what the stream producer between them produces.

The syntax $bagrules(<{container: <=X>, contained: <=Y>}>) is how to access the value named bagrules, the () were previously for array indexing but now for general projection-like things. In this case we do a selection on the relation value called bagrules (not yet implemented). The thing within angles <> is how I in Tailspin "draw a picture" of a condition, so a tuple/structure, by {}, containing an attribute "container" that is equal to X and an attribute "contained" that is equal to Y.

Sorry, don't get it. Maybe something more formal?

I hope the datalog parts is clear because my knowledge of datalog barely allowed me to produce that. I could add that I assume that proof is by existence so when putting in values into the rule, e.g. $Container('blue', 'red') you will either get back the pair ('blue', 'red') if the rule evaluates as true, or nothing at all if it evaluates as false. When putting in a free variable like X, you will receive back all the pairs containing possible values for X.

As for the Tailspin selector there, I could tell you that the SQL equivalent of "$bagrules(<{container: <=X>, contained: <=Y>}>)" would be "SELECT * from bagrules where container = X and contained = Y" which again is proof by existence, where the empty result means false.

Here again we have a happy coincidence that the selection syntax is exactly like a condition in a "when"-statement, and another serendipity that the same syntax could also be applied for selection/filtering on lists/arrays (not implemented there either, yet, but I thought about it long ago)

You may not have noticed, but Algebra-A eliminates 'when' in favour of Join with a Relcon. Join is all powerful.

The thing I want to make more Tailspinny is then how to specify the datalog rules and how to specify free variables. On reflection it does come quite close to the "draw a picture" ideal.

Again, I don't get it. But keep trying.

Quote from dandl on January 15, 2021, 10:30 am

Quote from tobega on January 15, 2021, 9:13 am

A tuple with named attributes is exactly what a Tailspin "structure" is, so creating a relation value is very similar to creating a list of structures.

Be careful: they look the same but a tuple in TTM is more like a map than a traditional struct. There is no order to the names.

Right, there can certainly be dragons here. In Tailspin a structure is also a map from keys to values. More concerning is that the instinctive interpretation of a structure would always be a "has-A"-like relationship between the attributes, regardless of which you pick as the subject of the statement, so that would be like an "and" between attributes, which may conflict with the predicate.

Be careful. These are not RA concepts. The RA deals only in relations: it has no concept of keys and values and picking an attribute. Relations in, relations out. The power comes mainly from JOIN, which looks nothing like indexing.

The idea of a "projection" extends to array indexing, albeit with different projection types for the array parts, and the same syntax for relations can be applied to structures embedded in multi-dimensional arrays. So, e.g., project onto the "foo" attribute of every fifth element of every column of a permutation of the rows.

The RA cannot in general do permutations, and there is no concept of indexing.

Certainly, the example was to show how the same/similar concept and syntax works the same/similarly in a non-RA context.

I don't see it.

OK, let's say we have a list/array of the first five primes [2,3,5,7,11]. In an array these have indices, I choose to start from 1, so indices 1,2,3,4,5. Let's say it is instead a tuple where the index is the name of the attribute, so I have { 1: 2, 2: 3, 3: 5, 4: 7, 5:11 } now I think projecting onto the attribute named 3 is rather similar to projecting the list onto its third element (or, as we tend to think of it, selecting the element at position 3). In many languages, Tailspin included, you can select ranges or even pick a list of indices to "project" the list onto a smaller list.

All operations in Tailspin can produce a stream of zero or more values which is perfect for ungrouping, but could also make any projection produce more than one tuple, if desired.

Newer languages have built-in Maps and Sets, but they are really just a poor man's relations (tada!) so in Tailspin I want to provide more powerful operations, like the possibility to do a "join" between maps, for example. Since Tailspin has very strict rules about where mutable variables can exist, I'll probably still provide Map and Set processors (i.e. mutable objects), and a more general Relation/Table to enable insertion/deletion semantics and then do relational operations on immutable views.

When I have the relation syntax, I realize that a relation works very well as a source of basic facts for Datalog, so then I suppose I might be getting closer to integrating it with the relational selection syntax.

Yes, a relation is 'just' a set of tuples. No, a tuple is not quite like anything found in any other programming language: it's part struct, part map, part novel.

My view is that the best answer to the 'D' question set by TTM is to implement the features of TTM in an existing language. I can get close in C# but would need a compiler modification; Ant has a solution in Haskell (with a customised version of records), but maybe it could be done in Shen? I haven't looked.

I am fairly certain that it can be done in Shen, or indeed any Lisp, because Lisp gives you access to the Read part of the REPL. If you want to go that route, I think your work might be greatly appreciated and heavily used if you chose to do it in Julia. Since you already have the heavy work of the ERA done, it's "only" implementation "details" left ;-).

Implementing a D to satisfy TTM requires static typing and a particular kind of type inference. REPL is no help, you need to be able to hook into the type system. Lisp does not come with the right kind of type system operators, but Shen just might. It wasn't around when I last surveyed the possibilities, but I admit, it does look promising. I've got another project going right now, but in due course...

Sure, whatever gives you joy. Shen's type system is of course optionally on or off, but that may be ok, and afaik the type system is turing complete. Julia is probably slightly stricter with its nominative parametric types although dynamic types (type Any) are the default and the structure of the system may not match what you require. Just saying that Shen has a shrinking user base that peaked at 500 people while Julia is fast becoming the language of choice for research scientists who manage lots of data and appreciate mathematical rigour.

The basic relation is bagrules over {container:, contained:, amount:} which states that a container has directly inside it amount of the contained.

Then we set up the datalog rule that we can say that X is a Container of Y either it directly contains it, or it directly contains Z that is a Container of Y.

Last we ask for all pairs (X, 'shiny gold') that satisfy the Container rule, capture them in a list and take the length of the list to find the requested answer. In Tailspin, the inspiration from XSLT is that we don't say in words what we create, but draw a picture, so '[' and ']' create a list containing what the stream producer between them produces.

The syntax $bagrules(<{container: <=X>, contained: <=Y>}>) is how to access the value named bagrules, the () were previously for array indexing but now for general projection-like things. In this case we do a selection on the relation value called bagrules (not yet implemented). The thing within angles <> is how I in Tailspin "draw a picture" of a condition, so a tuple/structure, by {}, containing an attribute "container" that is equal to X and an attribute "contained" that is equal to Y.

Sorry, don't get it. Maybe something more formal?

I hope the datalog parts is clear because my knowledge of datalog barely allowed me to produce that. I could add that I assume that proof is by existence so when putting in values into the rule, e.g. $Container('blue', 'red') you will either get back the pair ('blue', 'red') if the rule evaluates as true, or nothing at all if it evaluates as false. When putting in a free variable like X, you will receive back all the pairs containing possible values for X.

As for the Tailspin selector there, I could tell you that the SQL equivalent of "$bagrules(<{container: <=X>, contained: <=Y>}>)" would be "SELECT * from bagrules where container = X and contained = Y" which again is proof by existence, where the empty result means false.

Here again we have a happy coincidence that the selection syntax is exactly like a condition in a "when"-statement, and another serendipity that the same syntax could also be applied for selection/filtering on lists/arrays (not implemented there either, yet, but I thought about it long ago)

You may not have noticed, but Algebra-A eliminates 'when' in favour of Join with a Relcon. Join is all powerful.

The thing I want to make more Tailspinny is then how to specify the datalog rules and how to specify free variables. On reflection it does come quite close to the "draw a picture" ideal.

Again, I don't get it. But keep trying.

#33 · January 16, 2021, 8:40 am

Quote from dandl on January 16, 2021, 8:40 am

The idea of a "projection" extends to array indexing, albeit with different projection types for the array parts, and the same syntax for relations can be applied to structures embedded in multi-dimensional arrays. So, e.g., project onto the "foo" attribute of every fifth element of every column of a permutation of the rows.

The RA cannot in general do permutations, and there is no concept of indexing.

Certainly, the example was to show how the same/similar concept and syntax works the same/similarly in a non-RA context.

I don't see it.

OK, let's say we have a list/array of the first five primes [2,3,5,7,11]. In an array these have indices, I choose to start from 1, so indices 1,2,3,4,5. Let's say it is instead a tuple where the index is the name of the attribute, so I have { 1: 2, 2: 3, 3: 5, 4: 7, 5:11 } now I think projecting onto the attribute named 3 is rather similar to projecting the list onto its third element (or, as we tend to think of it, selecting the element at position 3). In many languages, Tailspin included, you can select ranges or even pick a list of indices to "project" the list onto a smaller list.

The similarity is superficial and deceptive.

First, the RA deals in relation (values). Operations on tuples are a feature of TTM, not the RA.

Projection is about removing attributes, but the result is still a tuple and the values do not get reindexed as they do in arrays.

You are describing 'slices', but a smaller list does not have the same indexes preserved.

Implementing a D to satisfy TTM requires static typing and a particular kind of type inference. REPL is no help, you need to be able to hook into the type system. Lisp does not come with the right kind of type system operators, but Shen just might. It wasn't around when I last surveyed the possibilities, but I admit, it does look promising. I've got another project going right now, but in due course...

Sure, whatever gives you joy. Shen's type system is of course optionally on or off, but that may be ok, and afaik the type system is turing complete. Julia is probably slightly stricter with its nominative parametric types although dynamic types (type Any) are the default and the structure of the system may not match what you require. Just saying that Shen has a shrinking user base that peaked at 500 people while Julia is fast becoming the language of choice for research scientists who manage lots of data and appreciate mathematical rigour.

Joy has nothing to do with it, and a Turing complete type system is not needed. To implement D requires operations on types (Haskell can do that) and a suitable record-like starting point (standard Haskell cannot). I'll have (another) look at Julia and see if it (now) has anything suitable.

The idea of a "projection" extends to array indexing, albeit with different projection types for the array parts, and the same syntax for relations can be applied to structures embedded in multi-dimensional arrays. So, e.g., project onto the "foo" attribute of every fifth element of every column of a permutation of the rows.

The RA cannot in general do permutations, and there is no concept of indexing.

Certainly, the example was to show how the same/similar concept and syntax works the same/similarly in a non-RA context.

I don't see it.

OK, let's say we have a list/array of the first five primes [2,3,5,7,11]. In an array these have indices, I choose to start from 1, so indices 1,2,3,4,5. Let's say it is instead a tuple where the index is the name of the attribute, so I have { 1: 2, 2: 3, 3: 5, 4: 7, 5:11 } now I think projecting onto the attribute named 3 is rather similar to projecting the list onto its third element (or, as we tend to think of it, selecting the element at position 3). In many languages, Tailspin included, you can select ranges or even pick a list of indices to "project" the list onto a smaller list.

The similarity is superficial and deceptive.

First, the RA deals in relation (values). Operations on tuples are a feature of TTM, not the RA.
Projection is about removing attributes, but the result is still a tuple and the values do not get reindexed as they do in arrays.
You are describing 'slices', but a smaller list does not have the same indexes preserved.

Implementing a D to satisfy TTM requires static typing and a particular kind of type inference. REPL is no help, you need to be able to hook into the type system. Lisp does not come with the right kind of type system operators, but Shen just might. It wasn't around when I last surveyed the possibilities, but I admit, it does look promising. I've got another project going right now, but in due course...

Sure, whatever gives you joy. Shen's type system is of course optionally on or off, but that may be ok, and afaik the type system is turing complete. Julia is probably slightly stricter with its nominative parametric types although dynamic types (type Any) are the default and the structure of the system may not match what you require. Just saying that Shen has a shrinking user base that peaked at 500 people while Julia is fast becoming the language of choice for research scientists who manage lots of data and appreciate mathematical rigour.

Joy has nothing to do with it, and a Turing complete type system is not needed. To implement D requires operations on types (Haskell can do that) and a suitable record-like starting point (standard Haskell cannot). I'll have (another) look at Julia and see if it (now) has anything suitable.

Andl - A New Database Language - andl.org

#34 · January 16, 2021, 4:20 pm

Quote from dandl on January 16, 2021, 8:40 am

Joy has nothing to do with it, and a Turing complete type system is not needed. To implement D requires operations on types (Haskell can do that) and a suitable record-like starting point (standard Haskell cannot). I'll have (another) look at Julia and see if it (now) has anything suitable.

I'm getting increasingly curious about the details of this. What was the problem in Haskell? What operations on types are needed? Is that where C# fell?

#35 · January 17, 2021, 12:08 am

Quote from dandl on January 17, 2021, 12:08 am

Quote from tobega on January 16, 2021, 4:20 pm

Quote from dandl on January 16, 2021, 8:40 am

Joy has nothing to do with it, and a Turing complete type system is not needed. To implement D requires operations on types (Haskell can do that) and a suitable record-like starting point (standard Haskell cannot). I'll have (another) look at Julia and see if it (now) has anything suitable.

I'm getting increasingly curious about the details of this. What was the problem in Haskell? What operations on types are needed? Is that where C# fell?

As I understand it, the issue is that the standard Haskell records type model is not suitable, and the TTM type system cannot be implemented in bog-standard GHC Haskell. However, the Haskell community has generated other records type models, and at least one of those can be used. I defer to Ant as the authority on this.

TTM deals with tuple/relation types in a rather idiosyncratic way.

A heading is a set of <name,type> pairs.

A tuple type is generated from a heading and acquires a name derived from that heading.

A relation type is generated from a heading and acquires a name derived from that heading.

A tuple type defines a set of tuple values.

A tuple value is a set of <name,value,type> triples

A relation type defines a set of relation values.

A relation value is a set of tuple values with the same heading.

Every relation/tuple type/value with that heading is the very same type/value.

To implement this in an existing language you have to add means to:

parse a heading, and either define a new one or reuse an existing one

define/reuse a heading inferred for values returned by operations of the RA.

generate/reuse a tuple/relation type and type name from a heading

make type information (mainly the heading) available for the implementation to use.

There is a simpler approach based on generic types, where the heading is a parameter of a tuple/relation type. This can be implemented in C#, but there is more obligation on the user to write the code in a particular way and some of the type checking has to be deferred to runtime. I got it working, but a good solution would require a compiler mod.

Without something along these lines, you can't really include the RA in a language.

Quote from tobega on January 16, 2021, 4:20 pm

Quote from dandl on January 16, 2021, 8:40 am

Joy has nothing to do with it, and a Turing complete type system is not needed. To implement D requires operations on types (Haskell can do that) and a suitable record-like starting point (standard Haskell cannot). I'll have (another) look at Julia and see if it (now) has anything suitable.

I'm getting increasingly curious about the details of this. What was the problem in Haskell? What operations on types are needed? Is that where C# fell?

As I understand it, the issue is that the standard Haskell records type model is not suitable, and the TTM type system cannot be implemented in bog-standard GHC Haskell. However, the Haskell community has generated other records type models, and at least one of those can be used. I defer to Ant as the authority on this.

TTM deals with tuple/relation types in a rather idiosyncratic way.

A heading is a set of <name,type> pairs.
A tuple type is generated from a heading and acquires a name derived from that heading.
A relation type is generated from a heading and acquires a name derived from that heading.
A tuple type defines a set of tuple values.
A tuple value is a set of <name,value,type> triples
A relation type defines a set of relation values.
A relation value is a set of tuple values with the same heading.
Every relation/tuple type/value with that heading is the very same type/value.

To implement this in an existing language you have to add means to:

parse a heading, and either define a new one or reuse an existing one
define/reuse a heading inferred for values returned by operations of the RA.
generate/reuse a tuple/relation type and type name from a heading
make type information (mainly the heading) available for the implementation to use.

There is a simpler approach based on generic types, where the heading is a parameter of a tuple/relation type. This can be implemented in C#, but there is more obligation on the user to write the code in a particular way and some of the type checking has to be deferred to runtime. I got it working, but a good solution would require a compiler mod.

Without something along these lines, you can't really include the RA in a language.

Andl - A New Database Language - andl.org

#36 · January 17, 2021, 7:49 am

Quote from tobega on January 17, 2021, 7:49 am

Quote from dandl on January 17, 2021, 12:08 am

Quote from tobega on January 16, 2021, 4:20 pm

Quote from dandl on January 16, 2021, 8:40 am

Joy has nothing to do with it, and a Turing complete type system is not needed. To implement D requires operations on types (Haskell can do that) and a suitable record-like starting point (standard Haskell cannot). I'll have (another) look at Julia and see if it (now) has anything suitable.

I'm getting increasingly curious about the details of this. What was the problem in Haskell? What operations on types are needed? Is that where C# fell?

As I understand it, the issue is that the standard Haskell records type model is not suitable, and the TTM type system cannot be implemented in bog-standard GHC Haskell. However, the Haskell community has generated other records type models, and at least one of those can be used. I defer to Ant as the authority on this.

TTM deals with tuple/relation types in a rather idiosyncratic way.

A heading is a set of <name,type> pairs.

A tuple type is generated from a heading and acquires a name derived from that heading.

A relation type is generated from a heading and acquires a name derived from that heading.

A tuple type defines a set of tuple values.

A tuple value is a set of <name,value,type> triples

A relation type defines a set of relation values.

A relation value is a set of tuple values with the same heading.

Every relation/tuple type/value with that heading is the very same type/value.

To implement this in an existing language you have to add means to:

parse a heading, and either define a new one or reuse an existing one

define/reuse a heading inferred for values returned by operations of the RA.

generate/reuse a tuple/relation type and type name from a heading

make type information (mainly the heading) available for the implementation to use.

There is a simpler approach based on generic types, where the heading is a parameter of a tuple/relation type. This can be implemented in C#, but there is more obligation on the user to write the code in a particular way and some of the type checking has to be deferred to runtime. I got it working, but a good solution would require a compiler mod.

Without something along these lines, you can't really include the RA in a language.

Right. I thought to dig into Julia a little and it has a NamedTuple type which seems to be really close to what you want and Julia makes it really easy to access all parts of it both on the type and value level and dynamically create new NamedTuples with dynamically derived headings.

The type declaration for Relation might be abstract type Relation<:AbstractSet{NamedTuple{N,T} where {N,T}} end although you probably want to make it a concrete type instead.

The only wrinkle is that despite having names, standard equality is dependent on the order of the names so that (a = 1, b = "hello") is not the same as (b = "hello", a = 1). It would however be possible to canonicalize the tuples by sorting names alphabetically, the following functions demonstrate how to do that and also how easily you manipulate types

sortednames(nt::NamedTuple{N,T}) where {N,T} = Tuple(sort([N...]))

sortednt(nt::NamedTuple) = NamedTuple{sortednames(nt)}(nt)
So sortednt((a = 1, b = "hello")) == sortednt((b = "hello", a = 1))

Quote from dandl on January 17, 2021, 12:08 am

Quote from tobega on January 16, 2021, 4:20 pm

Quote from dandl on January 16, 2021, 8:40 am

Joy has nothing to do with it, and a Turing complete type system is not needed. To implement D requires operations on types (Haskell can do that) and a suitable record-like starting point (standard Haskell cannot). I'll have (another) look at Julia and see if it (now) has anything suitable.

I'm getting increasingly curious about the details of this. What was the problem in Haskell? What operations on types are needed? Is that where C# fell?

As I understand it, the issue is that the standard Haskell records type model is not suitable, and the TTM type system cannot be implemented in bog-standard GHC Haskell. However, the Haskell community has generated other records type models, and at least one of those can be used. I defer to Ant as the authority on this.

TTM deals with tuple/relation types in a rather idiosyncratic way.

A heading is a set of <name,type> pairs.

A tuple type is generated from a heading and acquires a name derived from that heading.

A relation type is generated from a heading and acquires a name derived from that heading.

A tuple type defines a set of tuple values.

A tuple value is a set of <name,value,type> triples

A relation type defines a set of relation values.

A relation value is a set of tuple values with the same heading.

Every relation/tuple type/value with that heading is the very same type/value.

To implement this in an existing language you have to add means to:

parse a heading, and either define a new one or reuse an existing one

define/reuse a heading inferred for values returned by operations of the RA.

generate/reuse a tuple/relation type and type name from a heading

make type information (mainly the heading) available for the implementation to use.

There is a simpler approach based on generic types, where the heading is a parameter of a tuple/relation type. This can be implemented in C#, but there is more obligation on the user to write the code in a particular way and some of the type checking has to be deferred to runtime. I got it working, but a good solution would require a compiler mod.

Without something along these lines, you can't really include the RA in a language.

Right. I thought to dig into Julia a little and it has a NamedTuple type which seems to be really close to what you want and Julia makes it really easy to access all parts of it both on the type and value level and dynamically create new NamedTuples with dynamically derived headings.

The type declaration for Relation might be abstract type Relation<:AbstractSet{NamedTuple{N,T} where {N,T}} end although you probably want to make it a concrete type instead.

The only wrinkle is that despite having names, standard equality is dependent on the order of the names so that (a = 1, b = "hello") is not the same as (b = "hello", a = 1). It would however be possible to canonicalize the tuples by sorting names alphabetically, the following functions demonstrate how to do that and also how easily you manipulate types

sortednames(nt::NamedTuple{N,T}) where {N,T} = Tuple(sort([N...]))

sortednt(nt::NamedTuple) = NamedTuple{sortednames(nt)}(nt)
So sortednt((a = 1, b = "hello")) == sortednt((b = "hello", a = 1))

#37 · January 17, 2021, 7:54 am

Quote from tobega on January 17, 2021, 7:54 am

Quote from tobega on January 17, 2021, 7:49 am

Quote from dandl on January 17, 2021, 12:08 am

Quote from tobega on January 16, 2021, 4:20 pm

Quote from dandl on January 16, 2021, 8:40 am

Joy has nothing to do with it, and a Turing complete type system is not needed. To implement D requires operations on types (Haskell can do that) and a suitable record-like starting point (standard Haskell cannot). I'll have (another) look at Julia and see if it (now) has anything suitable.

I'm getting increasingly curious about the details of this. What was the problem in Haskell? What operations on types are needed? Is that where C# fell?

As I understand it, the issue is that the standard Haskell records type model is not suitable, and the TTM type system cannot be implemented in bog-standard GHC Haskell. However, the Haskell community has generated other records type models, and at least one of those can be used. I defer to Ant as the authority on this.

TTM deals with tuple/relation types in a rather idiosyncratic way.

A heading is a set of <name,type> pairs.

A tuple type is generated from a heading and acquires a name derived from that heading.

A relation type is generated from a heading and acquires a name derived from that heading.

A tuple type defines a set of tuple values.

A tuple value is a set of <name,value,type> triples

A relation type defines a set of relation values.

A relation value is a set of tuple values with the same heading.

Every relation/tuple type/value with that heading is the very same type/value.

To implement this in an existing language you have to add means to:

parse a heading, and either define a new one or reuse an existing one

define/reuse a heading inferred for values returned by operations of the RA.

generate/reuse a tuple/relation type and type name from a heading

make type information (mainly the heading) available for the implementation to use.

There is a simpler approach based on generic types, where the heading is a parameter of a tuple/relation type. This can be implemented in C#, but there is more obligation on the user to write the code in a particular way and some of the type checking has to be deferred to runtime. I got it working, but a good solution would require a compiler mod.

Without something along these lines, you can't really include the RA in a language.

Right. I thought to dig into Julia a little and it has a NamedTuple type which seems to be really close to what you want and Julia makes it really easy to access all parts of it both on the type and value level and dynamically create new NamedTuples with dynamically derived headings.

The type declaration for Relation might be abstract type Relation<:AbstractSet{NamedTuple{N,T} where {N,T}} end although you probably want to make it a concrete type instead.

The only wrinkle is that despite having names, standard equality is dependent on the order of the names so that (a = 1, b = "hello") is not the same as (b = "hello", a = 1). It would however be possible to canonicalize the tuples by sorting names alphabetically, the following functions demonstrate how to do that and also how easily you manipulate types

sortednames(nt::NamedTuple{N,T}) where {N,T} = Tuple(sort([N...]))

sortednt(nt::NamedTuple) = NamedTuple{sortednames(nt)}(nt)
So sortednt((a = 1, b = "hello")) == sortednt((b = "hello", a = 1))

Looking closer, you don't even need to sort the names, you can just coerce them into the same order by NamedTuple{headingNames}(x)

Quote from tobega on January 17, 2021, 7:49 am

Quote from dandl on January 17, 2021, 12:08 am

Quote from tobega on January 16, 2021, 4:20 pm

Quote from dandl on January 16, 2021, 8:40 am

Joy has nothing to do with it, and a Turing complete type system is not needed. To implement D requires operations on types (Haskell can do that) and a suitable record-like starting point (standard Haskell cannot). I'll have (another) look at Julia and see if it (now) has anything suitable.

I'm getting increasingly curious about the details of this. What was the problem in Haskell? What operations on types are needed? Is that where C# fell?

As I understand it, the issue is that the standard Haskell records type model is not suitable, and the TTM type system cannot be implemented in bog-standard GHC Haskell. However, the Haskell community has generated other records type models, and at least one of those can be used. I defer to Ant as the authority on this.

TTM deals with tuple/relation types in a rather idiosyncratic way.

A heading is a set of <name,type> pairs.

A tuple type is generated from a heading and acquires a name derived from that heading.

A relation type is generated from a heading and acquires a name derived from that heading.

A tuple type defines a set of tuple values.

A tuple value is a set of <name,value,type> triples

A relation type defines a set of relation values.

A relation value is a set of tuple values with the same heading.

Every relation/tuple type/value with that heading is the very same type/value.

To implement this in an existing language you have to add means to:

parse a heading, and either define a new one or reuse an existing one

define/reuse a heading inferred for values returned by operations of the RA.

generate/reuse a tuple/relation type and type name from a heading

make type information (mainly the heading) available for the implementation to use.

There is a simpler approach based on generic types, where the heading is a parameter of a tuple/relation type. This can be implemented in C#, but there is more obligation on the user to write the code in a particular way and some of the type checking has to be deferred to runtime. I got it working, but a good solution would require a compiler mod.

Without something along these lines, you can't really include the RA in a language.

Right. I thought to dig into Julia a little and it has a NamedTuple type which seems to be really close to what you want and Julia makes it really easy to access all parts of it both on the type and value level and dynamically create new NamedTuples with dynamically derived headings.

The type declaration for Relation might be abstract type Relation<:AbstractSet{NamedTuple{N,T} where {N,T}} end although you probably want to make it a concrete type instead.

The only wrinkle is that despite having names, standard equality is dependent on the order of the names so that (a = 1, b = "hello") is not the same as (b = "hello", a = 1). It would however be possible to canonicalize the tuples by sorting names alphabetically, the following functions demonstrate how to do that and also how easily you manipulate types

sortednames(nt::NamedTuple{N,T}) where {N,T} = Tuple(sort([N...]))

sortednt(nt::NamedTuple) = NamedTuple{sortednames(nt)}(nt)
So sortednt((a = 1, b = "hello")) == sortednt((b = "hello", a = 1))

Looking closer, you don't even need to sort the names, you can just coerce them into the same order by NamedTuple{headingNames}(x)

#38 · January 17, 2021, 8:36 am

Quote from tobega on January 17, 2021, 8:36 am

Quote from tobega on January 17, 2021, 7:54 am

Quote from tobega on January 17, 2021, 7:49 am

Quote from dandl on January 17, 2021, 12:08 am

Quote from tobega on January 16, 2021, 4:20 pm

Quote from dandl on January 16, 2021, 8:40 am

Joy has nothing to do with it, and a Turing complete type system is not needed. To implement D requires operations on types (Haskell can do that) and a suitable record-like starting point (standard Haskell cannot). I'll have (another) look at Julia and see if it (now) has anything suitable.

I'm getting increasingly curious about the details of this. What was the problem in Haskell? What operations on types are needed? Is that where C# fell?

As I understand it, the issue is that the standard Haskell records type model is not suitable, and the TTM type system cannot be implemented in bog-standard GHC Haskell. However, the Haskell community has generated other records type models, and at least one of those can be used. I defer to Ant as the authority on this.

TTM deals with tuple/relation types in a rather idiosyncratic way.

A heading is a set of <name,type> pairs.

A tuple type is generated from a heading and acquires a name derived from that heading.

A relation type is generated from a heading and acquires a name derived from that heading.

A tuple type defines a set of tuple values.

A tuple value is a set of <name,value,type> triples

A relation type defines a set of relation values.

A relation value is a set of tuple values with the same heading.

Every relation/tuple type/value with that heading is the very same type/value.

To implement this in an existing language you have to add means to:

parse a heading, and either define a new one or reuse an existing one

define/reuse a heading inferred for values returned by operations of the RA.

generate/reuse a tuple/relation type and type name from a heading

make type information (mainly the heading) available for the implementation to use.

There is a simpler approach based on generic types, where the heading is a parameter of a tuple/relation type. This can be implemented in C#, but there is more obligation on the user to write the code in a particular way and some of the type checking has to be deferred to runtime. I got it working, but a good solution would require a compiler mod.

Without something along these lines, you can't really include the RA in a language.

Right. I thought to dig into Julia a little and it has a NamedTuple type which seems to be really close to what you want and Julia makes it really easy to access all parts of it both on the type and value level and dynamically create new NamedTuples with dynamically derived headings.

The type declaration for Relation might be abstract type Relation<:AbstractSet{NamedTuple{N,T} where {N,T}} end although you probably want to make it a concrete type instead.

The only wrinkle is that despite having names, standard equality is dependent on the order of the names so that (a = 1, b = "hello") is not the same as (b = "hello", a = 1). It would however be possible to canonicalize the tuples by sorting names alphabetically, the following functions demonstrate how to do that and also how easily you manipulate types

sortednames(nt::NamedTuple{N,T}) where {N,T} = Tuple(sort([N...]))

sortednt(nt::NamedTuple) = NamedTuple{sortednames(nt)}(nt)
So sortednt((a = 1, b = "hello")) == sortednt((b = "hello", a = 1))

Looking closer, you don't even need to sort the names, you can just coerce them into the same order by NamedTuple{headingNames}(x)

Then again, NamedTuple itself is just defined in plain Julia code, so it might not be too hard to define a TTM-tuple type with the correct semantics based on that https://github.com/JuliaLang/julia/blob/master/base/namedtuple.jl

Quote from tobega on January 17, 2021, 7:54 am

Quote from tobega on January 17, 2021, 7:49 am

Quote from dandl on January 17, 2021, 12:08 am

Quote from tobega on January 16, 2021, 4:20 pm

Quote from dandl on January 16, 2021, 8:40 am

Joy has nothing to do with it, and a Turing complete type system is not needed. To implement D requires operations on types (Haskell can do that) and a suitable record-like starting point (standard Haskell cannot). I'll have (another) look at Julia and see if it (now) has anything suitable.

I'm getting increasingly curious about the details of this. What was the problem in Haskell? What operations on types are needed? Is that where C# fell?

As I understand it, the issue is that the standard Haskell records type model is not suitable, and the TTM type system cannot be implemented in bog-standard GHC Haskell. However, the Haskell community has generated other records type models, and at least one of those can be used. I defer to Ant as the authority on this.

TTM deals with tuple/relation types in a rather idiosyncratic way.

A heading is a set of <name,type> pairs.

A tuple type is generated from a heading and acquires a name derived from that heading.

A relation type is generated from a heading and acquires a name derived from that heading.

A tuple type defines a set of tuple values.

A tuple value is a set of <name,value,type> triples

A relation type defines a set of relation values.

A relation value is a set of tuple values with the same heading.

Every relation/tuple type/value with that heading is the very same type/value.

To implement this in an existing language you have to add means to:

parse a heading, and either define a new one or reuse an existing one

define/reuse a heading inferred for values returned by operations of the RA.

generate/reuse a tuple/relation type and type name from a heading

make type information (mainly the heading) available for the implementation to use.

There is a simpler approach based on generic types, where the heading is a parameter of a tuple/relation type. This can be implemented in C#, but there is more obligation on the user to write the code in a particular way and some of the type checking has to be deferred to runtime. I got it working, but a good solution would require a compiler mod.

Without something along these lines, you can't really include the RA in a language.

Right. I thought to dig into Julia a little and it has a NamedTuple type which seems to be really close to what you want and Julia makes it really easy to access all parts of it both on the type and value level and dynamically create new NamedTuples with dynamically derived headings.

The type declaration for Relation might be abstract type Relation<:AbstractSet{NamedTuple{N,T} where {N,T}} end although you probably want to make it a concrete type instead.

The only wrinkle is that despite having names, standard equality is dependent on the order of the names so that (a = 1, b = "hello") is not the same as (b = "hello", a = 1). It would however be possible to canonicalize the tuples by sorting names alphabetically, the following functions demonstrate how to do that and also how easily you manipulate types

sortednames(nt::NamedTuple{N,T}) where {N,T} = Tuple(sort([N...]))

sortednt(nt::NamedTuple) = NamedTuple{sortednames(nt)}(nt)
So sortednt((a = 1, b = "hello")) == sortednt((b = "hello", a = 1))

Looking closer, you don't even need to sort the names, you can just coerce them into the same order by NamedTuple{headingNames}(x)

Then again, NamedTuple itself is just defined in plain Julia code, so it might not be too hard to define a TTM-tuple type with the correct semantics based on that https://github.com/JuliaLang/julia/blob/master/base/namedtuple.jl

#39 · January 17, 2021, 1:15 pm

It's unlikely it would work. Here's the list again.

To implement this in an existing language you have to add means to:

- parse a heading, and either define a new one or reuse an existing one
- define/reuse a heading inferred for values returned by operations of the RA.
- generate/reuse a tuple/relation type and type name from a heading
- make type information (mainly the heading) available for the implementation to use.

I've had a trawl through the Julia docs, and it looks much as I remember it. There is an extremely powerful early stage macro facility that interacts with ASTs and can do symbol and code generation, like the Lisp it inherits from. What I did not find was any sign of interaction between this and the type system, or the kind of static type system smarts found in Haskell.

And if you really can't generate a heading that is a set of <name,type> but rather need to fake it by sorting, then chances are Julia can't do the rest either. But feel free to prove me wrong, I think everyone here would like to find that there is a language that can do this.

Andl - A New Database Language - andl.org

#40 · January 17, 2021, 3:54 pm

Quote from dandl on January 17, 2021, 1:15 pm

It's unlikely it would work. Here's the list again.

[...]

And if you really can't generate a heading that is a set of <name,type> but rather need to fake it by sorting, then chances are Julia can't do the rest either. But feel free to prove me wrong, I think everyone here would like to find that there is a language that can do this.

Yes. And more to the point, we'd like to find a popular language that can do this. Some of us have created or implemented languages that can do this, but that's largely only of academic interest in a world dominated by Python, C, Java, C#, JavaScript and C++, plus a host of second-tier (and third-tier) runners that don't do it either.

You can kind of approximate it in various ways -- or decide not to approximate it at all and go in various directions more or less in the spirit of TTM -- but these all involve compromises.

Of course, creating your own language is a compromise too. You gain all the ideal utility and lose the benefits of popularity, but I guess it's an uncompromising compromise.

I'm the forum administrator and lead developer of Rel. Email me at dave@armchair.mb.ca with the Subject 'TTM Forum'. Download Rel from https://reldb.org

The Forum for Discussion about The Third Manifesto and Related Matters

Meaning of a relation