The future of TTM

#21 · July 25, 2021, 12:37 am

The final result should be at least as capable as SQLite in every detail, but all written in the host language with no SQL. If you're competing with SQL Server/Oracle/MySQL you're on the wrong train!

A bit like Prevayler (https://prevayler.org/) but with relational operators, then?

Bit of a long bow. Definitely not 'objects in the cloud', not a hint of supporting or endorsing an in-memory object model linked by references. In many ways it's more like one of those NoSQL products where the code to create a database, store data in it and query it is all part of the same program written in the same language, but tables/relations, not a graph.

Note that one of the big selling points around SQLite (other than using SQL, which developers generally treat as a necessary but uninteresting evil, similar to the way regular expressions are treated) is that it works cross-language (your Python code can write a SQLite db and your Java code can read it) and you can browse it with external tools like DB Browser (https://sqlitebrowser.org/). Not sure how you'd handle that with native non-canonical types, but maybe the type-handling of cross-language object brokers like CORBA would provide some ideas, or maybe the focus would be on language integration and it simply wouldn't support cross-language portability or browsing.

Cross-language is of no interest if your database is never shared, but if you must do it, it's easy enough to stick to a common subset of types. JSON can do it, and so can products like Thrift and Protocol Buffers. I haven't looked at CORBA in years, so I don't know if that helps.

Yes, a browser is absolutely needed but that's a single tool and not too hard to create, maybe adapt one from the SQL world.

Andl - A New Database Language - andl.org

#22 · July 26, 2021, 1:29 am

Quote from dandl on July 26, 2021, 1:29 am

Quote from Dave Voorhis on July 23, 2021, 9:19 am

Quote from dandl on July 23, 2021, 4:33 am

I think it would be fair to say that the original goals of TTM, however expressed, are unlikely to be realised. SQL is centre stage, and probably has been since about the mid 1990s. So quo vadis TTM?

For managing and accessing a shared corporate application database, SQL serves well. For business application development accessing a shared database, modern GP languages such as Java and C# are sufficient. Despite the need for an ORM and 'glue' code the combination is good enough, and no-one is looking to replace it. For everything else, it has (or should have) competition.

Features that get me looking for something other than SQL+GP include:

embedded or in-process database (no server, not shared)

non-business data eg images, audio, time series, geographic, real-time

non-database data eg file hierarchies, CSV files, spreadsheets, documents

anytime setting up and maintaining a server does not seem feasible/justified

anytime writing SQL plus ORM/ODBC glue code does not seem feasible/justified.

There is competition, including:

SQLite: in-process, but still SQL+ORM

NoSQL: but you don't get relational queries

LINQ/Java Streams: but you don't get update

bare metal raw files plus various libraries.

So what does TTM offer? Stripping out the new language (which no-one seems to want), the main features seem to be:

A type system that can be applied to attributes

An in-language DQL based on the RA

Updates based on relational assignment

Database features including transactions and constraints.

So the addressable market is everyone using data that is not a good match for SQL+GP (because of any of the features listed above), especially those who are already using SQLite or NoSQL or raw files, and who would benefit from any of the TTM features. That's huge.

I agree. There are undoubtedly a vast number of tools that can be written -- in and/or for any number of different languages, platforms, and environments -- which embody one or more of the TTM & relational model & relational algebra ideas. SQL itself isn't yet ready to be replaced -- and promoting products solely on being faithful to the relational model doesn't work as nobody but us cares and the general assumption is that relational == SQL -- but languages, platforms, tools, utilities, you-name-it that do something good (i.e., that save/earn money and/or save effort and/or improve quality) and are based on TTM ideas even if not explicitly mentioned as such in the brochure?

Absolutely.

SQL is absolutely ready to be replaced and is being replaced for many of the above use cases. The stand-out is NoSQL, as the name implies. Client-side LINQ and Java Streams are being used to replace server-side SQL. Countless developers solving problems with the features listed above are cussing SQL and wishing for something better very day.

Yes people don't care about 'faithful' but they certainly care about solving their problems. What user/developer problem does TTM solve? The RM/RA are well known (ask Google) and SQL is a pain. For some X the pitch is:

solves problem X better than SQL

does absolutely everything you can do with X in SQL but better/faster/easier/cheaper

based on the RA/RM.

Now you have something to sell (for some X).

Quote from Dave Voorhis on July 23, 2021, 9:19 am

Quote from dandl on July 23, 2021, 4:33 am

I think it would be fair to say that the original goals of TTM, however expressed, are unlikely to be realised. SQL is centre stage, and probably has been since about the mid 1990s. So quo vadis TTM?

For managing and accessing a shared corporate application database, SQL serves well. For business application development accessing a shared database, modern GP languages such as Java and C# are sufficient. Despite the need for an ORM and 'glue' code the combination is good enough, and no-one is looking to replace it. For everything else, it has (or should have) competition.

Features that get me looking for something other than SQL+GP include:

embedded or in-process database (no server, not shared)

non-business data eg images, audio, time series, geographic, real-time

non-database data eg file hierarchies, CSV files, spreadsheets, documents

anytime setting up and maintaining a server does not seem feasible/justified

anytime writing SQL plus ORM/ODBC glue code does not seem feasible/justified.

There is competition, including:

SQLite: in-process, but still SQL+ORM

NoSQL: but you don't get relational queries

LINQ/Java Streams: but you don't get update

bare metal raw files plus various libraries.

So what does TTM offer? Stripping out the new language (which no-one seems to want), the main features seem to be:

A type system that can be applied to attributes

An in-language DQL based on the RA

Updates based on relational assignment

Database features including transactions and constraints.

So the addressable market is everyone using data that is not a good match for SQL+GP (because of any of the features listed above), especially those who are already using SQLite or NoSQL or raw files, and who would benefit from any of the TTM features. That's huge.

I agree. There are undoubtedly a vast number of tools that can be written -- in and/or for any number of different languages, platforms, and environments -- which embody one or more of the TTM & relational model & relational algebra ideas. SQL itself isn't yet ready to be replaced -- and promoting products solely on being faithful to the relational model doesn't work as nobody but us cares and the general assumption is that relational == SQL -- but languages, platforms, tools, utilities, you-name-it that do something good (i.e., that save/earn money and/or save effort and/or improve quality) and are based on TTM ideas even if not explicitly mentioned as such in the brochure?

Absolutely.

SQL is absolutely ready to be replaced and is being replaced for many of the above use cases. The stand-out is NoSQL, as the name implies. Client-side LINQ and Java Streams are being used to replace server-side SQL. Countless developers solving problems with the features listed above are cussing SQL and wishing for something better very day.

Yes people don't care about 'faithful' but they certainly care about solving their problems. What user/developer problem does TTM solve? The RM/RA are well known (ask Google) and SQL is a pain. For some X the pitch is:

solves problem X better than SQL
does absolutely everything you can do with X in SQL but better/faster/easier/cheaper
based on the RA/RM.

Now you have something to sell (for some X).

Andl - A New Database Language - andl.org

#23 · July 26, 2021, 7:12 am

Quote from Dave Voorhis on July 26, 2021, 7:12 am

Quote from dandl on July 26, 2021, 1:29 am

Quote from Dave Voorhis on July 23, 2021, 9:19 am

Quote from dandl on July 23, 2021, 4:33 am

I think it would be fair to say that the original goals of TTM, however expressed, are unlikely to be realised. SQL is centre stage, and probably has been since about the mid 1990s. So quo vadis TTM?

For managing and accessing a shared corporate application database, SQL serves well. For business application development accessing a shared database, modern GP languages such as Java and C# are sufficient. Despite the need for an ORM and 'glue' code the combination is good enough, and no-one is looking to replace it. For everything else, it has (or should have) competition.

Features that get me looking for something other than SQL+GP include:

embedded or in-process database (no server, not shared)

non-business data eg images, audio, time series, geographic, real-time

non-database data eg file hierarchies, CSV files, spreadsheets, documents

anytime setting up and maintaining a server does not seem feasible/justified

anytime writing SQL plus ORM/ODBC glue code does not seem feasible/justified.

There is competition, including:

SQLite: in-process, but still SQL+ORM

NoSQL: but you don't get relational queries

LINQ/Java Streams: but you don't get update

bare metal raw files plus various libraries.

So what does TTM offer? Stripping out the new language (which no-one seems to want), the main features seem to be:

A type system that can be applied to attributes

An in-language DQL based on the RA

Updates based on relational assignment

Database features including transactions and constraints.

So the addressable market is everyone using data that is not a good match for SQL+GP (because of any of the features listed above), especially those who are already using SQLite or NoSQL or raw files, and who would benefit from any of the TTM features. That's huge.

I agree. There are undoubtedly a vast number of tools that can be written -- in and/or for any number of different languages, platforms, and environments -- which embody one or more of the TTM & relational model & relational algebra ideas. SQL itself isn't yet ready to be replaced -- and promoting products solely on being faithful to the relational model doesn't work as nobody but us cares and the general assumption is that relational == SQL -- but languages, platforms, tools, utilities, you-name-it that do something good (i.e., that save/earn money and/or save effort and/or improve quality) and are based on TTM ideas even if not explicitly mentioned as such in the brochure?

Absolutely.

SQL is absolutely ready to be replaced and is being replaced for many of the above use cases. The stand-out is NoSQL, as the name implies. Client-side LINQ and Java Streams are being used to replace server-side SQL. Countless developers solving problems with the features listed above are cussing SQL and wishing for something better very day.

Yes people don't care about 'faithful' but they certainly care about solving their problems. What user/developer problem does TTM solve? The RM/RA are well known (ask Google) and SQL is a pain. For some X the pitch is:

solves problem X better than SQL

does absolutely everything you can do with X in SQL but better/faster/easier/cheaper

based on the RA/RM.

Now you have something to sell (for some X).

What I mean is that the world is not yet ready to embrace a MyTTM instead of MySQL, or a TTM Server instead of SQL Server, or a PostgreTTM instead of PostgreSQL, or a TTMLite instead of SQLite, etc.

Where NoSQL succeeds -- and isn't just a junior developer's choice over SQL because he's trying to avoid the stuff he found hard in college -- is where the relational model really doesn't apply, at least as we know it. Requirements for per-record dynamicity or document indexing -- where MongoDB or Cassandra or whatever are sound choices -- don't really suit TTM approaches or vice versa.

But when it comes to orchestrating containers or cloud services, or supporting machine learning, or providing custom data analytics, or supporting specialist operations in numerous vertical market niches, there is ample opportunity to produce new tools based on TTM principles and the relational model. They might not look anything like conventional DBMSs, but that's fine -- SQL can have them (and vice versa), because there are plenty of non-conventional possibly-non-database things that can benefit from TTM approaches.

As for LINQ and Streams being used to replace server-side SQL, that's often a junior developer's mistake. Indeed, one of the things I've been paid to do is optimise systems where developers have tried to retrieve multiple large tables from the database repeatedly (SELECT * FROM table;), and then (badly) manipulate them on the client side with Streams/LINQ/record-by-record, when they should be sending one query to the DBMS server side and letting it do the heavy lifting. Then LINQ/Streams become nice -- and appropriate -- tools to helpfully use the query result.

Quote from dandl on July 26, 2021, 1:29 am

Quote from Dave Voorhis on July 23, 2021, 9:19 am

Quote from dandl on July 23, 2021, 4:33 am

I think it would be fair to say that the original goals of TTM, however expressed, are unlikely to be realised. SQL is centre stage, and probably has been since about the mid 1990s. So quo vadis TTM?

For managing and accessing a shared corporate application database, SQL serves well. For business application development accessing a shared database, modern GP languages such as Java and C# are sufficient. Despite the need for an ORM and 'glue' code the combination is good enough, and no-one is looking to replace it. For everything else, it has (or should have) competition.

Features that get me looking for something other than SQL+GP include:

embedded or in-process database (no server, not shared)

non-business data eg images, audio, time series, geographic, real-time

non-database data eg file hierarchies, CSV files, spreadsheets, documents

anytime setting up and maintaining a server does not seem feasible/justified

anytime writing SQL plus ORM/ODBC glue code does not seem feasible/justified.

There is competition, including:

SQLite: in-process, but still SQL+ORM

NoSQL: but you don't get relational queries

LINQ/Java Streams: but you don't get update

bare metal raw files plus various libraries.

So what does TTM offer? Stripping out the new language (which no-one seems to want), the main features seem to be:

A type system that can be applied to attributes

An in-language DQL based on the RA

Updates based on relational assignment

Database features including transactions and constraints.

So the addressable market is everyone using data that is not a good match for SQL+GP (because of any of the features listed above), especially those who are already using SQLite or NoSQL or raw files, and who would benefit from any of the TTM features. That's huge.

I agree. There are undoubtedly a vast number of tools that can be written -- in and/or for any number of different languages, platforms, and environments -- which embody one or more of the TTM & relational model & relational algebra ideas. SQL itself isn't yet ready to be replaced -- and promoting products solely on being faithful to the relational model doesn't work as nobody but us cares and the general assumption is that relational == SQL -- but languages, platforms, tools, utilities, you-name-it that do something good (i.e., that save/earn money and/or save effort and/or improve quality) and are based on TTM ideas even if not explicitly mentioned as such in the brochure?

Absolutely.

SQL is absolutely ready to be replaced and is being replaced for many of the above use cases. The stand-out is NoSQL, as the name implies. Client-side LINQ and Java Streams are being used to replace server-side SQL. Countless developers solving problems with the features listed above are cussing SQL and wishing for something better very day.

Yes people don't care about 'faithful' but they certainly care about solving their problems. What user/developer problem does TTM solve? The RM/RA are well known (ask Google) and SQL is a pain. For some X the pitch is:

solves problem X better than SQL

does absolutely everything you can do with X in SQL but better/faster/easier/cheaper

based on the RA/RM.

Now you have something to sell (for some X).

What I mean is that the world is not yet ready to embrace a MyTTM instead of MySQL, or a TTM Server instead of SQL Server, or a PostgreTTM instead of PostgreSQL, or a TTMLite instead of SQLite, etc.

Where NoSQL succeeds -- and isn't just a junior developer's choice over SQL because he's trying to avoid the stuff he found hard in college -- is where the relational model really doesn't apply, at least as we know it. Requirements for per-record dynamicity or document indexing -- where MongoDB or Cassandra or whatever are sound choices -- don't really suit TTM approaches or vice versa.

But when it comes to orchestrating containers or cloud services, or supporting machine learning, or providing custom data analytics, or supporting specialist operations in numerous vertical market niches, there is ample opportunity to produce new tools based on TTM principles and the relational model. They might not look anything like conventional DBMSs, but that's fine -- SQL can have them (and vice versa), because there are plenty of non-conventional possibly-non-database things that can benefit from TTM approaches.

As for LINQ and Streams being used to replace server-side SQL, that's often a junior developer's mistake. Indeed, one of the things I've been paid to do is optimise systems where developers have tried to retrieve multiple large tables from the database repeatedly (SELECT * FROM table;), and then (badly) manipulate them on the client side with Streams/LINQ/record-by-record, when they should be sending one query to the DBMS server side and letting it do the heavy lifting. Then LINQ/Streams become nice -- and appropriate -- tools to helpfully use the query result.

I'm the forum administrator and lead developer of Rel. Email me at dave@armchair.mb.ca with the Subject 'TTM Forum'. Download Rel from https://reldb.org

#24 · July 26, 2021, 8:14 am

Quote from Dave Voorhis on July 26, 2021, 7:12 am

What I mean is that the world is not yet ready to embrace a MyTTM instead of MySQL, or a TTM Server instead of SQL Server, or a PostgreTTM instead of PostgreSQL, or a TTMLite instead of SQLite, etc.

Can we please say "D" rather than "TTM" in any discussions when we're substituting for "SQL" in cases like the above. To someone who knows what we're talking about your choice above reads terribly.

#25 · July 26, 2021, 8:29 am

Quote from Darren Duncan on July 26, 2021, 8:14 am

Quote from Dave Voorhis on July 26, 2021, 7:12 am

What I mean is that the world is not yet ready to embrace a MyTTM instead of MySQL, or a TTM Server instead of SQL Server, or a PostgreTTM instead of PostgreSQL, or a TTMLite instead of SQLite, etc.

Can we please say "D" rather than "TTM" in any discussions when we're substituting for "SQL" in cases like the above. To someone who knows what we're talking about your choice above reads terribly.

I... Guess...

D is only one letter. I was going for a recognisable three-letter substitute for maximum relevant equivalence.

I'm the forum administrator and lead developer of Rel. Email me at dave@armchair.mb.ca with the Subject 'TTM Forum'. Download Rel from https://reldb.org

#26 · July 26, 2021, 8:40 am

Quote from Dave Voorhis on July 26, 2021, 8:29 am

Quote from Darren Duncan on July 26, 2021, 8:14 am

Quote from Dave Voorhis on July 26, 2021, 7:12 am

What I mean is that the world is not yet ready to embrace a MyTTM instead of MySQL, or a TTM Server instead of SQL Server, or a PostgreTTM instead of PostgreSQL, or a TTMLite instead of SQLite, etc.

Can we please say "D" rather than "TTM" in any discussions when we're substituting for "SQL" in cases like the above. To someone who knows what we're talking about your choice above reads terribly.

I... Guess...

D is only one letter. I was going for a recognisable three-letter substitute for maximum relevant equivalence.

Though whilst I'm thinking of it... DLite is a great name, and I'm going to create it just because of that name.

Assuming there aren't a dozen projects/products already using it, of course.

I'm the forum administrator and lead developer of Rel. Email me at dave@armchair.mb.ca with the Subject 'TTM Forum'. Download Rel from https://reldb.org

#27 · July 26, 2021, 9:18 am

Quote from Dave Voorhis on July 26, 2021, 8:29 am

Quote from Darren Duncan on July 26, 2021, 8:14 am

Quote from Dave Voorhis on July 26, 2021, 7:12 am

What I mean is that the world is not yet ready to embrace a MyTTM instead of MySQL, or a TTM Server instead of SQL Server, or a PostgreTTM instead of PostgreSQL, or a TTMLite instead of SQLite, etc.

Can we please say "D" rather than "TTM" in any discussions when we're substituting for "SQL" in cases like the above. To someone who knows what we're talking about your choice above reads terribly.

I... Guess...

D is only one letter. I was going for a recognisable three-letter substitute for maximum relevant equivalence.

The point is that D is a language name like SQL while TTM is not.

Using TTM in names like that comes across like something an outsider would do who didn't really understand the difference between TTM and a D language.

A specific real life example your usage reminded me of years ago when on the SQLite mailing list someone proposed that SQLite adopt TTM and Richard Hipp was rather dismissive of doing so himself but that someone else was welcome to make a fork called "TTMLite".

#28 · July 26, 2021, 9:36 am

Quote from Darren Duncan on July 26, 2021, 9:18 am

Quote from Dave Voorhis on July 26, 2021, 8:29 am

Quote from Darren Duncan on July 26, 2021, 8:14 am

Quote from Dave Voorhis on July 26, 2021, 7:12 am

What I mean is that the world is not yet ready to embrace a MyTTM instead of MySQL, or a TTM Server instead of SQL Server, or a PostgreTTM instead of PostgreSQL, or a TTMLite instead of SQLite, etc.

Can we please say "D" rather than "TTM" in any discussions when we're substituting for "SQL" in cases like the above. To someone who knows what we're talking about your choice above reads terribly.

I... Guess...

D is only one letter. I was going for a recognisable three-letter substitute for maximum relevant equivalence.

The point is that D is a language name like SQL while TTM is not.

Using TTM in names like that comes across like something an outsider would do who didn't really understand the difference between TTM and a D language.

A specific real life example your usage reminded me of years ago when on the SQLite mailing list someone proposed that SQLite adopt TTM and Richard Hipp was rather dismissive of doing so himself but that someone else was welcome to make a fork called "TTMLite".

I think the meaning -- and intent -- was clear in my post.

I'm the forum administrator and lead developer of Rel. Email me at dave@armchair.mb.ca with the Subject 'TTM Forum'. Download Rel from https://reldb.org

#29 · July 26, 2021, 10:55 am

Quote from dandl on July 26, 2021, 10:55 am

Quote from Dave Voorhis on July 26, 2021, 7:12 am

Quote from dandl on July 26, 2021, 1:29 am

Quote from Dave Voorhis on July 23, 2021, 9:19 am

Quote from dandl on July 23, 2021, 4:33 am

I think it would be fair to say that the original goals of TTM, however expressed, are unlikely to be realised. SQL is centre stage, and probably has been since about the mid 1990s. So quo vadis TTM?

For managing and accessing a shared corporate application database, SQL serves well. For business application development accessing a shared database, modern GP languages such as Java and C# are sufficient. Despite the need for an ORM and 'glue' code the combination is good enough, and no-one is looking to replace it. For everything else, it has (or should have) competition.

Features that get me looking for something other than SQL+GP include:

embedded or in-process database (no server, not shared)

non-business data eg images, audio, time series, geographic, real-time

non-database data eg file hierarchies, CSV files, spreadsheets, documents

anytime setting up and maintaining a server does not seem feasible/justified

anytime writing SQL plus ORM/ODBC glue code does not seem feasible/justified.

There is competition, including:

SQLite: in-process, but still SQL+ORM

NoSQL: but you don't get relational queries

LINQ/Java Streams: but you don't get update

bare metal raw files plus various libraries.

So what does TTM offer? Stripping out the new language (which no-one seems to want), the main features seem to be:

A type system that can be applied to attributes

An in-language DQL based on the RA

Updates based on relational assignment

Database features including transactions and constraints.

So the addressable market is everyone using data that is not a good match for SQL+GP (because of any of the features listed above), especially those who are already using SQLite or NoSQL or raw files, and who would benefit from any of the TTM features. That's huge.

I agree. There are undoubtedly a vast number of tools that can be written -- in and/or for any number of different languages, platforms, and environments -- which embody one or more of the TTM & relational model & relational algebra ideas. SQL itself isn't yet ready to be replaced -- and promoting products solely on being faithful to the relational model doesn't work as nobody but us cares and the general assumption is that relational == SQL -- but languages, platforms, tools, utilities, you-name-it that do something good (i.e., that save/earn money and/or save effort and/or improve quality) and are based on TTM ideas even if not explicitly mentioned as such in the brochure?

Absolutely.

SQL is absolutely ready to be replaced and is being replaced for many of the above use cases. The stand-out is NoSQL, as the name implies. Client-side LINQ and Java Streams are being used to replace server-side SQL. Countless developers solving problems with the features listed above are cussing SQL and wishing for something better very day.

Yes people don't care about 'faithful' but they certainly care about solving their problems. What user/developer problem does TTM solve? The RM/RA are well known (ask Google) and SQL is a pain. For some X the pitch is:

solves problem X better than SQL

does absolutely everything you can do with X in SQL but better/faster/easier/cheaper

based on the RA/RM.

Now you have something to sell (for some X).

I'm going to delete every mention of a shared database as being off-topic.

What I mean is that the world is not yet ready to embrace ~~a MyTTM instead of MySQL, or a TTM Server instead of SQL Server, or a PostgreTTM instead of PostgreSQL, or a~~ TTMLite instead of SQLite, etc.

I think it is, if the specific (in-process) benefits are there: quicker, faster, easier, no SQL.

Where NoSQL succeeds ~~-- and isn't just a junior developer's choice over SQL because he's trying to avoid the stuff he found hard in college --~~ is where the relational model really doesn't apply, at least as we know it. Requirements for per-record dynamicity or document indexing ~~-- where MongoDB or Cassandra or whatever are sound choices --~~ don't really suit TTM approaches or vice versa.

No, the big reason NoSQL gets over-used is you get to write to local in-process storage in a single language, without having to fuss with servers or SQL. You find out how bad the query language is later.

But when it comes to orchestrating containers or cloud services, or supporting machine learning, or providing custom data analytics, or supporting specialist operations in numerous vertical market niches, there is ample opportunity to produce new tools based on TTM principles and the relational model. They might not look anything like conventional DBMSs, but that's fine -- SQL can have them (and vice versa), because there are plenty of non-conventional possibly-non-database things that can benefit from TTM approaches.

As for LINQ and Streams being used to replace server-side SQL, that's often a junior developer's mistake. Indeed, one of the things I've been paid to do is optimise systems where developers have tried to retrieve multiple large tables from the database repeatedly (SELECT * FROM table;), and then (badly) manipulate them on the client side with Streams/LINQ/record-by-record, when they should be sending one query to the DBMS server side and letting it do the heavy lifting. Then LINQ/Streams become nice -- and appropriate -- tools ~~to helpfully use the query result.~~

No, I use LINQ a lot and I almost never touch a shared database. LINQ gives you modest in-process SQL-like capabilities on in-process data, CSV files, XLS files, JSON, etc with no servers and no SQL.

The message is: for shared database stick to SQL; for key-value storage try NoSQL and good luck; but for everything else there is Relate-A-Lot (or whatever catchy name you can come up with).

Quote from Dave Voorhis on July 26, 2021, 7:12 am

Quote from dandl on July 26, 2021, 1:29 am

Quote from Dave Voorhis on July 23, 2021, 9:19 am

Quote from dandl on July 23, 2021, 4:33 am

I think it would be fair to say that the original goals of TTM, however expressed, are unlikely to be realised. SQL is centre stage, and probably has been since about the mid 1990s. So quo vadis TTM?

For managing and accessing a shared corporate application database, SQL serves well. For business application development accessing a shared database, modern GP languages such as Java and C# are sufficient. Despite the need for an ORM and 'glue' code the combination is good enough, and no-one is looking to replace it. For everything else, it has (or should have) competition.

Features that get me looking for something other than SQL+GP include:

embedded or in-process database (no server, not shared)

non-business data eg images, audio, time series, geographic, real-time

non-database data eg file hierarchies, CSV files, spreadsheets, documents

anytime setting up and maintaining a server does not seem feasible/justified

anytime writing SQL plus ORM/ODBC glue code does not seem feasible/justified.

There is competition, including:

SQLite: in-process, but still SQL+ORM

NoSQL: but you don't get relational queries

LINQ/Java Streams: but you don't get update

bare metal raw files plus various libraries.

So what does TTM offer? Stripping out the new language (which no-one seems to want), the main features seem to be:

A type system that can be applied to attributes

An in-language DQL based on the RA

Updates based on relational assignment

Database features including transactions and constraints.

So the addressable market is everyone using data that is not a good match for SQL+GP (because of any of the features listed above), especially those who are already using SQLite or NoSQL or raw files, and who would benefit from any of the TTM features. That's huge.

I agree. There are undoubtedly a vast number of tools that can be written -- in and/or for any number of different languages, platforms, and environments -- which embody one or more of the TTM & relational model & relational algebra ideas. SQL itself isn't yet ready to be replaced -- and promoting products solely on being faithful to the relational model doesn't work as nobody but us cares and the general assumption is that relational == SQL -- but languages, platforms, tools, utilities, you-name-it that do something good (i.e., that save/earn money and/or save effort and/or improve quality) and are based on TTM ideas even if not explicitly mentioned as such in the brochure?

Absolutely.

SQL is absolutely ready to be replaced and is being replaced for many of the above use cases. The stand-out is NoSQL, as the name implies. Client-side LINQ and Java Streams are being used to replace server-side SQL. Countless developers solving problems with the features listed above are cussing SQL and wishing for something better very day.

Yes people don't care about 'faithful' but they certainly care about solving their problems. What user/developer problem does TTM solve? The RM/RA are well known (ask Google) and SQL is a pain. For some X the pitch is:

solves problem X better than SQL

does absolutely everything you can do with X in SQL but better/faster/easier/cheaper

based on the RA/RM.

Now you have something to sell (for some X).

I'm going to delete every mention of a shared database as being off-topic.

What I mean is that the world is not yet ready to embrace ~~a MyTTM instead of MySQL, or a TTM Server instead of SQL Server, or a PostgreTTM instead of PostgreSQL, or a~~ TTMLite instead of SQLite, etc.

I think it is, if the specific (in-process) benefits are there: quicker, faster, easier, no SQL.

Where NoSQL succeeds ~~-- and isn't just a junior developer's choice over SQL because he's trying to avoid the stuff he found hard in college --~~ is where the relational model really doesn't apply, at least as we know it. Requirements for per-record dynamicity or document indexing ~~-- where MongoDB or Cassandra or whatever are sound choices --~~ don't really suit TTM approaches or vice versa.

No, the big reason NoSQL gets over-used is you get to write to local in-process storage in a single language, without having to fuss with servers or SQL. You find out how bad the query language is later.

But when it comes to orchestrating containers or cloud services, or supporting machine learning, or providing custom data analytics, or supporting specialist operations in numerous vertical market niches, there is ample opportunity to produce new tools based on TTM principles and the relational model. They might not look anything like conventional DBMSs, but that's fine -- SQL can have them (and vice versa), because there are plenty of non-conventional possibly-non-database things that can benefit from TTM approaches.

As for LINQ and Streams being used to replace server-side SQL, that's often a junior developer's mistake. Indeed, one of the things I've been paid to do is optimise systems where developers have tried to retrieve multiple large tables from the database repeatedly (SELECT * FROM table;), and then (badly) manipulate them on the client side with Streams/LINQ/record-by-record, when they should be sending one query to the DBMS server side and letting it do the heavy lifting. Then LINQ/Streams become nice -- and appropriate -- tools ~~to helpfully use the query result.~~

No, I use LINQ a lot and I almost never touch a shared database. LINQ gives you modest in-process SQL-like capabilities on in-process data, CSV files, XLS files, JSON, etc with no servers and no SQL.

The message is: for shared database stick to SQL; for key-value storage try NoSQL and good luck; but for everything else there is Relate-A-Lot (or whatever catchy name you can come up with).

Andl - A New Database Language - andl.org

#30 · July 26, 2021, 12:00 pm

Quote from Dave Voorhis on July 26, 2021, 12:00 pm

Quote from dandl on July 26, 2021, 10:55 am

Quote from Dave Voorhis on July 26, 2021, 7:12 am

Quote from dandl on July 26, 2021, 1:29 am

Quote from Dave Voorhis on July 23, 2021, 9:19 am

Quote from dandl on July 23, 2021, 4:33 am

I think it would be fair to say that the original goals of TTM, however expressed, are unlikely to be realised. SQL is centre stage, and probably has been since about the mid 1990s. So quo vadis TTM?

For managing and accessing a shared corporate application database, SQL serves well. For business application development accessing a shared database, modern GP languages such as Java and C# are sufficient. Despite the need for an ORM and 'glue' code the combination is good enough, and no-one is looking to replace it. For everything else, it has (or should have) competition.

Features that get me looking for something other than SQL+GP include:

embedded or in-process database (no server, not shared)

non-business data eg images, audio, time series, geographic, real-time

non-database data eg file hierarchies, CSV files, spreadsheets, documents

anytime setting up and maintaining a server does not seem feasible/justified

anytime writing SQL plus ORM/ODBC glue code does not seem feasible/justified.

There is competition, including:

SQLite: in-process, but still SQL+ORM

NoSQL: but you don't get relational queries

LINQ/Java Streams: but you don't get update

bare metal raw files plus various libraries.

So what does TTM offer? Stripping out the new language (which no-one seems to want), the main features seem to be:

A type system that can be applied to attributes

An in-language DQL based on the RA

Updates based on relational assignment

Database features including transactions and constraints.

So the addressable market is everyone using data that is not a good match for SQL+GP (because of any of the features listed above), especially those who are already using SQLite or NoSQL or raw files, and who would benefit from any of the TTM features. That's huge.

I agree. There are undoubtedly a vast number of tools that can be written -- in and/or for any number of different languages, platforms, and environments -- which embody one or more of the TTM & relational model & relational algebra ideas. SQL itself isn't yet ready to be replaced -- and promoting products solely on being faithful to the relational model doesn't work as nobody but us cares and the general assumption is that relational == SQL -- but languages, platforms, tools, utilities, you-name-it that do something good (i.e., that save/earn money and/or save effort and/or improve quality) and are based on TTM ideas even if not explicitly mentioned as such in the brochure?

Absolutely.

SQL is absolutely ready to be replaced and is being replaced for many of the above use cases. The stand-out is NoSQL, as the name implies. Client-side LINQ and Java Streams are being used to replace server-side SQL. Countless developers solving problems with the features listed above are cussing SQL and wishing for something better very day.

Yes people don't care about 'faithful' but they certainly care about solving their problems. What user/developer problem does TTM solve? The RM/RA are well known (ask Google) and SQL is a pain. For some X the pitch is:

solves problem X better than SQL

does absolutely everything you can do with X in SQL but better/faster/easier/cheaper

based on the RA/RM.

Now you have something to sell (for some X).

I'm going to delete every mention of a shared database as being off-topic.

What I mean is that the world is not yet ready to embrace ~~a MyTTM instead of MySQL, or a TTM Server instead of SQL Server, or a PostgreTTM instead of PostgreSQL, or a~~ TTMLite instead of SQLite, etc.

I think it is, if the specific (in-process) benefits are there: quicker, faster, easier, no SQL.

Where NoSQL succeeds ~~-- and isn't just a junior developer's choice over SQL because he's trying to avoid the stuff he found hard in college --~~ is where the relational model really doesn't apply, at least as we know it. Requirements for per-record dynamicity or document indexing ~~-- where MongoDB or Cassandra or whatever are sound choices --~~ don't really suit TTM approaches or vice versa.

No, the big reason NoSQL gets over-used is you get to write to local in-process storage in a single language, without having to fuss with servers or SQL. You find out how bad the query language is later.

But when it comes to orchestrating containers or cloud services, or supporting machine learning, or providing custom data analytics, or supporting specialist operations in numerous vertical market niches, there is ample opportunity to produce new tools based on TTM principles and the relational model. They might not look anything like conventional DBMSs, but that's fine -- SQL can have them (and vice versa), because there are plenty of non-conventional possibly-non-database things that can benefit from TTM approaches.

As for LINQ and Streams being used to replace server-side SQL, that's often a junior developer's mistake. Indeed, one of the things I've been paid to do is optimise systems where developers have tried to retrieve multiple large tables from the database repeatedly (SELECT * FROM table;), and then (badly) manipulate them on the client side with Streams/LINQ/record-by-record, when they should be sending one query to the DBMS server side and letting it do the heavy lifting. Then LINQ/Streams become nice -- and appropriate -- tools ~~to helpfully use the query result.~~

No, I use LINQ a lot and I almost never touch a shared database. LINQ gives you modest in-process SQL-like capabilities on in-process data, CSV files, XLS files, JSON, etc with no servers and no SQL.

The message is: for shared database stick to SQL; for key-value storage try NoSQL and good luck; but for everything else there is Relate-A-Lot (or whatever catchy name you can come up with).

No, for everything else there is LINQ and Streams.

You don't need Relate-A-Lot to replace LINQ and Streams because there is LINQ and Streams.

But that doesn't mean TTM ideas don't have a role. They do. For orchestrating containers or cloud services, supporting machine learning, providing custom data analytics, or supporting specialist operations in numerous vertical market niches. (I have a little, tiny bit of insider knowledge here... This is a big set of markets, and if TTM fans like us don't step up to the plate, we're going to get beaten to it by the Datalog fans.)

As a replacement for anything any SQL does in a dominant way -- whether shared or single user -- or for LINQ and Streams in general?

No, not really.

Or only in a small way, and you'll kick yourself later for missing out on opportunities in containers or cloud services, supporting machine learning, providing custom data analytics, supporting specialist operations in numerous vertical market niches, and almost certainly dozens, maybe hundreds, of other places I haven't mentioned.

Quote from dandl on July 26, 2021, 10:55 am

Quote from Dave Voorhis on July 26, 2021, 7:12 am

Quote from dandl on July 26, 2021, 1:29 am

Quote from Dave Voorhis on July 23, 2021, 9:19 am

Quote from dandl on July 23, 2021, 4:33 am

I think it would be fair to say that the original goals of TTM, however expressed, are unlikely to be realised. SQL is centre stage, and probably has been since about the mid 1990s. So quo vadis TTM?

For managing and accessing a shared corporate application database, SQL serves well. For business application development accessing a shared database, modern GP languages such as Java and C# are sufficient. Despite the need for an ORM and 'glue' code the combination is good enough, and no-one is looking to replace it. For everything else, it has (or should have) competition.

Features that get me looking for something other than SQL+GP include:

embedded or in-process database (no server, not shared)

non-business data eg images, audio, time series, geographic, real-time

non-database data eg file hierarchies, CSV files, spreadsheets, documents

anytime setting up and maintaining a server does not seem feasible/justified

anytime writing SQL plus ORM/ODBC glue code does not seem feasible/justified.

There is competition, including:

SQLite: in-process, but still SQL+ORM

NoSQL: but you don't get relational queries

LINQ/Java Streams: but you don't get update

bare metal raw files plus various libraries.

So what does TTM offer? Stripping out the new language (which no-one seems to want), the main features seem to be:

A type system that can be applied to attributes

An in-language DQL based on the RA

Updates based on relational assignment

Database features including transactions and constraints.

So the addressable market is everyone using data that is not a good match for SQL+GP (because of any of the features listed above), especially those who are already using SQLite or NoSQL or raw files, and who would benefit from any of the TTM features. That's huge.

I agree. There are undoubtedly a vast number of tools that can be written -- in and/or for any number of different languages, platforms, and environments -- which embody one or more of the TTM & relational model & relational algebra ideas. SQL itself isn't yet ready to be replaced -- and promoting products solely on being faithful to the relational model doesn't work as nobody but us cares and the general assumption is that relational == SQL -- but languages, platforms, tools, utilities, you-name-it that do something good (i.e., that save/earn money and/or save effort and/or improve quality) and are based on TTM ideas even if not explicitly mentioned as such in the brochure?

Absolutely.

SQL is absolutely ready to be replaced and is being replaced for many of the above use cases. The stand-out is NoSQL, as the name implies. Client-side LINQ and Java Streams are being used to replace server-side SQL. Countless developers solving problems with the features listed above are cussing SQL and wishing for something better very day.

Yes people don't care about 'faithful' but they certainly care about solving their problems. What user/developer problem does TTM solve? The RM/RA are well known (ask Google) and SQL is a pain. For some X the pitch is:

solves problem X better than SQL

does absolutely everything you can do with X in SQL but better/faster/easier/cheaper

based on the RA/RM.

Now you have something to sell (for some X).

I'm going to delete every mention of a shared database as being off-topic.

What I mean is that the world is not yet ready to embrace ~~a MyTTM instead of MySQL, or a TTM Server instead of SQL Server, or a PostgreTTM instead of PostgreSQL, or a~~ TTMLite instead of SQLite, etc.

I think it is, if the specific (in-process) benefits are there: quicker, faster, easier, no SQL.

Where NoSQL succeeds ~~-- and isn't just a junior developer's choice over SQL because he's trying to avoid the stuff he found hard in college --~~ is where the relational model really doesn't apply, at least as we know it. Requirements for per-record dynamicity or document indexing ~~-- where MongoDB or Cassandra or whatever are sound choices --~~ don't really suit TTM approaches or vice versa.

No, the big reason NoSQL gets over-used is you get to write to local in-process storage in a single language, without having to fuss with servers or SQL. You find out how bad the query language is later.

But when it comes to orchestrating containers or cloud services, or supporting machine learning, or providing custom data analytics, or supporting specialist operations in numerous vertical market niches, there is ample opportunity to produce new tools based on TTM principles and the relational model. They might not look anything like conventional DBMSs, but that's fine -- SQL can have them (and vice versa), because there are plenty of non-conventional possibly-non-database things that can benefit from TTM approaches.

As for LINQ and Streams being used to replace server-side SQL, that's often a junior developer's mistake. Indeed, one of the things I've been paid to do is optimise systems where developers have tried to retrieve multiple large tables from the database repeatedly (SELECT * FROM table;), and then (badly) manipulate them on the client side with Streams/LINQ/record-by-record, when they should be sending one query to the DBMS server side and letting it do the heavy lifting. Then LINQ/Streams become nice -- and appropriate -- tools ~~to helpfully use the query result.~~

No, I use LINQ a lot and I almost never touch a shared database. LINQ gives you modest in-process SQL-like capabilities on in-process data, CSV files, XLS files, JSON, etc with no servers and no SQL.

The message is: for shared database stick to SQL; for key-value storage try NoSQL and good luck; but for everything else there is Relate-A-Lot (or whatever catchy name you can come up with).

No, for everything else there is LINQ and Streams.

You don't need Relate-A-Lot to replace LINQ and Streams because there is LINQ and Streams.

But that doesn't mean TTM ideas don't have a role. They do. For orchestrating containers or cloud services, supporting machine learning, providing custom data analytics, or supporting specialist operations in numerous vertical market niches. (I have a little, tiny bit of insider knowledge here... This is a big set of markets, and if TTM fans like us don't step up to the plate, we're going to get beaten to it by the Datalog fans.)

As a replacement for anything any SQL does in a dominant way -- whether shared or single user -- or for LINQ and Streams in general?

No, not really.

Or only in a small way, and you'll kick yourself later for missing out on opportunities in containers or cloud services, supporting machine learning, providing custom data analytics, supporting specialist operations in numerous vertical market niches, and almost certainly dozens, maybe hundreds, of other places I haven't mentioned.

I'm the forum administrator and lead developer of Rel. Email me at dave@armchair.mb.ca with the Subject 'TTM Forum'. Download Rel from https://reldb.org

The Forum for Discussion about The Third Manifesto and Related Matters