Life after D
Quote from Dave Voorhis on March 27, 2021, 10:38 amQuote from dandl on March 27, 2021, 9:52 amM is a meta-programming language, for writing programs that read and write programs.
Template Haskell (see below) does not read programs; neither does it 'write' programs in the sense of generating source code.
I am highly confident that you cannot carry out the 4 tasks I outlined (and many others) with non-M "languages with desirable language features that compile (of which transpilation is a subset) to an object language target". If you are hostile to M/MJ, you will find workarounds and partial solutions, but I am confident that no matter what you propose, I can produce an example that M/MJ can solve and that what you propose cannot. If you are using any tools of any kind to generate or modify Java source code you are already acknowledging the need for MJ, but those are just the tip of the iceberg. Building serious software needs M.
I intend to follow up by finding others who are successfully using meta-programming and languages that show how it can be done.
Re "successfully using meta-programming and languages that show how it can be done," take a look at Nim (it even has macros!) and Haxe (with macros! https://haxe.org/manual/macro.html)
Though their macros are not text-replacement macros.
'Template Haskell' is also not text-replacement macros. Rather, it generates bits of AST that slot into the place of the template 'call'. The functions you call are actual Haskell-written-and-compiled functions, with a special result type of AST -- either declarations or expressions.
The template functions being written in Haskell means they can call Haskell library methods for reflection and generics. Also that the functions are type-checked and their call sites use standard type inference/overloading. Then you can't produce a template call that generates an ill-typed AST.
Templating also includes the ability to generate fresh names or names 'decorated' from names of artefacts in the program; and to pass those generated names around to other template calls.
Similar to what Dave has already identified up-thread; there's a general feeling in the Haskell community that templating is a 'last resort'/admission of failings in the language. There's a steady stream of proposals to incorporate frequently-used templating idioms into the language proper.
The heaviest use-cases for templating are ... fancy record systems. Haskell's standard records/field labelling are universally seen as a weakness. It's just that nobody can agree how to improve.
Quote from dandl on March 27, 2021, 9:52 amM is a meta-programming language, for writing programs that read and write programs. Macros are a subset of M.
MJ is M for Java. I have just learned about the Java annotations processor, which seems to be a clunky subset of MJ. Java has nothing else.
I am highly confident that you cannot carry out the 4 tasks I outlined (and many others) with non-M "languages with desirable language features that compile (of which transpilation is a subset) to an object language target". If you are hostile to M/MJ, you will find workarounds and partial solutions, but I am confident that no matter what you propose, I can produce an example that M/MJ can solve and that what you propose cannot. If you are using any tools of any kind to generate or modify Java source code you are already acknowledging the need for MJ, but those are just the tip of the iceberg. Building serious software needs M.
I intend to follow up by finding others who are successfully using meta-programming and languages that show how it can be done.
I was rather surprised when the talk about "shorter-safer-higher" ended up in meta-programming. Shorter, sure, higher, I suppose, but safer? Most developers have a hard enough time producing straight first-order procedural code, never mind thinking about code that makes code that does something. So, sure, if you can produce a system that is usable, all power to you! (As a side note, the talk of the language you need vs the language you have made me think of Guy Steele's excellent classic talk on "growing a language" https://www.cs.virginia.edu/~evans/cs655/readings/steele.pdf or if you prefer to watch a video https://www.youtube.com/watch?v=_ahvzDzKdB0 )
<aside>As an anecdote, in about 2004 I built a system where the html page could be suitably annotated and then transformed by one XSLT template into another XSLT template that would be used to transform the XML data into a page with data. Hugely successful because the business people came in one week before going live with a new html page that we could just annotate and jack in. But the other developers found it ridiculously difficult so they ripped it out in favour of ASP.NET pages. They weren't even able to comprehend how XSLT worked, while the business people easily learned to produce XSLT templates (I held classes for both categories). The developers were too stuck in thoughts like "<html> is the command that begins an html document".</aside>
There is a lot of meta-programming in use today, much of it has been mentioned in threads here as Spring, Lombok, JPA and more. Spring used to be configured (and still can be) by xml files, but it is easier and better to use annotations as a virtual sort of code-transforming function (and Lombok and JPA do the same). Some of it is fairly useful, like an "@Post" annotation in Spring taking care of all the tedium of dealing with the http-protocol and just fills in the expected data into your method signature. Although if you just use an embedded Undertow server, you have a nice OO way of doing it instead, without the huge overhead of the Spring framework. But of course the codeshitters (love the term, Erwin!) find it much more interesting to produce frameworks for other codeshitters than to produce working software for real users.
Now these meta-programming constructs are just for consumption by the general programmer, it is not an easy thing to get into producing them. Spring has taken care to always provide ways to customize the behaviour when it doesn't quite fit your needs, but even that is not easy and is often severely underdocumented.
AOP was intended to be a general way to add cross-cutting concerns, but on the C2 wiki even the supporters of it concede that there usually are better object-oriented ways of doing it. It doesn't look too difficult to do, but there seem to be some inherent problems in how they get applied and composed. In functional-land I think they would use Monads instead which are composable language constructs and so much better. And I suppose the ML module system is also an excellent programming construct for this kind of thing (but I admit I have still a very hazy idea of how that works).
If you're interested in a programming system that is built for creating a domain-specific language for every use, take a look at the Meta Programming System https://www.jetbrains.com/mps/
As regards "shorter-safer-higher" why would you write writeln Hello World! when you can just write Hello World! ? Even if it is not strictly valid html, any browser will display it as such and if you go back and study my html form example you should be able to see that it is a near-optimal representation of an input-form, which makes html demonstrably shorter than pretty much any alternative for displaying forms or semi-structured text data. It is also safer because that is all you can express with it, as well as being higher because there is almost no implementation detail at all, just what you want to display, tagged just enough to allow better interpretation of purpose and to enable variation of display properties.
Another shorter-safer-higher you have expressed a liking for is PEG parser syntax.
And of course regex is now a built-in part of every modern language. (although I don't know how safe everybody feels with it!)
Which brings me to the point that perhaps we shouldn't be looking for the one true language, or perhaps that the one true language will have sub-languages to more efficiently and clearly (and safer) express certain parts of the code. The trick would then be to fit that all together without having to undertake huge shifts in your mental model as happens in languages where you have tried to combine paradigms like object-functional in Scala or functional-logical in Shen (which even has two different types of logic programming, sequent logic for the type system and also prolog). Although Shen does strive in the right direction, IMO, by pattern-matching syntax and a built-in parser syntax.
As much as I like the shorter-safer-higher Browser-only "Hello, World", I feel we can benefit even further with a macro definition.
Thus, I hereby program your brain with a mental macro: From now on, whenever you see a full-stop aka period -- . -- anywhere in one of my posts, whether on its own or at the end of a sentence or part of an ellipsis or as dotted notation in some code, you will think "Hello, World."
That's about as short-safe-higher as you can get, methinks.
And again: .
.
Quote from Dave Voorhis on March 27, 2021, 10:38 amQuote from dandl on March 27, 2021, 9:52 amM is a meta-programming language, for writing programs that read and write programs. Macros are a subset of M.
MJ is M for Java. I have just learned about the Java annotations processor, which seems to be a clunky subset of MJ. Java has nothing else.
I am highly confident that you cannot carry out the 4 tasks I outlined (and many others) with non-M "languages with desirable language features that compile (of which transpilation is a subset) to an object language target". If you are hostile to M/MJ, you will find workarounds and partial solutions, but I am confident that no matter what you propose, I can produce an example that M/MJ can solve and that what you propose cannot. If you are using any tools of any kind to generate or modify Java source code you are already acknowledging the need for MJ, but those are just the tip of the iceberg. Building serious software needs M.
I intend to follow up by finding others who are successfully using meta-programming and languages that show how it can be done.
I don't think M exists yet, let alone MJ, so it's a bit difficult to evaluate it (except from your arm-waving about it, I guess.)
Using tools to generate (not modify) Java source code is not a need for MJ, but simply a recognition that pre-existing static structures in external definitions -- like XML DTDs or SQL schemas or RESTful API specifications -- may benefit from having static Java class or record analogues, and creating them can be automated.
That isn't a need for a distinct language of any kind; that's simply integration of pre-existing external data definitions into Java. You typically aren't writing the XML DTDs or SQL schemas or RESTful API specifications because they exist already (and exposing Java as a RESTful API is a good and proper use of Java annotations.)
You're merely running this Java program that's pointed to them, and Java code appears over there as if you'd written it by hand. You're still working entirely -- and desirably -- in a Java domain.
Any sufficiently complicated Java program contains an ad hoc, informally-specified, bug-ridden, slow implementation of parts of MJ.
Re "successfully using meta-programming and languages that show how it can be done," take a look at Nim (it even has macros!) and Haxe (with macros! https://haxe.org/manual/macro.html)
Though their macros are not text-replacement macros.
Yes, those are two languages I would look to for inspiration. They do indeed have macros, of the kind I had in mind even if you didn't think so.
Now take another look at your Java code generation and tell me if you could code that inline with macros like Nim or Haxe.
The only constraint I would place is that the developer should easily be able to see the expanded code in a readable format, preferably in the editor. Debugging macros without knowing exactly what the compiler turned it into is too hard.
Quote from AntC on March 27, 2021, 11:44 amQuote from Dave Voorhis on March 27, 2021, 10:38 amQuote from dandl on March 27, 2021, 9:52 amM is a meta-programming language, for writing programs that read and write programs.
Template Haskell (see below) does not read programs; neither does it 'write' programs in the sense of generating source code.
I am highly confident that you cannot carry out the 4 tasks I outlined (and many others) with non-M "languages with desirable language features that compile (of which transpilation is a subset) to an object language target". If you are hostile to M/MJ, you will find workarounds and partial solutions, but I am confident that no matter what you propose, I can produce an example that M/MJ can solve and that what you propose cannot. If you are using any tools of any kind to generate or modify Java source code you are already acknowledging the need for MJ, but those are just the tip of the iceberg. Building serious software needs M.
I intend to follow up by finding others who are successfully using meta-programming and languages that show how it can be done.
Re "successfully using meta-programming and languages that show how it can be done," take a look at Nim (it even has macros!) and Haxe (with macros! https://haxe.org/manual/macro.html)
Though their macros are not text-replacement macros.
'Template Haskell' is also not text-replacement macros. Rather, it generates bits of AST that slot into the place of the template 'call'. The functions you call are actual Haskell-written-and-compiled functions, with a special result type of AST -- either declarations or expressions.
The template functions being written in Haskell means they can call Haskell library methods for reflection and generics. Also that the functions are type-checked and their call sites use standard type inference/overloading. Then you can't produce a template call that generates an ill-typed AST.
Templating also includes the ability to generate fresh names or names 'decorated' from names of artefacts in the program; and to pass those generated names around to other template calls.
As you describe it Haskell templates could probably do most of what I have in mind, but it would seem they rely a lot of Haskell specifics.
Similar to what Dave has already identified up-thread; there's a general feeling in the Haskell community that templating is a 'last resort'/admission of failings in the language. There's a steady stream of proposals to incorporate frequently-used templating idioms into the language proper.
The heaviest use-cases for templating are ... fancy record systems. Haskell's standard records/field labelling are universally seen as a weakness. It's just that nobody can agree how to improve.
That's not one I'm interested in pursuing. My proposition is that M plus a widely used GP language (Java, say) can achieve short-safer-higher, where adding in-compiler features cannot.
Quote from tobega on March 27, 2021, 1:50 pmQuote from dandl on March 27, 2021, 9:52 amM is a meta-programming language, for writing programs that read and write programs. Macros are a subset of M.
MJ is M for Java. I have just learned about the Java annotations processor, which seems to be a clunky subset of MJ. Java has nothing else.
I am highly confident that you cannot carry out the 4 tasks I outlined (and many others) with non-M "languages with desirable language features that compile (of which transpilation is a subset) to an object language target". If you are hostile to M/MJ, you will find workarounds and partial solutions, but I am confident that no matter what you propose, I can produce an example that M/MJ can solve and that what you propose cannot. If you are using any tools of any kind to generate or modify Java source code you are already acknowledging the need for MJ, but those are just the tip of the iceberg. Building serious software needs M.
I intend to follow up by finding others who are successfully using meta-programming and languages that show how it can be done.
I was rather surprised when the talk about "shorter-safer-higher" ended up in meta-programming. Shorter, sure, higher, I suppose, but safer? Most developers have a hard enough time producing straight first-order procedural code, never mind thinking about code that makes code that does something. So, sure, if you can produce a system that is usable, all power to you! (As a side note, the talk of the language you need vs the language you have made me think of Guy Steele's excellent classic talk on "growing a language" https://www.cs.virginia.edu/~evans/cs655/readings/steele.pdf or if you prefer to watch a video https://www.youtube.com/watch?v=_ahvzDzKdB0 )
<aside>As an anecdote, in about 2004 I built a system where the html page could be suitably annotated and then transformed by one XSLT template into another XSLT template that would be used to transform the XML data into a page with data. Hugely successful because the business people came in one week before going live with a new html page that we could just annotate and jack in. But the other developers found it ridiculously difficult so they ripped it out in favour of ASP.NET pages. They weren't even able to comprehend how XSLT worked, while the business people easily learned to produce XSLT templates (I held classes for both categories). The developers were too stuck in thoughts like "<html> is the command that begins an html document".</aside>
There is a lot of meta-programming in use today, much of it has been mentioned in threads here as Spring, Lombok, JPA and more. Spring used to be configured (and still can be) by xml files, but it is easier and better to use annotations as a virtual sort of code-transforming function (and Lombok and JPA do the same). Some of it is fairly useful, like an "@Post" annotation in Spring taking care of all the tedium of dealing with the http-protocol and just fills in the expected data into your method signature. Although if you just use an embedded Undertow server, you have a nice OO way of doing it instead, without the huge overhead of the Spring framework. But of course the codeshitters (love the term, Erwin!) find it much more interesting to produce frameworks for other codeshitters than to produce working software for real users.
Now these meta-programming constructs are just for consumption by the general programmer, it is not an easy thing to get into producing them. Spring has taken care to always provide ways to customize the behaviour when it doesn't quite fit your needs, but even that is not easy and is often severely underdocumented.
Your points are well made, and mesh nicely with my concept of the tool maker vs tool builder. In my experience tool makers are about 1 programmer in 10. 9 of 10 want to work 'concretely' with the end product using whatever tools they have and can understand and rely on. I belong to the 1 in 10 that prefer to think: what tool can I build to make this job much easier? Maybe you do too. [A few like Dave straddle the fence.]
But it behoves us tool-makes to put a complete, working, finished, reliable tool in the hands of the tool users, and you didn't do that. Your XSLT page was not a finished, reliable tool. Spring falls short if it's not documented. I wrote nearly 2000 pages of documentation for Powerflex, and it still falls short in places. [But I rely on it heavily for the bits I've forgotten.]
AOP was intended to be a general way to add cross-cutting concerns, but on the C2 wiki even the supporters of it concede that there usually are better object-oriented ways of doing it. It doesn't look too difficult to do, but there seem to be some inherent problems in how they get applied and composed. In functional-land I think they would use Monads instead which are composable language constructs and so much better. And I suppose the ML module system is also an excellent programming construct for this kind of thing (but I admit I have still a very hazy idea of how that works).
I know AOP but this is wrong. Case in point: I want a small library of tracing/logging functions I can add to my C# programs, with the following features:
- It produces nicely formatted text output including options for date, time, argument values, etc
- It uses the minimum amount of space on the output device (collapse/merge similar messages)
- It can send output to the console, window, text file, debugger or stream
- It works on any platform: Windows console/UI/DLL/service, Unity, Unix, etc
- Trace calls can be enabled/disabled at runtime by program or configuration
- Disabled trace calls are 'free' (little or no CPU impact)
- Trace calls can be excluded from a production build
You can't do that with objects, but you can do it with C++ templates and macros.
If you're interested in a programming system that is built for creating a domain-specific language for every use, take a look at the Meta Programming System https://www.jetbrains.com/mps/
I looked at this a while back and didn't like it, but I don't remember why. I'll look again.
As regards "shorter-safer-higher" why would you write
writeln Hello World!
writeln Hello World!when you can just writeHello World!
Hello World!? Even if it is not strictly valid html, any browser will display it as such and if you go back and study my html form example you should be able to see that it is a near-optimal representation of an input-form, which makes html demonstrably shorter than pretty much any alternative for displaying forms or semi-structured text data. It is also safer because that is all you can express with it, as well as being higher because there is almost no implementation detail at all, just what you want to display, tagged just enough to allow better interpretation of purpose and to enable variation of display properties.
I'm not sure if you're serious, but yes, if that's the way you want your programs to look, that too should be possible.
Another shorter-safer-higher you have expressed a liking for is PEG parser syntax.
And of course regex is now a built-in part of every modern language. (although I don't know how safe everybody feels with it!)
Regex is awful, but indispensable.
Which brings me to the point that perhaps we shouldn't be looking for the one true language, or perhaps that the one true language will have sub-languages to more efficiently and clearly (and safer) express certain parts of the code. The trick would then be to fit that all together without having to undertake huge shifts in your mental model as happens in languages where you have tried to combine paradigms like object-functional in Scala or functional-logical in Shen (which even has two different types of logic programming, sequent logic for the type system and also prolog). Although Shen does strive in the right direction, IMO, by pattern-matching syntax and a built-in parser syntax.
There is no one true language, but there are languages in widespread use that make tool-users comfortable. Better to find someone working wood with chisel and plane and give them power tools than try to get them using metal or plastic instead.
Quote from dandl on March 27, 2021, 11:21 pmQuote from Dave Voorhis on March 27, 2021, 10:38 amQuote from dandl on March 27, 2021, 9:52 amM is a meta-programming language, for writing programs that read and write programs. Macros are a subset of M.
MJ is M for Java. I have just learned about the Java annotations processor, which seems to be a clunky subset of MJ. Java has nothing else.
I am highly confident that you cannot carry out the 4 tasks I outlined (and many others) with non-M "languages with desirable language features that compile (of which transpilation is a subset) to an object language target". If you are hostile to M/MJ, you will find workarounds and partial solutions, but I am confident that no matter what you propose, I can produce an example that M/MJ can solve and that what you propose cannot. If you are using any tools of any kind to generate or modify Java source code you are already acknowledging the need for MJ, but those are just the tip of the iceberg. Building serious software needs M.
I intend to follow up by finding others who are successfully using meta-programming and languages that show how it can be done.
I don't think M exists yet, let alone MJ, so it's a bit difficult to evaluate it (except from your arm-waving about it, I guess.)
Using tools to generate (not modify) Java source code is not a need for MJ, but simply a recognition that pre-existing static structures in external definitions -- like XML DTDs or SQL schemas or RESTful API specifications -- may benefit from having static Java class or record analogues, and creating them can be automated.
That isn't a need for a distinct language of any kind; that's simply integration of pre-existing external data definitions into Java. You typically aren't writing the XML DTDs or SQL schemas or RESTful API specifications because they exist already (and exposing Java as a RESTful API is a good and proper use of Java annotations.)
You're merely running this Java program that's pointed to them, and Java code appears over there as if you'd written it by hand. You're still working entirely -- and desirably -- in a Java domain.
Any sufficiently complicated Java program contains an ad hoc, informally-specified, bug-ridden, slow implementation of parts of MJ.
I don't think anything contains any kind of implementation of MJ because it doesn't exist, even as a specification. Thus far, it's not sufficiently defined to be considered more than a vague idea.
Indeed, it reminds me of a presentation I saw many, many years ago on the forthcoming dBase IV, which an Ashton-Tate presenter introduced not by showing any prototype or work-in-progress product or feature list or specification, but by asking the audience of dBase III application developers what features they'd like to see in dBase IV.
The answer to every request, no matter what, was "It does that!"
Of course, as it turned out, it didn't.
Re "successfully using meta-programming and languages that show how it can be done," take a look at Nim (it even has macros!) and Haxe (with macros! https://haxe.org/manual/macro.html)
Though their macros are not text-replacement macros.
Yes, those are two languages I would look to for inspiration. They do indeed have macros, of the kind I had in mind even if you didn't think so.
Now take another look at your Java code generation and tell me if you could code that inline with macros like Nim or Haxe.
My own Java code generation -- and others that I use -- are based on pre-existing XML DTD, SQL schemas, resultsets for specified SQL queries, and RESTful API specs like WSDL. It's pre-existing standard-format specification in; Java out. As such, whilst Nim and Haxe are interesting in and of themselves, they don't serve a purpose for the work I do.
If a hypothetical M or MJ will solve problems in your domain, excellent.
But based on the idea of it, I don't see it solving problems in mine. Though I'm happy to defer final evaluation to the point at which it exists.
The only constraint I would place is that the developer should easily be able to see the expanded code in a readable format, preferably in the editor. Debugging macros without knowing exactly what the compiler turned it into is too hard.
Yes, though note that whilst this is helpful to the professional programmer, it's unlikely to be helpful to the semi-technical developer that I understand you're targeting. If he's using M to generate the code he doesn't want to -- or can't -- write, I'm not sure what will be gained by showing him the code he wouldn't or couldn't write.
Better, I think, that the language semantics be self-standing and regulated, so that the compiler can't be (mis)led into generating code that demands viewing in order to make sense of it (except if you're the compiler author, of course.)
Quote from dandl on March 28, 2021, 12:00 amQuote from tobega on March 27, 2021, 1:50 pmQuote from dandl on March 27, 2021, 9:52 amM is a meta-programming language, for writing programs that read and write programs. Macros are a subset of M.
MJ is M for Java. I have just learned about the Java annotations processor, which seems to be a clunky subset of MJ. Java has nothing else.
I am highly confident that you cannot carry out the 4 tasks I outlined (and many others) with non-M "languages with desirable language features that compile (of which transpilation is a subset) to an object language target". If you are hostile to M/MJ, you will find workarounds and partial solutions, but I am confident that no matter what you propose, I can produce an example that M/MJ can solve and that what you propose cannot. If you are using any tools of any kind to generate or modify Java source code you are already acknowledging the need for MJ, but those are just the tip of the iceberg. Building serious software needs M.
I intend to follow up by finding others who are successfully using meta-programming and languages that show how it can be done.
I was rather surprised when the talk about "shorter-safer-higher" ended up in meta-programming. Shorter, sure, higher, I suppose, but safer? Most developers have a hard enough time producing straight first-order procedural code, never mind thinking about code that makes code that does something. So, sure, if you can produce a system that is usable, all power to you! (As a side note, the talk of the language you need vs the language you have made me think of Guy Steele's excellent classic talk on "growing a language" https://www.cs.virginia.edu/~evans/cs655/readings/steele.pdf or if you prefer to watch a video https://www.youtube.com/watch?v=_ahvzDzKdB0 )
<aside>As an anecdote, in about 2004 I built a system where the html page could be suitably annotated and then transformed by one XSLT template into another XSLT template that would be used to transform the XML data into a page with data. Hugely successful because the business people came in one week before going live with a new html page that we could just annotate and jack in. But the other developers found it ridiculously difficult so they ripped it out in favour of ASP.NET pages. They weren't even able to comprehend how XSLT worked, while the business people easily learned to produce XSLT templates (I held classes for both categories). The developers were too stuck in thoughts like "<html> is the command that begins an html document".</aside>
There is a lot of meta-programming in use today, much of it has been mentioned in threads here as Spring, Lombok, JPA and more. Spring used to be configured (and still can be) by xml files, but it is easier and better to use annotations as a virtual sort of code-transforming function (and Lombok and JPA do the same). Some of it is fairly useful, like an "@Post" annotation in Spring taking care of all the tedium of dealing with the http-protocol and just fills in the expected data into your method signature. Although if you just use an embedded Undertow server, you have a nice OO way of doing it instead, without the huge overhead of the Spring framework. But of course the codeshitters (love the term, Erwin!) find it much more interesting to produce frameworks for other codeshitters than to produce working software for real users.
Now these meta-programming constructs are just for consumption by the general programmer, it is not an easy thing to get into producing them. Spring has taken care to always provide ways to customize the behaviour when it doesn't quite fit your needs, but even that is not easy and is often severely underdocumented.
Your points are well made, and mesh nicely with my concept of the tool maker vs tool builder. In my experience tool makers are about 1 programmer in 10. 9 of 10 want to work 'concretely' with the end product using whatever tools they have and can understand and rely on. I belong to the 1 in 10 that prefer to think: what tool can I build to make this job much easier? Maybe you do too. [A few like Dave straddle the fence.]
But it behoves us tool-makes to put a complete, working, finished, reliable tool in the hands of the tool users, and you didn't do that. Your XSLT page was not a finished, reliable tool. Spring falls short if it's not documented. I wrote nearly 2000 pages of documentation for Powerflex, and it still falls short in places. [But I rely on it heavily for the bits I've forgotten.]
Veering off-topic here, but Spring is heavily documented. There are perhaps edge cases and work-in-progress bits, but the vast majority of Spring in daily use has a plethora of vendor-supplied documentation (https://spring.io), books (https://medium.com/javarevisited/10-best-spring-framework-books-for-java-developers-360284c37036), and online tutorials (https://www.baeldung.com/spring-tutorial).
That said, Spring is not magic, it's not a panacea, and there is plenty not to like about it, but lack of documentation isn't it.
For whatever weaknesses Java and its popular frameworks and libraries may have, weak documentation is rarely one of them.
AOP was intended to be a general way to add cross-cutting concerns, but on the C2 wiki even the supporters of it concede that there usually are better object-oriented ways of doing it. It doesn't look too difficult to do, but there seem to be some inherent problems in how they get applied and composed. In functional-land I think they would use Monads instead which are composable language constructs and so much better. And I suppose the ML module system is also an excellent programming construct for this kind of thing (but I admit I have still a very hazy idea of how that works).
I know AOP but this is wrong. Case in point: I want a small library of tracing/logging functions I can add to my C# programs, with the following features:
- It produces nicely formatted text output including options for date, time, argument values, etc
- It uses the minimum amount of space on the output device (collapse/merge similar messages)
- It can send output to the console, window, text file, debugger or stream
- It works on any platform: Windows console/UI/DLL/service, Unity, Unix, etc
- Trace calls can be enabled/disabled at runtime by program or configuration
- Disabled trace calls are 'free' (little or no CPU impact)
- Trace calls can be excluded from a production build
You can't do that with objects, but you can do it with C++ templates and macros.
That's standard logging; ILogger and friends in .NET, and a number of tools both inbuilt (java.util.logging) and popular 3rd party like log4j2 in Java. I'm even a minor contributor to that world with my rather specialist (logs by default to a SQLite database) https://github.com/DaveVoorhis/dbLogger
All you've described and more is typical of modern logging frameworks. The only missing (if we can call it that) bits are "Disabled trace calls are 'free'" and (this one baffles me) "Trace calls can be excluded from a production build."
First, disabled trace calls are not free. But they're very low cost, usually costing as little as almost nothing for the invocation of stub calls via a 'NoLogging' or similar implementation, plus the config load or activation/deactivation time, which is always present with a logging framework because you want to be able to turn it on/off at least at launch, if not at runtime, for production debugging.
The second one, "Trace calls can be excluded from a production build": Logging disabled or turned down to some minimal level during normal production is typical, along with the ability to turn logging on or up to any level, but being not-compiled-into-the-code excluded from enabling logging in production systems is oh-my-god levels of bad.
I can only assume that's not really what you meant.
Quote from Dave Voorhis on March 28, 2021, 12:27 amQuote from dandl on March 28, 2021, 12:00 amQuote from tobega on March 27, 2021, 1:50 pmQuote from dandl on March 27, 2021, 9:52 amM is a meta-programming language, for writing programs that read and write programs. Macros are a subset of M.
MJ is M for Java. I have just learned about the Java annotations processor, which seems to be a clunky subset of MJ. Java has nothing else.
I am highly confident that you cannot carry out the 4 tasks I outlined (and many others) with non-M "languages with desirable language features that compile (of which transpilation is a subset) to an object language target". If you are hostile to M/MJ, you will find workarounds and partial solutions, but I am confident that no matter what you propose, I can produce an example that M/MJ can solve and that what you propose cannot. If you are using any tools of any kind to generate or modify Java source code you are already acknowledging the need for MJ, but those are just the tip of the iceberg. Building serious software needs M.
I intend to follow up by finding others who are successfully using meta-programming and languages that show how it can be done.
I was rather surprised when the talk about "shorter-safer-higher" ended up in meta-programming. Shorter, sure, higher, I suppose, but safer? Most developers have a hard enough time producing straight first-order procedural code, never mind thinking about code that makes code that does something. So, sure, if you can produce a system that is usable, all power to you! (As a side note, the talk of the language you need vs the language you have made me think of Guy Steele's excellent classic talk on "growing a language" https://www.cs.virginia.edu/~evans/cs655/readings/steele.pdf or if you prefer to watch a video https://www.youtube.com/watch?v=_ahvzDzKdB0 )
<aside>As an anecdote, in about 2004 I built a system where the html page could be suitably annotated and then transformed by one XSLT template into another XSLT template that would be used to transform the XML data into a page with data. Hugely successful because the business people came in one week before going live with a new html page that we could just annotate and jack in. But the other developers found it ridiculously difficult so they ripped it out in favour of ASP.NET pages. They weren't even able to comprehend how XSLT worked, while the business people easily learned to produce XSLT templates (I held classes for both categories). The developers were too stuck in thoughts like "<html> is the command that begins an html document".</aside>
There is a lot of meta-programming in use today, much of it has been mentioned in threads here as Spring, Lombok, JPA and more. Spring used to be configured (and still can be) by xml files, but it is easier and better to use annotations as a virtual sort of code-transforming function (and Lombok and JPA do the same). Some of it is fairly useful, like an "@Post" annotation in Spring taking care of all the tedium of dealing with the http-protocol and just fills in the expected data into your method signature. Although if you just use an embedded Undertow server, you have a nice OO way of doing it instead, without the huge overhead of the Spring framework. But of course the codeshitters (love the term, Erwin!) find it much more interesting to produce frameworks for other codeshitters than to produce working software for real users.
Now these meta-programming constructs are just for consumption by the general programmer, it is not an easy thing to get into producing them. Spring has taken care to always provide ways to customize the behaviour when it doesn't quite fit your needs, but even that is not easy and is often severely underdocumented.
Your points are well made, and mesh nicely with my concept of the tool maker vs tool builder. In my experience tool makers are about 1 programmer in 10. 9 of 10 want to work 'concretely' with the end product using whatever tools they have and can understand and rely on. I belong to the 1 in 10 that prefer to think: what tool can I build to make this job much easier? Maybe you do too. [A few like Dave straddle the fence.]
But it behoves us tool-makes to put a complete, working, finished, reliable tool in the hands of the tool users, and you didn't do that. Your XSLT page was not a finished, reliable tool. Spring falls short if it's not documented. I wrote nearly 2000 pages of documentation for Powerflex, and it still falls short in places. [But I rely on it heavily for the bits I've forgotten.]
Veering off-topic here, but Spring is heavily documented. There are perhaps edge cases and work-in-progress bits, but the vast majority of Spring in daily use has a plethora of vendor-supplied documentation (https://spring.io), books (https://medium.com/javarevisited/10-best-spring-framework-books-for-java-developers-360284c37036), and online tutorials (https://www.baeldung.com/spring-tutorial).
That said, Spring is not magic, it's not a panacea, and there is plenty not to like about it, but lack of documentation isn't it.
For whatever weaknesses Java and its popular frameworks and libraries may have, weak documentation is rarely one of them.
True, for the normal expected uses the documentation is fairly to very good with plenty of tutorials. The problem here is rather that it is needed at all, i.e. it is not Java. Many things are not obvious what they do, perhaps especially when considering how different annotations interplay, but there are also gotchas in more normal use, e.g. that calling the method from within the same class bypasses the generated code.
For the case where you want to do something different than they anticipated, the documentation is usually not great. That's to be expected, of course, since you essentially have to understand enough about their implementation to override the pertinent parts, if it's even possible to do what you wanted.
Most of the time it would probably have been easier to do everything yourself with the aid of libraries of ordinary Java objects, then you would understand the process better to be able to deviate where needed, but Spring led you into a trap where you understand nothing beyond that their annotation magically solves a few predetermined cases that you might normally want. Frameworks make easy things trivial and difficult things impossible.