Life after D with Safe Java
Java was always designed to function both as an implementation language and an end-user application programming language. The driver for that is to keep the JVM small and portable, but it puts a lot of the plumbing on display in the front parlour. C# is at least as bad: it has pointers, low-level structs, DLL calls. We need them, but they make the language unsafe.
So I propose Safe Java (SJ) as a dialect of Java with a smarter compiler and all unsafe features removed, along the lines of:
- The primitive (inheritable) types are boolean, number, string, time, enum, blob. No characters, bits, unsigned, etc.
- Enum is a collection of string values written without quotes. Blob is opaque data.
- Full set of library functions operating on those types.
- The compiler tracks value usage and may require a conversion (such as round or truncate) eg before a number can be used as an integer, or do it automatically if safe.
- No new and no constructors. A declared variable or member is initialised to a default value, or one supplied by the declaration.
- No for(;;) loop.
- No reflection.
- A harmonised set of type-safe generic collections replacing array, list etc: start with just list<>, dictionary<>, set<>, and streams-like operations on those collections.
This leaves open the question of mutable state. If you leave it in it's still OO, if you take it out it's FP. My inclination is to have both: the primitives are immutable and new types can be mutable or not as required. The issue is that assignment semantics are different, and that causes bugs. It needs thinking through, but the compiler can certainly help. Pure FP is too big a stretch.
SJ is accompanied by Plain Old Java, for writing low level extensions, and M, used to extend the language itself. M has compile-time logic sufficient to implement and type-check language extensions such as
- relations based on headings
- type aliases
- serialisation (Json, XML, etc)
- various shorthands (like INSERT/DELETE/UPDATE).
It would seem a meta-programming capability equivalent to Rust is sufficient for M. Most of Rust is far too low level (Linus thinks it might be used to write code for Linux).
It's a thought experiment at this stage but please note: SJ is finished when there is nothing left to remove and nothing needs to be added.
Dave, is your proposal something you could see yourself implementing?
Quote from dandl on April 14, 2021, 2:08 am
- The primitive (inheritable) types are boolean, number, string, time, enum, blob. No characters, bits, unsigned, etc.
- Enum is a collection of string values written without quotes. Blob is opaque data.
- A harmonised set of type-safe generic collections replacing array, list etc: start with just list<>, dictionary<>, set<>, and streams-like operations on those collections.
For my part, with my own type system, I have presently narrowed it down to just 10 "primary" types which are the ones in terms of which all other types are defined:
- Ignorance - A singleton whose meaning is that a regular value isn't present and we have no explanation for why.
- Boolean - An enumeration of false and true.
- Integer - An integer with unlimited possible magnitude.
- Bits - A string of bits.
- Blob - A string of octets.
- Text - A string of 21-bit Unicode codepoints, which is also used to represent an unqualified identifier.
- Nesting - Characterized by a nonempty sequence of Text, is used to represent a multi-level-qualified identifier.
- Pair - Characterized by an ordered pair of any 2 other values.
- Tuple - Characterized by an unordered set of name+value pairs that are Text and any value respectively; can also fake an ordered tuple.
- Lot - Characterized by an ordered sequence of any+any pairs, though typically it is any+count and typically count=1; used to represent both arrays and sets etc.
Notably, non-integer numbers and dates and other collections are logically considered "secondary" types defined in terms of the above or each other, typically as a typename+payload Pair.
General enums are defined in terms of constraints on other types.
Implementations can optimize as they see fit behind the scenes, but what I mentioned above is exposed.
Perhaps this may give you ideas for SJ.
Quote from dandl on April 14, 2021, 2:08 am
- ...
This leaves open the question of mutable state. If you leave it in it's still OO, if you take it out it's FP.
In (modern) Functional Programming, mutable state is handled by something called the 'State Monad', possibly overloaded with 'Monad Transformers'. Note both of those have substantial programming (type) theory behind them, plus actual implementations in actual software (libraries and compiler hooks).
... Pure FP is too big a stretch.
"Pure" FP means what?
SJ is
yet another acronym you've invented out of thin air. Is what it does any different from the 'smart compiler'/compiler magic of earlier threads?
accompanied by Plain Old Java, for writing low level extensions, and M,
'M' stands for 'magic'?
It's a thought experiment at this stage but please note: SJ is finished when there is nothing left to remove and nothing needs to be added.
Stop posting your thought experiments here. Come back when SJ and M are finished as actual software artefacts.
Quote from Darren Duncan on April 14, 2021, 5:12 amDave, is your proposal something you could see yourself implementing?
Creating a complete functioning programming language is a massive undertaking (I should know). The kind of 'smart compiler' I've been talking about is well beyond anything I've ever done. So probably not.
The 'need for safety' is evidently a theme starting to emerge, but so far mainly in the community of 'burnt by C++' developers. Low-level memory management and threads are two areas where C/C++ has the power and the features but achieving safety is beyond reasonable human effort and all compilers. The language has to want to change first, but the changes to do this are no the kid of safety I've been talking about.
I do plan to spend some time getting more familiar with Rust and particularly its meta-programming. I'll be really interested to see whether it's possible to create a type-checked RA, as I suspect it is. Otherwise Rust is too low lever, too much about implementing and not enough about using.
But on the safety theme, until there is a community of interest around safer ways of developing business and data-oriented applications I don't see much progress. I'll keep looking, but go and read Dave's thoughts on modern app development, and weep. He and countless others like things just as they are and this is a reasonable thing to do, but progress (as Shaw reminds us) depends on the unreasonable man, who finds the world not to his taste and persists in trying to change it.
Quote from dandl on April 15, 2021, 12:54 am... progress (as Shaw reminds us) depends on the unreasonable man, who finds the world not to his taste and persists in trying to change it.
You mean "unreasonable" like the men mentioned here, or these, or these? (Sorry to be sexist, but they are mostly men.)
Shaw's own attempts at language engineering were ineffective.
Quote from AntC on April 15, 2021, 2:49 amQuote from dandl on April 15, 2021, 12:54 am... progress (as Shaw reminds us) depends on the unreasonable man, who finds the world not to his taste and persists in trying to change it.You mean "unreasonable" like the men mentioned here, or these, or these? (Sorry to be sexist, but they are mostly men.)
Shaw's own attempts at language engineering were ineffective.
My favourite is https://en.wikipedia.org/wiki/Phlogiston_theory.
There are good grounds for a limited flat earth theory. In the same way we use Newtonian mechanics most of the time and switch to Einstein as needed, we all use a flat earth model to navigate locally, create maps, work out angles and distances and so on. Then we switch to a 3D earth only when the extra complexity justifies it. The point is: competing models can co-exist.
My own experience with 'unreasonable' theories includes continental drift, Pylobacter and diverticular disease (where the treatment of choice went from low fibre to high fibre overnight. There are many more: Newton, Darwin, Einstein, Galileo and so on. In the end, science gets it right, but it very often takes an 'unreasonable man' to help it along.
But this is not science. We have effective ways to write software and mostly we've been playing catchup with new devices. There is no simple experiment to demonstrate an inconvenient truth: that much of our programming is weighed down by accidental complexity. If it takes 10 years to master programming, it probably takes at least another 10 (and the luxury of time to think) to see the problem and what might be done about it.
My gut feeling tells me that meta-programming and safer-higher-shorter are directions worth exploring. I'm more than happy to listen to other 'unreasonable' opinions.
Quote from dandl on April 15, 2021, 12:54 amQuote from Darren Duncan on April 14, 2021, 5:12 amDave, is your proposal something you could see yourself implementing?
Creating a complete functioning programming language is a massive undertaking (I should know). The kind of 'smart compiler' I've been talking about is well beyond anything I've ever done. So probably not.
The 'need for safety' is evidently a theme starting to emerge, but so far mainly in the community of 'burnt by C++' developers. Low-level memory management and threads are two areas where C/C++ has the power and the features but achieving safety is beyond reasonable human effort and all compilers. The language has to want to change first, but the changes to do this are no the kid of safety I've been talking about.
I do plan to spend some time getting more familiar with Rust and particularly its meta-programming. I'll be really interested to see whether it's possible to create a type-checked RA, as I suspect it is. Otherwise Rust is too low lever, too much about implementing and not enough about using.
But on the safety theme, until there is a community of interest around safer ways of developing business and data-oriented applications I don't see much progress. I'll keep looking, but go and read Dave's thoughts on modern app development, and weep. He and countless others like things just as they are ...
Nope. Remember that the recent activity started with my suggestion that we look at future languages and/or or features to support implementing D in-language.
I recommend working with real programmers on real applications to understand what real problems we face (one or two of your own personal projects are not enough) and what we are, in various ways, trying to solve; what apparent problems aren't really a problem; what apparent non-problems are a big problem; what approaches have been tried and rightfully discarded; and what solutions -- even if not yet (or ever) widely used -- are actually working.
In fact there is a lot of progress, but it's evolutionary, not revolutionary.
But on the safety theme, until there is a community of interest around safer ways of developing business and data-oriented applications I don't see much progress. I'll keep looking, but go and read Dave's thoughts on modern app development, and weep. He and countless others like things just as they are ...
Nope. Remember that the recent activity started with my suggestion that we look at future languages and/or or features to support implementing D in-language.
I can show how to do that, based on Rust-style meta-programming. But TTM originates in the concerns of the mid 1990s, I think we can aim a little higher now.
I recommend working with real programmers on real applications to understand what real problems we face (one or two of your own personal projects are not enough) and what we are, in various ways, trying to solve; what apparent problems aren't really a problem; what apparent non-problems are a big problem; what approaches have been tried and rightfully discarded; and what solutions -- even if not yet (or ever) widely used -- are actually working.
I've been doing that for years. My personal projects are specifically to track solutions that appear to be working. I choose my projects for the level of discomfort above all else.
In fact there is a lot of progress, but it's evolutionary, not revolutionary.
So says the reasonable man. In the world of the early 1970s Ritchie was the unreasonable man suggesting an operating system could be written in a higher level language. Bill Gates with Windows in the 1980s, Gosling with Java, Berners-Lee with the Web, Jobs with the iPos and then the iPhone, these are all the work of unreasonable men.
I know that Java/C# for general application development will be replaced by the work of an unreasonable man, that you will hate it, fight it and eventually accept it.
Even so, I would appreciate your insights into real progress made and real problems remaining.
Quote from dandl on April 15, 2021, 10:37 amBut on the safety theme, until there is a community of interest around safer ways of developing business and data-oriented applications I don't see much progress. I'll keep looking, but go and read Dave's thoughts on modern app development, and weep. He and countless others like things just as they are ...
Nope. Remember that the recent activity started with my suggestion that we look at future languages and/or or features to support implementing D in-language.
I can show how to do that, based on Rust-style meta-programming. But TTM originates in the concerns of the mid 1990s, I think we can aim a little higher now.
Fewer boasts and attempts at intellectual one-upmanship and more "here's a thing I wrote; it does these things" would be a reasonable aim.
I recommend working with real programmers on real applications to understand what real problems we face (one or two of your own personal projects are not enough) and what we are, in various ways, trying to solve; what apparent problems aren't really a problem; what apparent non-problems are a big problem; what approaches have been tried and rightfully discarded; and what solutions -- even if not yet (or ever) widely used -- are actually working.
I've been doing that for years. My personal projects are specifically to track solutions that appear to be working. I choose my projects for the level of discomfort above all else.
In fact there is a lot of progress, but it's evolutionary, not revolutionary.
So says the reasonable man. In the world of the early 1970s Ritchie was the unreasonable man suggesting an operating system could be written in a higher level language. Bill Gates with Windows in the 1980s, Gosling with Java, Berners-Lee with the Web, Jobs with the iPos and then the iPhone, these are all the work of unreasonable men.
Actually, every one of those was iterative work, clearly building on what came before. It's perhaps a public perception that each was revolutionary -- it makes for good stories, I guess -- but within the industry, they were evolutionary and effectively addressing a clear need or want.
Indeed, the unreasonable man might have been Jack Goldman at Xerox Parc, or Ted Nelson with Xanadu, or whoever at Microsoft advocated Windows CE and Windows Mobile.
I know that Java/C# for general application development will be replaced by the work of an unreasonable man, that you will hate it, fight it and eventually accept it.
No one fights -- or hates -- useful tools that genuinely solve problems, as long as they don't add more technical debt. For example, Java and C# weren't hated and fought; most of us embraced them because they solved problems and reduced development effort (compared to C++) and technical debt.
But note what is perhaps the biggest difference between the casual, hobbyist, or small-business developer and the enterprise developer: The former isn't anywhere near as profoundly averse to technical debt as the latter, because the latter knows that no language, tool, framework or platform -- no matter what it claims on the marketing brochure -- is worth anything unless the inevitable mountain of technical debt either gets smaller or at least doesn't get any bigger.
Far too many suggestions add only a whisper of reduced development effort, with a lurking mountain of new technical debt.