Quote from Dave Voorhis on June 15, 2021, 11:07 am

Quote from dandl on June 15, 2021, 10:37 am

You lost me. Most languages do not force any kind of exception handling, Java is the odd man out. I'm talking about new language features to achieve safety, and CQS with language amd library support is what I propose. It's shorter and safer, easier to reason about, and you can still have global exceptions for 'get me out of here now!'. Why do you object?

Exactly, most languages do not force any kind of exception handling and that is the problem.

Languages should force exception handling, because exceptions, no matter how rare, will occur.

This is wrong. In my proposal exceptions are rare, unforseeable and catastrophic. Many programs will be happy to let nature take its course; a few will include code to recover and continue.

What does "happy to let nature take its course" mean in terms of program operation?

[Did you know the moon lander computer had 5  exceptions caused by a hardware bug in the final descent, and recovered by restarting? That's exception handling.]

Most error conditions handled by exceptions are not exceptional at all. They are the ordinary consequences of writing code the compiler can't check: null exceptions, missing external dependencies, data conversion, malformed input, etc. There is no need whatsoever for exception handling. I rarely use it, and never miss it. Java annoys me deeply in this respect.

Null exceptions, missing external dependencies at compile-time, data conversion / malformed input can all, with appropriate language support, be obligated to be handled by runtime code paths that are verified to exist at compile time. Thus, these errors should never cause a run-time crash, and the compiler can enforce this through type systems and/or checked exceptions.

Run-time environmental catastrophes -- the DBMS goes away, the disk becomes corrupt, a query that returned CustomerAddress yesterday doesn't have a CustomerAddress today -- are the sort of errors that inevitably occur but tend to be poorly handled (at least by default) in most languages except Java. In other languages, appropriately handling them relies entirely on developer discipline. Only Java takes some of that load off the developer because of its use of checked exceptions, even if it's just a "here be dragons" way.

There are good reasons Java is pervasive in the big enterprise world and this is one of the reasons.

Java was on the right track with checked exceptions, but it didn't go far enough. It should have made (almost) all exceptions be checked exceptions.

Instead, it distinguished checked (must handle; designed for recoverable exceptions, often in external dependencies) from unchecked (no need to handle; mainly intended for programming errors or irrecoverable like out-of-memory) exceptions. It was done with good intent -- to address precisely the sort of problem we're talking about here -- but the inevitable result is that methods exist that can throw unchecked exceptions without the user of those methods knowing it can happen, so code gets written that can fail in an uncontrolled fashion.

So if you want to write programs that keeping running no matter what (if it compiles, it runs), you make it a practice to wrap everything in general exception handlers. If something throws an exception, any exception, you catch it and degrade gracefully or restart after a delay or whatever.  That works, but it could be better.

That's now. As you say, we can do better.

The problem is still that nothing obligates the developer to implement such mechanisms, and there's no (easy) way for higher levels to intercept exceptions thrown and caught inside lower-level mechanisms.

Thus, all exceptions should be checked exceptions, and in the case of "system" exceptions like running out of memory or stack that can occur everywhere, there should be an obligatory root-level handler.

Command-and-query is fine for "happy path" code, but it's not really an exception handling mechanism. There is nothing in it to obligate handling exceptions, so it encourages writing broken code that does nothing but issue commands and fail silently. That's not resilience and safety, that's just quiet breakage.

But if it had mechanisms that obligate checking status after issuing commands, then it would be better. It would still tediously require an explicit check at every 'command' rather than concisely wrapping a collection of related commands in a single any-failure-in-these-should-do-this exception handler, but at least it would obligate dealing with exceptions, and it still needs a mechanism to optionally intercept low-level exception handling at higher levels.

Which is where I started. The proposal is language features in the pursuit of safer-higher-shorter. The requirements:

• All points of failure are known by the compiler (from library info)
• Compiler requires program to specify how to handle every failure point (from list of permitted strategies)
  • it's easier to choose a canned strategy than to write a bogus handler
  • there is no 'ignore': you have to do something, even if it's just to substitute a default value
• For external dependencies, status can be known before execution, or during execution but before failure (library info).
• CQS is provided as a means to deal safely with failures inline (if needed)
• Global panic handler allows programs to recover and restart.

The language part is easy. The hard bit is getting library routines to anticipate possible failures and provide mitigation strategies. Take file open for read as an example. Strategies include:

• On missing file program will not run, write to log
• On missing file, write to log with user data, panic (before trying to open file)
• On open failure write to log and panic
• On open failure substitute default data
• On open failure execute handler.

And so on.

Sounds awful to me -- unduly limited, pre-specified, inflexible, verbose, and awkward -- except your first bullet point: "All points of failure are known by the compiler."

That I absolutely agree with. The rest, no, except partially the last bullet point -- "Global panic handler allows programs to recover and restart" -- but not as a distinct mechanism because it should be just the compiler obligating you to handle whatever exceptions haven't been handled anywhere else.

But if you feel your approach will work, try it.

Quote from dandl on June 16, 2021, 2:16 pm

Quote from Dave Voorhis on June 15, 2021, 11:07 am

Quote from dandl on June 15, 2021, 10:37 am

You lost me. Most languages do not force any kind of exception handling, Java is the odd man out. I'm talking about new language features to achieve safety, and CQS with language amd library support is what I propose. It's shorter and safer, easier to reason about, and you can still have global exceptions for 'get me out of here now!'. Why do you object?

Exactly, most languages do not force any kind of exception handling and that is the problem.

Languages should force exception handling, because exceptions, no matter how rare, will occur.

This is wrong. In my proposal exceptions are rare, unforseeable and catastrophic. Many programs will be happy to let nature take its course; a few will include code to recover and continue.

What does "happy to let nature take its course" mean in terms of program operation?

I guess it means do nothing. Assume that built in defaults are good enough.

[Did you know the moon lander computer had 5  exceptions caused by a hardware bug in the final descent, and recovered by restarting? That's exception handling.]

Most error conditions handled by exceptions are not exceptional at all. They are the ordinary consequences of writing code the compiler can't check: null exceptions, missing external dependencies, data conversion, malformed input, etc. There is no need whatsoever for exception handling. I rarely use it, and never miss it. Java annoys me deeply in this respect.

Null exceptions, missing external dependencies at compile-time, data conversion / malformed input can all, with appropriate language support, be obligated to be handled by runtime code paths that are verified to exist at compile time. Thus, these errors should never cause a run-time crash, and the compiler can enforce this through type systems and/or checked exceptions.

Agreed, but raising and handling exception is the wrong way. Every exception handler is a goto, obscures the code path, allows state to be corrupted, is a source of its own hard-to-test bugs and is just plain not safe. Don't do it.

Run-time environmental catastrophes -- the DBMS goes away, the disk becomes corrupt, a query that returned CustomerAddress yesterday doesn't have a CustomerAddress today -- are the sort of errors that inevitably occur but tend to be poorly handled (at least by default) in most languages except Java. In other languages, appropriately handling them relies entirely on developer discipline. Only Java takes some of that load off the developer because of its use of checked exceptions, even if it's just a "here be dragons" way.

There are good reasons Java is pervasive in the big enterprise world and this is one of the reasons.

No, this is not one of the reasons. Once the (bad) decision is made to base error handling on exceptions, it is useful to distinguish those intended to be caught from those due to bugs, but either way it's just not safe.

Java was on the right track with checked exceptions, but it didn't go far enough. It should have made (almost) all exceptions be checked exceptions.

Instead, it distinguished checked (must handle; designed for recoverable exceptions, often in external dependencies) from unchecked (no need to handle; mainly intended for programming errors or irrecoverable like out-of-memory) exceptions. It was done with good intent -- to address precisely the sort of problem we're talking about here -- but the inevitable result is that methods exist that can throw unchecked exceptions without the user of those methods knowing it can happen, so code gets written that can fail in an uncontrolled fashion.

So if you want to write programs that keeping running no matter what (if it compiles, it runs), you make it a practice to wrap everything in general exception handlers. If something throws an exception, any exception, you catch it and degrade gracefully or restart after a delay or whatever.  That works, but it could be better.

That's now. As you say, we can do better.

The problem is still that nothing obligates the developer to implement such mechanisms, and there's no (easy) way for higher levels to intercept exceptions thrown and caught inside lower-level mechanisms.

Thus, all exceptions should be checked exceptions, and in the case of "system" exceptions like running out of memory or stack that can occur everywhere, there should be an obligatory root-level handler.

Command-and-query is fine for "happy path" code, but it's not really an exception handling mechanism. There is nothing in it to obligate handling exceptions, so it encourages writing broken code that does nothing but issue commands and fail silently. That's not resilience and safety, that's just quiet breakage.

But if it had mechanisms that obligate checking status after issuing commands, then it would be better. It would still tediously require an explicit check at every 'command' rather than concisely wrapping a collection of related commands in a single any-failure-in-these-should-do-this exception handler, but at least it would obligate dealing with exceptions, and it still needs a mechanism to optionally intercept low-level exception handling at higher levels.

Which is where I started. The proposal is language features in the pursuit of safer-higher-shorter. The requirements:

• All points of failure are known by the compiler (from library info)
• Compiler requires program to specify how to handle every failure point (from list of permitted strategies)
  • it's easier to choose a canned strategy than to write a bogus handler
  • there is no 'ignore': you have to do something, even if it's just to substitute a default value
• For external dependencies, status can be known before execution, or during execution but before failure (library info).
• CQS is provided as a means to deal safely with failures inline (if needed)
• Global panic handler allows programs to recover and restart.

The language part is easy. The hard bit is getting library routines to anticipate possible failures and provide mitigation strategies. Take file open for read as an example. Strategies include:

• On missing file program will not run, write to log
• On missing file, write to log with user data, panic (before trying to open file)
• On open failure write to log and panic
• On open failure substitute default data
• On open failure execute handler.

And so on.

Sounds awful to me -- unduly limited, pre-specified, inflexible, verbose, and awkward -- except your first bullet point: "All points of failure are known by the compiler."

That I absolutely agree with. The rest, no, except partially the last bullet point -- "Global panic handler allows programs to recover and restart" -- but not as a distinct mechanism because it should be just the compiler obligating you to handle whatever exceptions haven't been handled anywhere else.

But if you feel your approach will work, try it.

My experience of writing code to this style is that it's shorter, cleaner and easier to understand and to debug. I do make some use of exceptions, but as a rule only to 'escape' from some deeply nested call to a higher level, never to catch and retain control. I find exception handling a frequent source of state-related bugs, and keep it to a minimum. In a sense, this particular proposal is really about how to get rid of exception handling, not to sanitise it (somewhat). There is no way to make exceptions (as we know them) safe.

CQS is not the only way to avoid exception handlers. In FP the functions are pure and the I/O is confined to monads, which places dependencies together at the highest level. This avoids the need ever to 'escape' from a low level dependency failure. I have used this style too with some success.

The problem we all face now is too much existing code. You like most of us are deeply invested in doing things the way they've always been done,but I see no sign that the software we build is evolving into something better. From an end-user point of view it's definitely getting worse: we moved house and the effort evolved in doing address changes just gets more and more time-consuming. Almost every web-site we use is buggy; I registered with a new doctor and was given a tablet to enter my details: I found 5 bugs without even trying, 2 serious. If this is the product of a lifetime in IT, it's not something to be proud of.

So I speculate on how a focus on safer might change what we do, but I don't expect it to see the light of day. This industry just doesn't work like that.

and there are so few opportunities to try something new, at scale.

 

 

Quote from Dave Voorhis on June 17, 2021, 1:41 pm

Quote from dandl on June 16, 2021, 2:16 pm

Quote from Dave Voorhis on June 15, 2021, 11:07 am

Quote from dandl on June 15, 2021, 10:37 am

You lost me. Most languages do not force any kind of exception handling, Java is the odd man out. I'm talking about new language features to achieve safety, and CQS with language amd library support is what I propose. It's shorter and safer, easier to reason about, and you can still have global exceptions for 'get me out of here now!'. Why do you object?

Exactly, most languages do not force any kind of exception handling and that is the problem.

Languages should force exception handling, because exceptions, no matter how rare, will occur.

This is wrong. In my proposal exceptions are rare, unforseeable and catastrophic. Many programs will be happy to let nature take its course; a few will include code to recover and continue.

What does "happy to let nature take its course" mean in terms of program operation?

I guess it means do nothing. Assume that built in defaults are good enough.

[Did you know the moon lander computer had 5  exceptions caused by a hardware bug in the final descent, and recovered by restarting? That's exception handling.]

Most error conditions handled by exceptions are not exceptional at all. They are the ordinary consequences of writing code the compiler can't check: null exceptions, missing external dependencies, data conversion, malformed input, etc. There is no need whatsoever for exception handling. I rarely use it, and never miss it. Java annoys me deeply in this respect.

Null exceptions, missing external dependencies at compile-time, data conversion / malformed input can all, with appropriate language support, be obligated to be handled by runtime code paths that are verified to exist at compile time. Thus, these errors should never cause a run-time crash, and the compiler can enforce this through type systems and/or checked exceptions.

Agreed, but raising and handling exception is the wrong way. Every exception handler is a goto, obscures the code path, allows state to be corrupted, is a source of its own hard-to-test bugs and is just plain not safe. Don't do it.

Run-time environmental catastrophes -- the DBMS goes away, the disk becomes corrupt, a query that returned CustomerAddress yesterday doesn't have a CustomerAddress today -- are the sort of errors that inevitably occur but tend to be poorly handled (at least by default) in most languages except Java. In other languages, appropriately handling them relies entirely on developer discipline. Only Java takes some of that load off the developer because of its use of checked exceptions, even if it's just a "here be dragons" way.

There are good reasons Java is pervasive in the big enterprise world and this is one of the reasons.

No, this is not one of the reasons. Once the (bad) decision is made to base error handling on exceptions, it is useful to distinguish those intended to be caught from those due to bugs, but either way it's just not safe.

Java was on the right track with checked exceptions, but it didn't go far enough. It should have made (almost) all exceptions be checked exceptions.

Instead, it distinguished checked (must handle; designed for recoverable exceptions, often in external dependencies) from unchecked (no need to handle; mainly intended for programming errors or irrecoverable like out-of-memory) exceptions. It was done with good intent -- to address precisely the sort of problem we're talking about here -- but the inevitable result is that methods exist that can throw unchecked exceptions without the user of those methods knowing it can happen, so code gets written that can fail in an uncontrolled fashion.

So if you want to write programs that keeping running no matter what (if it compiles, it runs), you make it a practice to wrap everything in general exception handlers. If something throws an exception, any exception, you catch it and degrade gracefully or restart after a delay or whatever.  That works, but it could be better.

That's now. As you say, we can do better.

The problem is still that nothing obligates the developer to implement such mechanisms, and there's no (easy) way for higher levels to intercept exceptions thrown and caught inside lower-level mechanisms.

Thus, all exceptions should be checked exceptions, and in the case of "system" exceptions like running out of memory or stack that can occur everywhere, there should be an obligatory root-level handler.

Command-and-query is fine for "happy path" code, but it's not really an exception handling mechanism. There is nothing in it to obligate handling exceptions, so it encourages writing broken code that does nothing but issue commands and fail silently. That's not resilience and safety, that's just quiet breakage.

But if it had mechanisms that obligate checking status after issuing commands, then it would be better. It would still tediously require an explicit check at every 'command' rather than concisely wrapping a collection of related commands in a single any-failure-in-these-should-do-this exception handler, but at least it would obligate dealing with exceptions, and it still needs a mechanism to optionally intercept low-level exception handling at higher levels.

Which is where I started. The proposal is language features in the pursuit of safer-higher-shorter. The requirements:

• All points of failure are known by the compiler (from library info)
• Compiler requires program to specify how to handle every failure point (from list of permitted strategies)
  • it's easier to choose a canned strategy than to write a bogus handler
  • there is no 'ignore': you have to do something, even if it's just to substitute a default value
• For external dependencies, status can be known before execution, or during execution but before failure (library info).
• CQS is provided as a means to deal safely with failures inline (if needed)
• Global panic handler allows programs to recover and restart.

The language part is easy. The hard bit is getting library routines to anticipate possible failures and provide mitigation strategies. Take file open for read as an example. Strategies include:

• On missing file program will not run, write to log
• On missing file, write to log with user data, panic (before trying to open file)
• On open failure write to log and panic
• On open failure substitute default data
• On open failure execute handler.

And so on.

Sounds awful to me -- unduly limited, pre-specified, inflexible, verbose, and awkward -- except your first bullet point: "All points of failure are known by the compiler."

That I absolutely agree with. The rest, no, except partially the last bullet point -- "Global panic handler allows programs to recover and restart" -- but not as a distinct mechanism because it should be just the compiler obligating you to handle whatever exceptions haven't been handled anywhere else.

But if you feel your approach will work, try it.

My experience of writing code to this style is that it's shorter, cleaner and easier to understand and to debug. I do make some use of exceptions, but as a rule only to 'escape' from some deeply nested call to a higher level, never to catch and retain control. I find exception handling a frequent source of state-related bugs, and keep it to a minimum. In a sense, this particular proposal is really about how to get rid of exception handling, not to sanitise it (somewhat). There is no way to make exceptions (as we know them) safe.

I disagree. Without checked exceptions, it requires considerable discipline to make code entirely safe, i.e., so that if it compiles it runs and stays running. "If it compiles it stays running for a while" or "if it compiles it stays running only if the developer has been sufficiently disciplined" are straightforward.

I'm not clear how command/query is intended to enforce handling problems, unless the intent is that commands never "fail."

That's a strategy we've seen before -- notably I recall a dark period of Microsoft software installers that would run through the entire process and create a broken installation with no warnings, no errors, no logs, no sign of any failure indication. That's not a solution. That's if it compiles it "runs", where "runs" is used too loosely to be meaningful.

CQS is not the only way to avoid exception handlers. In FP the functions are pure and the I/O is confined to monads, which places dependencies together at the highest level. This avoids the need ever to 'escape' from a low level dependency failure. I have used this style too with some success.

I'm assuming what are essentially tweaks to the existing semantics of popular mixed-paradigm languages, rather than paradigm shifts.

The problem we all face now is too much existing code. You like most of us are deeply invested in doing things the way they've always been done,but I see no sign that the software we build is evolving into something better. From an end-user point of view it's definitely getting worse: we moved house and the effort evolved in doing address changes just gets more and more time-consuming. Almost every web-site we use is buggy; I registered with a new doctor and was given a tablet to enter my details: I found 5 bugs without even trying, 2 serious. If this is the product of a lifetime in IT, it's not something to be proud of.

So I speculate on how a focus on safer might change what we do, but I don't expect it to see the light of day. This industry just doesn't work like that.

and there are so few opportunities to try something new, at scale.

Burn the disk packs!

Unfortunately there's little incentive toward quality IT, and much incentive to hammer out rubbish at maximum velocity. The reality is that for most things, IT is a luxury convenience, not a necessity, and if it fails it's (at worst) annoying.

In that respect, IT and software engineering have much in common with the mass market clothing industry -- for which product failure is, at worst, slightly embarrassing -- and almost nothing in common with true engineering disciplines like mechanical engineering and electrical engineering. They have professional bodies providing oversight and accreditation. We have "thought leaders" providing opinions and technical fashion trends.

Indeed, you can become a software engineer merely by deciding that's what you want to be called.

But talk to mechanical and electrical engineers, and you quickly realise the same problems exist even there, despite professional bodies providing oversight and accreditation.

That's because consumers prioritise paying the lowest price over quality, and (for the most part) are willing to put up with the cheap crap they get for it rather than pay a bit more to get something good.