Re: Re: Re: [PATCH 0/4] Add import builtin

[Koichi Murase]

If your purpose is just to solve a small inconvenience of the `source'
builtin mixing the namespaces of local scripts, "libraries", and
executables, I think the suggested `source -i' or `source -l' would be
fine. I think no additional reasoning is needed. In this case, the
"#pragma once" feature wouldn't be needed or should be a separate
flag. You can of course add extra reasoning about the necessity of
this small improvement, but the problem is that I don't think those
arguments are valid.

If your goal is to facilitate the primitive parts needed for library
management, I don't think the current suggestion would actually help.
The source builtin is already the primitive feature that can be used
to develop a module manager on top of it. I don't think any additions
to the source builtin is useful for developing a module manager
because the typical use cases by the module managers are different
from your assumptions (as will be discussed later in this reply). This
is a YAGNI feature, which hasn't been tested by real use cases of the
intended usage. Then, you don't have a motivation to modify the source
builtin or add a new builtin at present.

The question is whether you would still value `source -i' if it were
not useful for module managers at all. To me, you seem to raise the
bar by yourself. Where is your bar?

2024年5月4日(土) 11:44 Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>:
> I think we were unable to reach an understanding because we were both
> using the same words to talk about different things.

Can I call what you describe the "module loader"? Then,

> Those are problems better
> solved by Linux distribution packagers or a custom package manager for bash.
> I may be wrong but I believe that is the role many of those bash frameworks
> are fulfilling.

Let me call the above the "module manager".

I understand that the purpose of the *present* patch is the module
loader, but I thought you tried to prepare the primitive useful for a
unique module manager. This is because you mention that a language
should have only one module system so that the library developers do
not need to care about the variation. The variations in the module
managers is a problem since the library itself needs to be adjusted
under the rule of the module manager (for dependency resolution, etc).
This is a general problem for a programming language, and recent
programming languages have respective standard ways of module
management. But maybe you've been discussing something else.

> So at least to me this is all about the "import" primitive.
> Which particular form it takes is up to debate, it's something
> that's worth discussing and getting right before it's released
> due to the difficulty in changing such core APIs after the fact.
> However, the fact is that I think many of those other considerations
> are actually out of scope!

I understand you are not going to develop the module manager but just
want to focus on the module-loader part. Nevertheless, I think you can
agree that we want to discuss the actual use cases of the "import"
primitive in order to shape the needed core API in a usable form
(Without mentioning the use cases, what would be discussed?). Besides,
you seem to assume the future uses of the "import" primitive in module
managers. Then, we cannot avoid the discussion about the use cases in
the module managers.

> Those are problems better
> solved by Linux distribution packagers or a custom package manager for bash.
> I may be wrong but I believe that is the role many of those bash frameworks
> are fulfilling. In order to do that, they _had_ to create an import primitive
> for themselves because bash lacked such facilities. My intention is to fix
> the lack of the import facilities, not to replace those frameworks.

Yes, we had to create the "import" command by ourselves, but the
situation doesn't change at all even after the suggested modification.
.

> Ideally, they'll be using bash's native import under the hood!

Yes, module managers still need to implement their own "import"
command while using the proposed "import" primitive under the hood,
and it's simply interchangeable with the source builtin we already
have.

* Module managers typically try to identify the file under more
detailed rules, which cannot be implemented by just the `source -i'
builtin. For example, module managers allow dropping the filename
extension from the specified name; the "import moduleA" command would
search the file "moduleA", "moduleA.sh", or "moduleA.bash". Or some
other managers identify the version of a module based on the requested
range. Some frameworks try to first identify the system version of the
module and fall back to the bundled version depending on some
conditions. Those rules depend on the module manager. Also, some
module managers want to manage the actual paths of the modules that
are loaded. For these reasons, most module managers actually resolve
the path by itself using their rules and specify *the absolute path*
to the source builtin. As far as the absolute path is specified, there
is no difference between the source builtin and the suggested `source
-i' or `import'.

* Even if the module manager directly uses the builtin to identify the
module without any special rules, the module manager wants to take
over the variable BASH_IMPORT_PATH in calling the suggested feature
(as mentioned three times in your reply. I will skip replying to the
respective comments). However, in that case, I don't see much
difference from « PATH=<modpath> source » (as far as the module just
defines a set of functions).

* If the module manager wouldn't even set BASH_IMPORT_PATH under the
hood and uses the global BASH_IMPORT_PATH, the suggested feature
essentially just reduces one line « local PATH=<modpath> » from the
manager code. Does reducing one line from the manager code deserve the
new feature in the Bash language?

> By introducing the "module/library system" I want to do the following:
>
>   1. Add a builtin primitive that can be used to load libraries

The source builtin already works.

>   2. Establish a convention for where bash will look for libraries

PATH can already be used and is the standard POSIX way.

>   3. Separate the libraries from the commands/executables

This is the only new part of the suggestion, though I think this is
important (when module managers are not used). BASH_IMPORT_PATH and
the flag to restrict the search there would accomplish that. However,
when module managers are involved, it's irrelevant as already
discussed.

> > I feel we should ask the package maintainers to include those files.
>
> I think the fact they are called "examples" might be having a major impact
> on how such files are perceived by packagers and distributors. "Examples"
> sound like documentation, example code meant for learning how some bash
> feature works. They generally copied and built upon, not used as libraries.

Yeah, some of the files in /examples/functions are just examples and
not so useful to make it a part of the standard library. But it is
still good to prepare a place to put the Bash libraries somewhere in
$prefix/share/bash/...

> So instead of everyone having to define such a mechanism for themselves,
> only the bash frameworks would have to. Users could be well served by
> running `source -l some-file` directly in the terminal without being
> forced to use a framework at all, but they can still choose to use one
> if that's what they want.

I'm confused. While you say you just work on the module loader and not
intend to deny the module managers, you try to promote not using the
module manager here.  Would you try to exclude the possibility of a
module manager in the future of Bash?

> That's true. It's not my intention to go to war with anyone,
> but I can't deny that I do have a vision for bash and how it
> should evolve. One of the coolest things about free software
> is how we can actually make it happen, that motivates me.
>
> I don't think my vision is mutually exclusive with the vision of others.
> I think it could even make their visions easier to achieve.
> So I do hope we can reach a consensus.

If you start to talk about the vision about the Bash language, that
would certainly raise the bar. You'll have to tell a story about how
the flag `-i' or the import builtin changes the language in detail
using a concrete example of an ecosystem and why it is the best
approach among other different possibilities. So far I don't think the
`-i' flag would have such a large impact because it can be easily
emulated by the existing feature.