Re: Sync up the org in emacs master to org maint branch?

[David Engster]

Edward John Steere writes:
>> It's not like packages communicate with Emacs over a well
>> defined RESTful interface. In other words: CEDET and Gnus are not
>> loosely coupled, quite the opposite: they are extremely dependend on
>> many obscure Emacs internals (not sure about Org).
>
> Shouldn't we be trying to avoid this situation?  It's sure to come back
> and bite us in the future.  If we continue to develop a package
> (wherever it ends up being developed) which is so tightly coupled that
> any kind of re factoring in core becomes a nightmare, because we have to
> consider the umpteen ways in which it'll break the package, then surely
> that's a bad methodology to continue?

I don't know what you have in mind. All I can say is: CEDET couldn't do
what it does if we'd restrict ourselves to some subset of Emacs.

>> As a consequence, we and also the Gnus guys decided to not do
>> separate releases anymore.  We used to provide CEDET for different
>> Emacs versions, and it was a *huge* amount of work. I had a buildbot
>> running with 7 or 8 Emacs versions to run the test suite, and things
>> broke pretty regularly.  Now you might say: fine, only release a
>> package for current master. But let's say we put CEDET into
>> ELPA. Emacs 26 gets released, and work on Emacs 27 starts. Now there
>> are changes happening in Emacs 27 that require changes in CEDET,
>> which make it incompatible with Emacs 26. So you have to create two
>> packages: one for 26, one for 27? Not only is this confusing, but it
>> most definitely increases my workload.
>
> I feel like this problem isn't intractable.

I didn't say it's intractable. I just said it means more work for me.

> We can mark some state of CEDET as being stable under the various
> versions of Emacs (because it was at the time) and then only support
> the current release for the latest package.  This would most likely
> require changes to package to ensure that you get an appropriate
> version of the package when you download it.

As I said: we did that. It was a huge amount of work. Please understand
where I'm coming from: if you look through the CEDET history, you will
see that in the past few years I almost exclusively did infrastructure
work and no real coding. All I can say is: *I* won't do this anymore,
and I don't want to be part in something which will increase grunt
work. We did not make this decision lightly. But with the amount of
developers we have, I'm convinced it's the right thing to do.

> Consider the problem which our users currently face.  The built in CEDET
> is miles behind and missing very important bug fixes and features.  The
> upstream CEDET can be a real pain to setup, but it has the latest
> features.  This is not a good place to be.  If we merge CEDET in and
> only release it with the realeses of Emacs then we are heading for a
> state where you only get the new features and bug fixes every time Emacs
> is realesed, i.e. our users might have to wait up to two years to have
> something fixed.  This is also a bad place to be.

Bug fixes can go with point release, which we should have every
year. But yes, if you want the latest, greatest and buggiest, you'll
have to use Emacs master. But that goes for a lot of Emacs features, so
I don't see why it's particularly bad for CEDET.

>>> We can arrange things so that a Git clone of Emacs includes pulling the
>>> submodules (or trees, or ELPA.git, or what not) that are considered part of
>>> "main Emacs development", including some of those batteries. I see this all 
>>> as
>>> a process issue, and that "living in one Git repository" has just been an
>>> implementation strategy to satisfy that process.
>>
>> Obviously, I'm very skeptical towards such an approach. Our tooling
>> around Emacs development is already very intricate and only works
>> because a few people work quietly behind the scenes to keep this all
>> running. Introducing even more complexity through
>> submodules/subtrees/whatever will do the opposite of what you want to
>> achieve: it creates more work for those few people who manage the Emacs
>> infrastructure. But I'd love to hear what for instance Glenn and Paul
>> think about this.
>
> I'm interested in exploring more with regards to how the subtree
> approach would work.  In particular how it would impact the Makefiles
> and build process.  I don't think that introducing features like this
> necessarily increases the burden of maintaining our tooling.  If we get
> it right it could reduce it.

Well, we cannot really discuss this since there's no real plan how this
all should work. I can only speak from experience.

> I think that it's a worthwhile goal to make core smaller.  It may not be
> a gigantic enterprise system with tens of thousands of source files,
> like some of us are accustomed to working on, but I think that it
> becomes easier to reason about the features and behaviour of core when
> it's smaller.

How does CEDET, Gnus and Org affect the rest of Emacs? They strongly
depend on Emacs "core" (whatever exactly that is), not the other way
round.

>>> updating packages within it is no longer a major issue, and (I hope)
>>> it will be clearer when something is a core issue vs. a package issue.
>>
>> I find this whole core vs package argument strange. If you ship Emacs
>> with Org, Gnus and CEDET, they are part of Emacs, so it's in the
>> interest of all Emacs developers that they work well, whether they use
>> them or not. The users won't care if they originate from a separate repo
>> and are considered a "package".
>
> I think that the distinction becomes clearer when you consider what
> other parts of Emacs ought to be able to depend on.  If mode-x started
> building dependencies on CEDET because the author discovered some useful
> functions in CEDET.  Then mode-x would build a dependency which is
> difficult to maintain because changes in CEDET might have unintended
> effects on mode-x.  If the function really is useful, then I think
> that we should instead consider extracting it from CEDET and placing it
> into some core library.  As far as I can tell this has already happened
> with numerous functions which originated from CEDET.

Aside from EIEIO, I wouldn't know any. And with EIEIO it had exactly the
opposite effect: CEDET has to adapt, not the other way round.

> I don't think that anyone is suggesting that we "copy them over and hope
> for the best."  This would have to form part of the QA process.  As for
> testing -- I would be surprised if CEDET in core has really received the
> amount of testing it needs to declare that it's a stable release.

Well, that's precisely why I want to move development to the Emacs
repository.

-David