Re: [Discuss-gnuradio] Changes to the development model

[CEL]

Yes, I wished I had a definite number at this point; we'll probably
actually will orient on the release cycles of the more popular Linux
distros, but, to be honest, I don't think it'd be 100% appropriate to
promise that "we will _fully_ support 3.7 till 2023, because it's what
Ubuntu 18.04 LTS ships" at this point. That's just too long a timeline
to make certain statements about. Also, I'd argue that people really
into long-term support will probably rather use RHEL or CentOS, but
that's just a wild guess, and certainly doesn't make the support
timeframe shorter. An interesting question really also is how
interested users of 3.7 in e.g. two years will be in more bugfixing –
at that point, you're probably using a specific piece of software that
relies on specific, potentially buggy, behaviour of your framework, so
you might be hesitant to even install bugfix updates.

So, the honest answer really is: we'll be listening to our users before
we shut anything down. Also, "not actively maintained" doesn't mean
"breaks instantly"; if you're on an old platform using old software
with an old GNU Radio release, nothing would ever break (I do know a
couple of GNU Radio 3.4.2 users. You can't even build 3.4.2 on a modern
Linux without a lot of handholding, but they are happy with their
current installation and don't *need* updates).
The interesting question really is how to move forward without
sacrificing all backwards compatibility in the future, and how to make
it easy for users of previous versions to take advantage of new
developments without burdening down the development process so that GNU
Radio becomes obsolete by relying on obsolete dependencies. That's why
for 3.8, not only the move to Qt5 is on our agenda, but also the move
to Python3, and away from some of the dependencies that we used to
carry around in-tree. Pretty exciting stuff, but an unavoidable change
in API. 
Still, I'm confident that most of GNU Radio itself translates to this
new dependencies relatively smoothly in the next weeks, and that the
OOT and flow graph ecosystem will in fact adapt with relative ease –
we've literally got a heap of work in the merge pipeline that makes a
lot of this saner than it used to be. So, I'm afraid 3.8 will bring
quite a bit of breakage, but we've got the best glue to put everyone's
applications back together, and resolve a couple of the stranger things
that people had to do to make things work under 3.7.

With that, I'll leave the discussion for tonight; in the end, what we
do doesn't only depend on the stable release cycle vision that I have,
but simply on how demand for new versions and features turns out, and
very importantly, on how many developers can dedicate steady time into
the project, and on which parts they work. A long term goal is
definitely to upstream more commonly reinvented blocks – making the
project more complete ("batteries included"), but also allowing us to
actually notice when we break applications. So, and I know this really
doesn't apply to you overly much, as you've been working with the rest
of GR very intimately on your code, the best way to make sure that
it'll be easy to run your own code with GNU Radio of 2023 will be
making sure that your blocks are universally useful, and getting them
into the main GNU Radio source tree medium- to longterm. That way, the
responsibility of making it work with the next release of GNU Radio
becomes part of the maintenance cycle – and hence, of automated
testing, influencing patches and feature branches to be merged, whilst
saving you developer time (and money, maybe?) by only bringing your
code up to current GNU Radio standards once instead of perpetually.
That, I hope, works rather well because stakeholders simply contribute
their code, ensuring that their interests are being respected on a
technical level; in exchange for their contribution and time caring for
the more specific aspects of their code later on, the GNU Radio project
offers them long-term integration testing and thus, stability.
This is going to be an interesting balance between upstreaming as much
as feasible and keeping the amount of cruft and in-tree duplication at
bay, whilst simultaneously keeping dependencies manageable. On a
packaging level, that might mean that we will opt for more modular
builds with separately installable gnuradio-runtime, gr-uhd, gr-qtgui,
gr-dtv, gr-world-dominance-over-the-web etc with separate dependencies
each. Some distros actually already have a pretty good way of doing
that, and I'd like to technologically foster that in our build system.
Maybe that actually means that we might be able to declare even stuff
that has far-off-core-dependencies to become part of the official GNU
Radio source tree one day. We'll see how this plays out!

I'm pretty set on not making any rash decisions in that direction.
There'll be plenty of discussion revolving around how to make sure we
don't break everyone's application whilst still gathering enough
developers, and as said, interaction with the people that actually use
GNU Radio is key here. That's why I specifically asked about you being
worried about something!

Best regards,
Marcus
On Mon, 2018-02-26 at 16:49 -0600, Kartik Patel wrote:
> Hello Marcus,
> 
> All I wanted to know was the typical time frame of the maint-X.Y
> branch (not 3.7 in specific). I believe the answer is (from your
> reply), it will depend on the particular version. Maybe maint-3.7
> will have a long life and other maint-X.Y may not. 
> 
> With the Debian and QT4 example, it's more clear on why we need this.
> I am not worried about anything in particular though. It was a
> general question for more information on this workflow.
> 
> Thanks for the detailed response. :)
> 
> Regards,
> Kartik Patel
> 
> 
> On Mon, Feb 26, 2018 at 4:18 PM, Müller, Marcus (CEL) <address@hidden
> u> wrote:
> > Hello Kartik,
> > 
> > there will not be a "maint-3.7.12", there will only be a "maint-
> > 3.7";
> > that's enough fragmentation for me :)
> > 
> > Yes, it will be actively maintained for a finite time, just like
> > anything in this world :) That timeframe probably will be a bit
> > longer
> > than for future maint-X.Y branches, as 3.7 has been around for so
> > very
> > long that there's by now a lot of software that actually relies on
> > all
> > the special kinks and features that GNU Radio 3.7 has, and we don't
> > want to stop supporting any of that!
> > Still, it's important to be forward-facing: Dependencies are always
> > moving, and for example, without extensive manual labour of
> > Maitland,
> > GNU Radio 3.7 would already have been thrown out of Debian because
> > it's
> > still using Qt4! We must work on all contributions, especially
> > user-
> > facing stuff like GUI frontends, become compatible with 3.8; don't
> > worry, we won't realease 3.8 without making sure it's usable, nor
> > will
> > we abandon versions that are widely used.
> > 
> > What I cannot tell you at this point is how many years the support
> > for
> > maint-3.7 will go on. Time will tell, and if there's high demand,
> > we
> > might at some point even find a maintainer just for that branch
> > who's
> > job would be to backport patches and keep 3.7 alive for those who
> > can't
> > switch. Again, this is by no means us trying to abandon existing
> > users,
> > this is us being very worried that you won't even be able to build
> > upstream GNU Radio on mainstream linux distros for much longer, and
> > actively working on making it work for the future, thus, ensuring
> > that
> > applications continue to run, with as little modification as
> > possible.
> > 
> > Does that answer your question? Are you worried about something in
> > particular? This is an excellent time to come forward about that,
> > either here or in private, if it's something sensitive; the GR
> > leadership is more than interested in not breaking everyone's use
> > case!
> > 
> > Best regards,
> > Marcus
> > 
> > On Mon, 2018-02-26 at 11:16 -0600, Kartik Patel wrote:
> > > Hello Marcus,
> > >
> > > I believe this workflow is an excellent improvement and stepping
> > > stone to acknowledge that GNU Radio is now a "medium-to-large"
> > size
> > > projects.
> > >
> > > Only thing I did not understand was the connection with Linux
> > > Distros. I think you want to say that suppose 3.7.12 version is
> > > released, then the corresponding maint-3.7.12 will be active only
> > for
> > > a given timeframe? Is that correct? How long would that timeframe
> > be?
> > >
> > > Looking forward to the blog post!
> > >
> > > Regards,
> > > Kartik Patel
> > >
> > > On Mon, Feb 26, 2018 at 10:49 AM, Müller, Marcus (CEL) <address@hidden
> > it.e
> > > du> wrote:
> > > > Hello everyone,
> > > >
> > > > as the last days have been pretty heavy on discussion between
> > the
> > > > new
> > > > GNU Radio lead team, it’s now become clearer how we’ll actually
> > > > deal
> > > > with the day-to-day maintenance work as well with the release
> > > > management.
> > > >
> > > > So, as you might have noticed, we’ve been crunching through the
> > > > Pull
> > > > Request Queue on github. We weren’t able to merge all the PRs,
> > but
> > > > some
> > > > of these are blocked by the legal side of things, others simply
> > > > need
> > > > small tweaks. What we were, however, able to do: Defining what
> > > > should
> > > > be in release 3.7.12 (not all of this is visible on the PR
> > tracker,
> > > > but
> > > > we have a pretty good idea about it). So, once these remaining
> > PRs
> > > > are
> > > > closed, and all issues tagged with this milestone are solved,
> > > > 3.7.12
> > > > will be released.
> > > >
> > > > That release will also mark a radical change to the git
> > workflow
> > > > that
> > > > we’ll stick to:
> > > >
> > > > We’re killing the idea that you usually submit PRs against
> > maint.
> > > > In
> > > > fact, there won’t be a maint branch anymore:
> > > >
> > > > New releases will be tagged from the master branch. That means
> > once
> > > > we
> > > > released a version, for example 4.10, there will be a maint-
> > 4.10
> > > > branch, onto which we can merge fixes.
> > > > As versioning scheme, we’ll be adhering to Semantic Versioning,
> > > > with
> > > > the first digit being the "paradigm" digit ("3" for the time
> > > > being),
> > > > the second the "API" digit, meaning that we won't change how
> > > > programmers use GNU Radio without increasing the second digit,
> > the
> > > > third being the "ABI" digit, meaning that as long as the first
> > > > three
> > > > digit stay the same, you can just replace one libgnuradio*.so
> > with
> > > > another one without rebuilding your binary (that's a feature
> > that I
> > > > think software distributors will be most happy about), and the
> > > > fourth
> > > > digit being the patchlevel.
> > > >
> > > > The lifetime of these branches will reflect the life time of
> > the
> > > > (primarily) Linux distributions that ship that package; it’s
> > our
> > > > outspoken goal to not break GNU Radio for anyone who uses it on
> > > > widely
> > > > used, actively maintained platforms. This will necessitate
> > > > maintenance
> > > > of Long-Term Support releases.
> > > >
> > > > Development happens on and against master. If there is a bug
> > fix,
> > > > we do
> > > > that on master (if it applies to master, at least), and
> > backport
> > > > fixes
> > > > to the maint-X.Y branches that we still support. This requires
> > a
> > > > bit of
> > > > consequence from PR authors: If you do happen to submit a PR
> > that
> > > > contains a bug fix, please do note that in the PR itself, and
> > make
> > > > the
> > > > bugfix a commit of its own, so that it's easily cherry-pickable 
> > on
> > > > the
> > > > appropriate maint-X.Y branch. We don't know yet whether that'll
> > be
> > > > easy
> > > > for every possible bugfix out there, but that's something we'd
> > have
> > > > to
> > > > figure out from case to case, anyways.
> > > >
> > > > These branches are there so that distros etc. can get GNU Radio
> > > > bugfixes, and we don't force users to upgrade GNU Radio to
> > > > "Bleeding
> > > > Edge" just to get a bugfix.
> > > >
> > > > How does the "next" branch fit into this? Long story short: in
> > its
> > > > current shape, it doesn’t. As soon we released the next 3.7.12
> > > > release
> > > > (which means tagging master “v3.7.12.0”), “master” becomes
> > “maint-
> > > > 3.7”;
> > > > “next” becomes “master”. The following release that we’ll do is
> > > > 3.8, if
> > > > all goes well (but honestly, at this point, what shouldn’t?).
> > > > The v3.8.0.0 tagged commit will also the anchor of the “maint-
> > 3.8”
> > > > branch. Note that it’s not “maint-3.8.0.0”. We'll limit the
> > number
> > > > of
> > > > heads we have to merge things into as far as possible (very
> > much in
> > > > the
> > > > sense of not letting this become a hydra).
> > > >
> > > > Obviously, this implies that we have to increase the X.Y
> > release
> > > > frequency. But: I think that’s totally doable. Only thing we
> > need
> > > > to
> > > > make sure is that our changelogs are really good enough for
> > people
> > > > to
> > > > track when features were added.
> > > >
> > > > This all isn’t really easy to grok for people who’ve not worked
> > > > with
> > > > medium-to-large git projects before, so I’ll have to make a
> > nice
> > > > drawing and a blog post, but I recon it’s better to keep the
> > > > mailing
> > > > list updated now than having a nice blog post in a month.
> > > >
> > > > So, looking forward to countless PRs,
> > > >
> > > > Marcus
> > > > _______________________________________________
> > > > Discuss-gnuradio mailing list
> > > > address@hidden
> > > > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> > > >
> > >
> > > _______________________________________________
> > > Discuss-gnuradio mailing list
> > > address@hidden
> > > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> 
> _______________________________________________
> Discuss-gnuradio mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

smime.p7s
Description: S/MIME cryptographic signature