Thomas,
I think this thread needs to be opened for wider discussion. As you are
basically question the need for Octave to have a package manager at all.
This is something that was extensively discussed over the last few years
and the consensus was that it is something that is needed to make
distribution of software for Octave easier for users. This goal should
also make distribution not difficult for debian either.
In summary I believe that you have made several mis-assumption is the
way the package manager works and the split of the responsibility
between the package and the package manager. Basically some of the
things you complain about for the package manager were either not
possible previous either or are perfectly feasible even now.
Thomas Weber wrote:
> Hi,
>
> Am Montag, 4. Juni 2007 00:27 schrieb David Bateman:
>
>> I strongly suggest against not using pkg.m to do the major part of the
>> work, as otherwise debian users will end up with packages that behave
>> different than on other systems.
>>
>
> Time to answer, I guess. This mail will be long, so I give a short summary:
> a) I had only a cursory look at the attached example (though I did study the
> earlier example you gave quite extensively)
> and
> b) I won't use pkg.m for creating Debian package [0]
>
This is a shame. I'd hoped that the sample SRPM and SPEC I'd sent
(attached for those interested), that I'm currently using on my own
machine now with the premise of eating my own dog food, would be
sufficient to show the way for a debian package. I have no debian
machine to test with or experience in debian packaging and so cn't
easily do the debian packaging myself. Otherwise I would already have
produced a debian source package for you to examine.
Again, if you go the way of not using the Octave package manager under
debian, then the experience of Octave for the debian users will be
different than for other systems, which can only detract from both
Octave and debian. So I put to you that the above statement essential
comes down to saying that Octave should completely remove its package
manager.
> So, where to begin for the reasoning...
>
> I guess I best start at the end. This means the OctPlot 0.4.0 release (sorry,
> Shai). It came out while I was already working on creating packages using
> pkg.m. Suffice to say that I failed miserably with that approach: I worked
> for weeks (when I had time, which is not often and part of the problem, see
> below) on this and didn't really achieve much. Then Shai released Octplot: I
> had it patched and packaged in only a few hours. Most of this time was spent
> searching on how to add a "-lGLU" to configure.ac[1]. At that point I
> realized that something was fundamentally wrong with my approach.
>
>
May I put to you that you are already experienced with the install
process of Octplot and so the investment of time needed to package it is
minimum for you. I see here that you are criticizing that Octave
packaging process is new rather than any fundamental issue with the
process itself. Sorry I choose to ignore the above..
> Now, if you have problems with packaging a software, we have a debian-mentors
> list. And then I realized that asking there was useless: changes are that of
> the 1000+ Debian developers, Rafael, Dirk and I are the only one who work(ed)
> with Octave and can therefore easily read pkg.m.
>
I also put to you that the list address@hidden serves the same
purpose for the Octave package manager. Yes the combination of the two
might have particular concerns, that we must address and unfortunately
the right person to address them hasn't been found.
> Which brings me to the core problem: pkg.m is Octave-specific.
An Octave package manager must be Octave specific if it is to also
Octave user Lambda, whether he is running on Debian, Fedora or Windows
to be able to create a package for Octave that he can distribute to
other Octave users..
> It's so
> specific that probably less than ten people are actively working with it (in
> the sense of changing, normal usage is outside of my scope).
Its only a few years ago (certainly less than 15 years) that both RPM
and DEB packages were new. So what, if we accept the need for Octave
packages at all, then the users will come if Octave itself survives.
> It combines
> autoconf with one of the strictest systems I have seen: there's almost no
> chance for the normal user/packager to change anything:
>
I disagree completely... The autoconf/configure/make process is under
the control of the package itself and not the package manager. The
configure scripts for the packages are all based on the original
octave-forge configuration script. Therefore anything you could do with
the octave-forge process you should be able to do with the packages
themselves.
> - want a different compiler => patch pkg.m
>
Wrong.. Firstly is it a good idea to set the compiler to something
different than what was used to compile octave itself... The old
octave-forge process for this was in configure to have
CC=`$MKOCTFILE -p CC`
mkoctfile has
: ${CC=%OCTAVE_CONF_CC%}
so the default is to take the same compiler as used to compile Octave,
but respect the CC environment variable..
In any case all of this is under the control of the packages themselves
and their configure/Makeconf files.. If you have a problem with this
then a patch is needed for the individual package and not the package
manager.
> - need to pass something to ./configure => patch pkg.m
>
Ok, yes there is no way currently to pass additional information to the
configuration process from Octave itself, except as noted above through
the environment variables.. I'm perfectly willing to implement this
feature myself.. I'd process a syntax like
pkg("install","image-1.0.0.tar.gz","-configure","configuration
flags","-make","make flags");
this might be used in commandline form as
pkg install image-1.0.0.tar.gz -configure configure_flag -make make_flag
but in that case each and every configure/make flag would need to be
prepended with "-configure" or "-make".. Would that be acceptable? If so
I'll give you a patch in a few days.
> - want to install in a different place (which with autoconf normally is just
> a matter of setting DESTDIR) => change global and local files
>
Octave has to know where the packages are installed. It uses the
octave_packages files (controlled by the global and local lists) to
determine where to find these files. At the moment the package manager
then forces the install location to be in the same directory as where
the octave_packages file is found. No such assumption exists on loading
the file, as the octave_packages file itself contains the name of the
directory where the package is installed.
So yes, there is nothing to stop us adding an option to force the
installation location to be somewhere else than the same directory as
the octave_packages file. This would break the "pkg rebuild" option that
Michael uses to install under windows, and this would need to be
rethought (perhaps making the rebuild based on a sub-directories of a
directory rather than the location of the octave_packages file.
However, the question I ask is why is this important to you? I added the
option to allow architecture dependent directories to respect the LFS.
The location of the octave packages themselves will be determined by the
PREFIX variable you used to compile Octave itself. So indirectly you
have control of the installation location. If you can explain the exact
needs you have, then I have no objections to implementing something like
pkg install image-1.0.0.tar.gz -destdir <prefix>
pkg rebuild <destdir>
> It ignores almost anything I'm used to when building software under Linux: I
> just expect a build system to honor "CC" and "CXX" environment variables or
> something like "make -j2" (and probably a dozen things more, but these just
> hurt me during packaging).
>
The respect of CC and CXX as noted above is a question for the packages
themselves and not the package manager. Most of the packages respect the
CC and CXX environment variables.. The "make -j2" issue I can address
easily with the above proposed change.
> On the other hand, it has so many optional components, that almost nothing is
> guaranteed to exist; the only mandatory components of a package are
> DESCRIPTION and COPYING. I guess one could shorten pkg.m by a hundred lines
> just by demanding that doc/, inst/, src/ and bin/ directories exist (they may
> be empty) and that all *_install.m files exist (they may be empty as well),
> but I digress.
>
The goal is to make it easy for the users to make a package very simply,
and making files they don't need non optional doesn't go in that
direction. I accept the cruft in the name of easing the users life..
> Now, the obvious answer is: let's change pkg.m. And the even more obvious
> answer from me: that won't change anything[2]. We currently have about 10
> packages, that already come with lots of different systems
> - autoconf without automake
> - autoconf with automake
> - pure Makefiles
> - cmake
>
> I just don't want to learn another system. Every added system makes it more
> difficult to package stuff and it makes it also more difficult to find other
> people who would like to help.
>
So, you advocate throwing the package manager out entirely. I disagree
that you have to learn much. Take a look at the relevant parts of the
proposed SPEC file
%build
cd image-1.0.1
HOME=`pwd` %{octave} -H -q --no-site-file --eval
"pkg('build','-nodeps','-verbose','tmp','image-1.0.1.tar.gz');"
cd tmp
tar xf image-1.0.1.tar
cat << EOF > image/on_uninstall.m
function on_uninstall (desc)
error ("Can not uninstall image-1.0.1 installed by the RPM package
manager");
endfunction
EOF
tar czf ../%{builtpackage} image
There is one octave specific command (and I supplied you don't need to
learn it).... Most of the code is just to install an on_uninstall.m file
so the user can't remove the package without using apt. Thinking about
it the above needs a small modification to allow the case of an existing
on_uninstall.m file to be respected, though none exist in octave-forge
at the moment. I suppose it might even be possible to get pkg itself to
manage the on_uninstall.m files, though hey its only 5 lines of code in
the SPEC file..
%install
cd image-1.0.1
rm -rf %{buildroot}
mkdir -p %{buildroot}%{_datadir}
cp %{builtpackage} %{buildroot}%{_datadir}
Put the binary version of the package in a specific location that can be
arbitrary.
%post
%{octave} -H -q --no-site-file --eval
"warning('off','all');pkg('install','-global','%{_datadir}/%{builtpackage}')"
rm %{_datadir}/%{builtpackage}
and then use Octave's package manager to install the prebuilt file and
remove the binary package file... I don't see how I can make this
process much simpler for a debian package. What do you really need to
learn? Is what you need to learn, offset by the ability of an Octave
user to create and distribute their own packages independent of the OS
or distribution on which they are running?
> Which leads to the next problem: scalability. If we package everything in
> octave-forge[3], we will go from 10 packages to well over 50. Each of these
> 40 packages will need a recompile for every new Octave release. If I only
> need two minutes per package, that already is well over an hour (if
> everything goes well - with the exception of octave-epstk, I have yet to see
> this happen. Even if it is just me forgetting to add something to the
> changelog). Add to that a build system that no one else uses and we have a
> recipe for desaster.
>
Huhhhh? How does going from one monolithic package with a number of
sub-packages that each might break individually, to a number of separate
packages that might break individually multiply your work.. The
monolithic versions of octave-forge were built with "make -k" to prevent
individual package breakages from preventing the building of the rest of
octave-forge, and so to me the case was in fact worse previous.. You
might have a breakage, but not be aware of it.. So in terms of the
octave-forge specific parts I see no difference in the amount of work
now or previously..
The majority of the rpm/debian specific code for a package is identical
between packages, and so this is a case where a script to write your
package is the right way to go, which is exactly what I did for the
SRPMs in octave-forge. This script is less than 250 lines comments and
blank lines included. I'd hope that the debian build code use similar
tools to reduce the workload for the packaging specific task.
I believe you are putting too much influence on what is the Octave build
system here. Keep it simple, and I believe the examples I supply above
do indeed make the whole thing simple..
> So, what do I want? Ideally, a simple
> ./configure
> make
> make install
> Well known, tons of documentation out there and in every distribution you have
> several gurus who can help you with this stuff.
> Now, I guess I won't get this :)
>
Yes, but Octave is not just appealing to the guru's. We want to make it
easy for non guru users to make their own packages for distribution. You
serious expect a Windows users who just wants to write a package with a
few m-files, to learn autoconf and perhaps automake to be able to do
so??? Who are we writing the code for, the packager or the user?
> But clearly, "Debian users will end up with packages that behave different
> than somewhere else" is the least of my concerns. If I use pkg.m, they won't
> have any packages at all, because I will freak out at some point on the way.
>
As far as I can see you already have. If I knew debian packaging I would
have already have supplied you with a sample of how I think it should be
done. Anyone out there want to supply me with the equivalent of a
template source debian package that has post-install and pre-uninstall
scripts? If so I might be able to write an example, but not test it.. If
someone could show be how to pin octave 2.9.12 on my Knoppmyth machine,
I might even be able to test it (though I didn't want to touch a
production machine).
> [0] Others might think different. They are welcome to create packages and to
> join the "active team of Debian packagers" (thanks Jordi, that really made my
> day), although I guess that "overworked and short on time just like everybody
> else" would be a better description.
>
I can't figure out the reference you are refer
> [1] There's a Debian specific change to "fltk-config", so I needed to patch
> configure.ac.
>
Octplot's problem :-)
> [2] Everything that's needed is already available through autoconf. I don't
> see any sense in throwing more code at a problem, that exists just because of
> this very code.
>
When windows user lambda learns autoconf, I'd agree that this is a solution.
> [3] I don't see this happen. There are packages that consist of one (1!)
> m-file. I believe there's zero chance our ftp-masters will let such packages
> in.
>
Why? Yes I agree that a package with a single m-file is a bit of an
extreme case, and perhaps such packages should disappear into other
larger packages..
> Regards
> Thomas
>
>
Thomas, if the consensus the Octave developers is that packages are
necessary for the future of Octave, then we must find a way to make this
work for debian. Note that the originator of this idea is Paul Kienzle.
I'm willing to do a significant portion of the work myself to make this
work for debian, and can envisage any change to pkg that you perceive as
necessary. However, looking at your message "On NOT packaging
octave-forge"
(http://lists.alioth.debian.org/pipermail/pkg-octave-devel/2007-June/002452.html)
<quote>
E.g., in the following situation, the octave-audio package is installed and
enabled (the first line is a symlink to the second, which contains the
actual
file)
/etc/octave.d/enabled/octave-audio => /etc/octave.d/available/octave-audio
/etc/octave.d/available/octave-audio
If the user wants to override this, he must call
debian_remove_package("/etc/octave.d/available/octave-audio")
</quote>
I believe you are adding a lot of additional cruft. If the user installs
with "apt-get" an octave-forge package, then why not just let then use
"pkg list" to see what is available and loaded and "pkg load" and "pkg
unload" to modify what packages are actually on the path. If you accept
this situation then the code sample from my SPEC files I supply above is
relatively simple.. Why is it that you don't like it?
Regards
David
--
David Bateman address@hidden
Motorola Labs - Paris +33 1 69 35 48 04 (Ph)
Parc Les Algorithmes, Commune de St Aubin +33 6 72 01 06 33 (Mob)
91193 Gif-Sur-Yvette FRANCE +33 1 69 35 77 01 (Fax)
The information contained in this communication has been classified as:
[x] General Business Information
[ ] Motorola Internal Use Only
[ ] Motorola Confidential Proprietary
Name: octave_image
Version: 1.0.1
Release: 1%{?distsuffix}
Summary: Image Processing
Group: Applications/Engineering
License: GPL version 2 or later
URL: http://octave.sourceforge.net
Source0: http://downloads.sourceforge.net/octave/image-1.0.1.tar.gz
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root
## Package installs in a directory tree specific to the installed
## version of octave, so the following version dependency is necessary.
%define octave_ep %(rpm -q --qf '%%{epoch}' octave)
%define octave_ver %(rpm -q --qf '%%{version}' octave)
Requires: octave = %{octave_ep}:%{octave_ver}
Requires: ImageMagick
BuildRequires: octave >= 2.9.7
BuildRequires: gcc-c++ gcc-gfortran libstdc++-devel
BuildRequires: libstdc++-devel
BuildRequires: %(rpm -q --qf '%%{name}' %(rpm -q -f `which mkoctfile`))
BuildRequires: png-devel
BuildRequires: jpeg-devel
%define builtpackage image-1.0.1.tar.gz
%define octave octave-%{octave_ver}
%description
The Octave-forge Image package provides functions for reading, writing, and
processing images. The package supports almost all image formats through the
use of ImageMagick. The package also provides functions for feature extraction,
image statistics, spatial and geometric transformations, morphological
operations, linear filtering, and much more.
%prep
rm -fr image-1.0.1
mkdir image-1.0.1
cd image-1.0.1
cp %{_sourcedir}/image-1.0.1.tar.gz .
chmod a+r image-1.0.1.tar.gz
mkdir tmp
%build
cd image-1.0.1
HOME=`pwd` %{octave} -H -q --no-site-file --eval
"pkg('build','-nodeps','-verbose','tmp','image-1.0.1.tar.gz');"
cd tmp
tar xf image-1.0.1.tar
cat << EOF > image/on_uninstall.m
function on_uninstall (desc)
error ("Can not uninstall image-1.0.1 installed by the RPM package manager");
endfunction
EOF
tar czf ../%{builtpackage} image
%install
cd image-1.0.1
rm -rf %{buildroot}
mkdir -p %{buildroot}%{_datadir}
cp %{builtpackage} %{buildroot}%{_datadir}
%post
%{octave} -H -q --no-site-file --eval
"warning('off','all');pkg('install','-global','%{_datadir}/%{builtpackage}')"
rm %{_datadir}/%{builtpackage}
%postun
%{octave} -H -q --no-site-file --eval
"prefix=pkg('prefix');unlink(fullfile(prefix,'image-1.0.1','packinfo','on_uninstall.m'));pkg('uninstall','image')"
%clean
rm -fr %{buildroot}
%files
%defattr(-,root,root)
%{_datadir}/*
%changelog
# Changelog for packages rebuilt on stable releases (leave it first):
%{!?_with_unstable:* %(LC_ALL=C date +"%a %b %d %Y") %{packager}
%{version}-%{release}}
%{!?_with_unstable: - rebuild of %{version}-%{rel}%{?distsuffix} for
%{distversion}}
* Mon Jun 4 2007 The Octave Community 1.0.1-1%{?distsuffix}
- Automatically created from octave-forge package