Re: Conda packaging (Ethan C)

Hello Ethan -

I had a chance to try this out today to see how it all works together. Your instructions were very clear, and I had Miniforge installed easily and then created the gnustep environment. It all works as advertised! After enabling the gnustep environment with

$ conda activate gnustep

I typed

$ openapp SystemPreferences

and it fired right up. Also,

$ gnustep-config

showed all the flags and compilers used.

I wasn't sure if this process should result in a development environment though. I do not believe "clang" was installed as a by-product of installing the base packages. I wanted to try building some of my own code, so I installed clang. Another problem I ran into when compiling an ObjC++ file was a missing c++ header. (clang-18 seems to need g++-14 headers ...)

The two dependencies I ended up needing were

$ sudo apt install clang g++-14

and then I was able to compile my project code.

Perhaps a "build" environment is a different type of beast anyway and there should be a separate package called "gnustep-developer" that ensures the compilers are all there.

Thanks for this work and the nice writeup on how to use it! This is a very nice way to get an environment up and running quickly.

Regards,

Tom

On Mon, Dec 23, 2024 at 7:04 PM <discuss-gnustep-request@gnu.org> wrote:

Send Discuss-gnustep mailing list submissions to
discuss-gnustep@gnu.org

To subscribe or unsubscribe via the World Wide Web, visit
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
or, via email, send a message with subject or body 'help' to
discuss-gnustep-request@gnu.org

You can reach the person managing the list at
discuss-gnustep-owner@gnu.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Discuss-gnustep digest..."

Today's Topics:

1. Conda packaging (Ethan C)

----------------------------------------------------------------------

Message: 1
Date: Mon, 23 Dec 2024 21:03:41 -0600
From: Ethan C <echaroenpitaks@gmail.com>
To: discuss-gnustep@gnu.org
Subject: Conda packaging
Message-ID: <fae52c3c-beb7-49f6-9561-0cd3bfadf869@gmail.com" target="_blank">fae52c3c-beb7-49f6-9561-0cd3bfadf869@gmail.com>
Content-Type: text/plain; charset="utf-8"; Format="flowed"

Hi everyone,

I wanted to introduce my recent work on Conda packaging of GNUstep.
Conda, and its relatives|mamba|,|micromamba|, and|pixi|, are
cross-platform package managers which can manage "environments". These
environments are similar to Python's "virtual environments", as they
allow you to create environments that contain certain packages, which
will only be available after the environment is activated. This way, you
can install different versions of the same software in different
environments, reduce dependency hell, and share your environments with
others in order to allow them to easily get a working setup. Unlike
Python's "virtual environments", it is able to manage both Python
software and native software like shared objects, configuration files,
and native executables. This has made it very popular amongst Python
datascience and machine learning users which often need to manage Python
packages with C, Rust, and Fortran dependencies tightly coupled to the
Python packages and the Python interpreter. Conda, along
with|nix|and|guix|, are the only package managers I know of that are
able to have multiple environments which stack on top of the user's
operating system (as opposed to a chroot).

For GNUstep, this can make it a lot easier for new users to get started,
and it could also make it easier to distribute GNUstep software. GNUstep
libraries could be distributed as conda packages, and GNUstep apps could
be distributed as conda packages which could then be turned into other
formats. Conda environments can already be converted into AppImages, but
the scripts to do so are kind of outdated and depend on the
Anaconda|defaults|channel and the Miniconda installer. I intend to work
on improving that and to make more output formats.

I have packaged the core GNUstep software (in
the|ng-gnu-gnu|configuration; Clang 18 as compiler,|ld.gold|as
linker,|libdispatch|,|libobjc2|as linker, and using GNUstep Base for
Foundation and GNUstep GUI for AppKit) on top
of|conda-forge|for|linux-64|(glibc Linux on x86_64). glibc Linux on
arm64 and ppc64le, macOS on x86_64 and arm64, and Windows on x86_64 are
also supported by|conda-forge|. The package definitions are available
athttps://github.com/ethanc8/gnustep-forge-feedstocks/and the binary
packages are available athttps://prefix.dev/channels/gnustep-forge.

Remaining contents

* More information on Conda ecosystem
* TODO
* Quick start/tutorial
* Packaging and distributing your own software

More information on Conda ecosystem

There are two "distributions" of conda packages in the
package-repository sense. Anaconda, Inc. publishes the Anaconda and
Miniconda installers, and publish the|defaults|channel (package
repository). It is released with a EULA that prohibits certain
commercial uses, and is missing many useful packages.
Meanwhile,|conda-forge|is a community distribution which includes many
more packages and is quite popular, but is ABI-incompatible
with|defaults|. They publish the Miniforge installer.

|conda|is written in Python, and is kind of slow.|mamba|is a faster
reimplementation of|conda|in C++.|micromamba|is based on|mamba|but does
not need its own conda environment with Python installed in order to
work.|pixi|is a more|npm|/|cargo|-style package manager, which is very
fast and written in Rust, and which ties environments to specific projects.

Packages are either specified in|meta.yaml|files (which include
semantically significant and Turing-complete comments!) and are built
by|conda-build|, or are specified in|recipe.yaml|files (which are much
saner) and built by|rattler-build|(implemented in Rust).

TODO

* Figure out application bundling (fix AppImage, see if we can do
Flatpak, Snap, portable|.exe|,|.msi|, and|.msix|)
* Ensure runtime dependencies are properly specified
* Fix cross-compilation on GNU/Linux
* Support Windows (should we do MinGW or MSVC?)
* Support macOS (should we do|ng-gnu-gnu|,|apple-gnu-gnu|,
or|apple-apple-apple|?)

Quick start/tutorial

This will guide you through setting up a basic GNUstep environment and
verifying that it works by installing SystemPreferences.app in it and
running it. Right now it only works on x86_64 glibc Linux.

First you should download Miniforge:

|wget
"https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname
-m).sh" bash Miniforge3-$(uname)-$(uname -m).sh rm
Miniforge3-$(uname)-$(uname -m).sh |

Close and reopen your shell, and run:

|# Prevent Conda from polluting your environment when you're not working
on Conda-managed projects. conda config --set auto_activate_base false |

Create a file named|environment.yml|and in it, write: (you can omit the
comments if you want)

|name: gnustep channels: # package repositories -
https://repo.prefix.dev/gnustep-forge # My GNUstep packages -
conda-forge # Community distribution of conda packages - nodefaults #
Disable the `defaults` channel if for some reason it's enabled
dependencies: - gnustep-make - gnustep-libdispatch - gnustep-libobjc2 -
libgnustep-base - libgnustep-gui - libgnustep-back -
gnustep-systempreferences - imagemagick # This is here because I need to
fix the dependency list |

Now, you can use|conda|or|mamba|to install the dependencies:

|mamba env create -f environment.yml mamba activate gnustep |

If you modify|environment.yml|after creating the environment, you need
to run

|mamba env update -f environment.yml |

Now, you can test out SystemPreferences:

|SystemPreferences |

Remember that the environment must be activated using|mamba activate
gnustep|every time you open a new shell.

You can deactivate the environment using|mamba deactivate|. Then GNUstep
should no longer be present. (I set it up to call|GNUstep-reset.sh|;
this might not work exactly as expected. It's better to simply open a
new shell.)

All the packages are built with debug symbols and headers included, so
you can try them out.

The GNUstep installation is located at|$CONDA_PREFIX/GNUstep|. On my
system, that's|~/miniforge3/envs/gnustep/GNUstep|.

Packaging and distributing your own software

Clone my repositoryhttps://github.com/ethanc8/gnustep-forge-feedstocks/,
and copy the|gnustep-systempreferences|directory into a new directory
with the name of your software.

These are the requirements I usually specify in|recipe.yaml|:

|requirements: build: - ${{ compiler('c') }} - ${{ compiler('cxx') }} -
${{ stdlib('c') }} - ninja # only for CMake projects host: - ${{
stdlib('c') }} - gnustep-make - gnustep-libobjc2 - gnustep-libdispatch -
libgnustep-base - libgnustep-gui - libgnustep-back |

If you have more dependencies, specify them in the|host|section in
addition to the ones above.

Most of the changes you'll need to make are in|recipe.yaml|: the name,
the GitHub URLs (or the homepage and source tarball download URL if you
don't use GitHub), the version number, and the Git tag (or commit hash,
etc). Also remember to change the metadata at the bottom
of|recipe.yaml|, and make sure that the|about.license_file|is properly
specified (this is usually|COPYING|or|LICENSE|).

If you need to add more build steps, such as running|./configure|, just
modify|build-systempreferences.sh|(you can rename this if you want, but
you'll also have to change the name of the build script in|recipe.yaml|).

You can now build the package using

|rattler-build build -c conda-forge -c https://prefix.dev/gnustep-forge |

The built package will be in|output/linux-64|.

If you want to publish it tohttps://prefix.dev/gnustep-forge, submit
your package as a pull request to my repo and I will look over and merge
it. If you want to make your own channel and host it at prefix.dev:

1. Sign in tohttps://prefix.dev/.
2. Create a channel athttps://prefix.dev/channels/create.
3. Modify|.github/workflows/build.yml|:

|- name: Run code in changed subdirectories shell: bash env:
TARGET_PLATFORM: ${{ matrix.target }} run: | rattler-build build
--recipe-dir . \ --skip-existing=all --target-platform=$TARGET_PLATFORM
\ -c conda-forge -c https://prefix.dev/gnustep-forge -c
https://prefix.dev/<<<NAME OF YOUR CHANNEL>>> # --experimental $(for
file in **/variants.yaml; do echo "-m$file"; done) # no longer necessary
because it should be discovered by default - name: Upload all packages
shell: bash # do not upload on PR if: github.event_name == 'push' env:
PREFIX_API_KEY: ${{ secrets.PREFIX_API_KEY }} run: | # ignore errors
because we want to ignore duplicate packages for file in
output/**/*.conda; do rattler-build upload prefix -c gnustep-forge -c
<<<NAME OF YOUR CHANNEL>>> "$file" || true done |

4. Commit your changes and push them to a GitHub repository.
5. Grab an API key fromhttps://prefix.dev/settings/api_keysand set it
as a repository secret named|PREFIX_API_KEY|in the GitHub repository
(for me, the appropriate settings page is
athttps://github.com/ethanc8/gnustep-forge-feedstocks/settings/secrets/actions)
6. Enable GitHub Actions on your GitHub repository.

You can also self-host the channel or host it at anaconda.org, but I
don't know how to do that. You should be able to find instructions
online on how to do it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnu.org/archive/html/discuss-gnustep/attachments/20241223/02ef625c/attachment.htm>

------------------------------

Subject: Digest Footer

_______________________________________________
Discuss-gnustep mailing list
Discuss-gnustep@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnustep

------------------------------

End of Discuss-gnustep Digest, Vol 265, Issue 24
************************************************

From:	Tom Sheffler
Subject:	Re: Conda packaging (Ethan C)
Date:	Thu, 26 Dec 2024 15:26:11 -0800