Re: [DotGNU]Concept: Box files

[Anselm Hook]

On a vaguely related note:

If I recall correctly one of the weaknesses of the jar system was that
without the right permissions the jars would not be persistent beyond the
end of a web session.  That meant that a server could not deliver say a
'game engine' once and then have to only deliver different 'game apps'
relying on the engine already present.  In essence Java needed a zero cost
persistent sand box; a region still blocked from being able to tamper with
real stuff, but where the sand-castles could stay up overnight.

Of course developers could build in the right security policy and keys;
but one goal of java was to be able to easily deploy applets over the web
at zero cost.  Getting signed by verisign was not zero cost.  So without
persistence of some pieces doing anything real became a hassle - applets
had no inter-session client-side memory and always had to be re-delivered.
Sun always had a kind of retarded thin client heavy server policy designed
to sell their Sun boxes and so never really put much work into client
side IMHO.

It looks like your scheme doesn't have this problem.  Sounds good to me.

 - a

On Tue, 27 Aug 2002, Peter Minten wrote:

> Hi,
>
> I've did some more thinking about the .box concept, this is the result:
>
> --- CONCEPT ---
>
> Name: DotGNU box files standard
>
> Description:
>
> Java has jar files that make it possible to send a program in a nice small
> package over the internet and to run it straight from that package without
> unzipping or anything. This maybe useful to IL bytecode programs too, 
> especially
> since IL programs tend to have extras like resource files that need to be send
> with the executable.
>
> What I'm proposing are .box files which can:
> * contain executables
> * contain support files (resource files)
> * contain configuration files
> * contain a whole project directory with source code
>
> The box files would enable a user to download a whole program + source in a
> simple efficient package. The box files would have a description in them 
> telling
> what they contain and some details of using it (somewhat like the .la libtool
> files). Programs wanting to link with the contents of a box file can do so
> without complicated manoeuvres (executables are unzipped into temp dir and 
> run,
> libraries are only unzipped into a tempdir) simply by using a central package
> management program that keeps track of all the boxes on a system and notes 
> their
> contents in a registry. This makes difficult linking commands superfluous.
>
> Of course there is the matter of support files that need to be present
> somewhere, these files are simply unzipped into a temp dir where they stay 
> until
> the last process of the program ends. The configuration files are simply put
> into the DotGNU Registry where they stay even when the program is terminated.
> This requires that a programmer knows what files are non-changing and what are
> changing, support files should not change, configuration files could change.
>
> The box files could be automatically signed with a GPG key for security, a 
> sign
> can also be added to an existing box. This can be used for distribution over
> central servers: programmer A signs the box with his private key, server B has
> A's public key and uses it to check the box A send in, Server B replaces key A
> with key B which public part is known to user C, D, E, etc. Box files can also
> contain an md5sum, in combination with signing this allows for secure 
> downloads
> from trusted parties.
>
> The box files on a system are managed by an application called 'Warehouse'.
> Warehouse keeps track of the packages installed on a system, every time a
> package is removed or added Warehouse updates it's inventory, updating may 
> also
> be put in as a cron job. Warehouse also checks the requirements of new 
> programs
> and alarms the user if a new version of an existing or a completely new 
> package
> needs to be downloaded, if the user has specified downloading locations (I'll
> tell more about these later) Warehouse can automatically download the
> applications. It also checks automatically if the file is signed with a key in
> it's trusted key ring and if the md5sum is correct.
>
> Apps seldom have to deal with Warehouse if the runtime engine interfaces with 
> it
> (which is so easy that it shouldn't pose any problems). An app simply 
> specifies
> the libraries it want's to link to and Warehouse tells the runtime engine 
> where
> the files are.
>
> The user will also seldom deal with Warehouse since there will be a graphical
> interface to Warehouse called 'Warehouse Manager'. This program simply gives 
> the
> user easy access to all functions of Warehouse.
>
> Warehouse will know (using information in the boxes and/or information on the
> net) what version of a program/library is interface compatible with what other
> versions. If a program requires a higher version of a file that is interface
> compatible with the version on the machine Warehouse will suggest to the user 
> to
> upgrade the package, if it isn't interface compatible Warehouse will make sure
> that both versions of the package live safely besides each other on the 
> system.
> Warehouse can also do a cleanup scan in which it can suggest the removal of
> library boxes which no other box needs, bye DLL hell (note that the DLL hell
> can come to GNU/Linux with IL libraries carrying the .dll extension).
>
> Warehouse can find needed boxes on the net, using a list of download locations
> or by simply searching the web. It can also offer downloads or offer linking 
> to
> boxes on it's system (making it the DotGNU app server). This functionality is
> based on p2p. Every warehouse knows a number of other warehouses, when it 
> needs
> something it asks the warehouses it knows, if they don't know it either they
> will ask the warehouses they know, the search stops however at the first match
> (so that the network won't get overloaded). Also a warehouse can offer boxes 
> for
> linking or for download.
>
> A warehouse can also keep an inventory list of the warehouses it knows, the
> number of inventory lists it has can be changed by the user, also a warehouse
> can decide if it offers an inventory list. Some warehouses may even do nothing
> else than keep inventory lists (becoming a webservice directory), a number of
> these warehouses placed on big servers and with good interaction and
> co-ordination can speed up the search a lot (a normal search could only ask a
> webservice directory, for most of the software this would give good results).
>
> The webservice directories could have some keyword tracking mechanisms to 
> purge
> the inventories from cracked software, unauthorized copied software will only
> help the proprietary software firms in the long run (by increasing their
> marketshare). Besides that, it avoids the BSA hunting the providers of the
> webservice directories. This would by the way not prevent the distribution of
> unauthorized copied software over the warehouse network since the p2p concept
> makes that impossible, but it would keep that kind of software out of the 
> search
> results of most users.
>
> --- END CONCEPT ---
>
> And here are a few details on how I want to do it:
>
> --- DETAILS ---
>
> The box files are build up like this:
> 1. Make is called in a project dir to create the binaries
> 2. A manifest file is put up out of data in a box.manifest file and other data
> gathered by the 'box' utility.
> 3. Binaries, source stuff (the whole project dir without files included
> elsewhere), support files and configuration files are put into 4 separate temp
> dirs.
> 4. The temp dirs are tarballed and gzipped.
> 5. The manifest file is gzipped.
> 6. The tempdir tarballs and the manifest file are tarred together.
>
> When a user wants to run a program support files are unpacked, configuration
> files are loaded into the local registry (DotGNU version) and the binary is
> loaded into memory. If the user wants to work with the source code the box
> utility uses the manifesto to create a project dir in which is places all the
> files from the source, support and config packages in their original 
> locations.
>
> The manifesto is a simple text file.
>
> The 'box', 'warehouse' and 'warehouse-manager' programs are written in C
> (because of speed matters and using C makes it easier to expand the use of
> Warehouse beyond IL executables if that becomes feasible).
>
> The programs will use the system or execve functions for using 'gzip' 'tar' 
> and
> 'gpg', this is a lot easier than using library interfaces.
>
> --- END DETAILS ---
>
> In short: the warehouse system:
> * Will make it easier for users to work with IL programs.
> * Will encourage distributing source (by making binary + source distribution 
> the
> standard in the box app).
> * Will enable secure downloads and usage of IL programs.
> * Will be a DotGNU replacement for UDDI.
> * Will be the DotGNU app server.
>
> Greetings,
>
> Peter
>
> _______________________________________________
> Developers mailing list
> address@hidden
> http://www.dotgnu.org/mailman/listinfo/developers
>