Re: [Gzz] Raw pools?

[Benja Fallenstein]

Tuomas Lukka wrote:

> On Mon, Nov 11, 2002 at 08:45:23PM +0100, Benja Fallenstein wrote:
>  
>
> > Tuomas Lukka wrote:
> >
> > >         - persistent blocks: operations
> > > 		- get a block's bits EXACTLY or "sorry, can't get them" 
> > > 		-answer
> > >                 - store a block, get an ID
> > > 	- pointers 
> > >
> > >         - ...
> > >
> > >      
> > I think we should have several related PEGs for these.
> >    
>
> Please propose a set of PEGs.

Hmm...

- Storm core
- Concept and rationale: Blocks to move easily between pools
- The persistency commitment
- API
- Block format (header with content-type, opaque body), id format
- Backward compatibility: Old specs we need to continue to support
- Applications
- Pointers [could possibly be a single PEG]
- Concept: rich branching versioning without central repository
- Pointer block format
- API
- Challenges when used on the network
(digital signatures; what to do if not all pointer blocks are known)
- Diffs
- Xanalogical media
- Concept: Content stored in blocks, other blocks refer to it
(and there's a reverse indexing mechanism for referals)
- Formats
- API

Quite a suite. Would greatly help people trying to understand what we're 
up to, though.

> > I still don't agree that this makes a big difference *at this point in 
> > time*. So far, I have *never* entered a SHA-1 in a 
> > content-based-retrieval system. Besides, if the papers used in xupdf 
> > were available in one, that would be illegal, while they're also legally 
> > available from the respective web sites *we* got them from-- except if 
> > the copyright holders would make them available there, which isn't 
> > paricularly likely.
> >
> > I'm saying this to explain why I'm still not convinced we should change 
> > Storm at this time. Can you explain why you feel this is so important 
> > right now?
> >    
>
> Yes.
>
> I want to be able to *now* make a *permanent, eternal* link which uniquely
> identifies a file, which I can't unfortunately redistribute, 
>
> So that if someone enters all the files on their disk to a SHA-1 index,
> they'd get a hit.
>
> So that anyone who *does* have access to the file can verify that it *is* the 
> same
> file that I made the link to.
>
> The permanence is the point.

Would making "these blocks are the same"-style links and signing them 
digitally not work?

> Canonical blocks (with trivial header) feel like a very unclean solution;
> I expect that sha-1 of files will see much more use in a few years.

It *may* very well. However, as I said I'm for being as conservative as 
possible in changing the Storm spec, meaning: Only change it when the 
change is needed right now-- not because of an anticipated need or a 
cool new idea, even if it seems extremely probable that this new feature 
will be needed in the future. Only place the additional burden on the 
shoulders of future implementors if you *know* it is necessary.

With this, at the time this is in common use it's entirely possible that 
a new, more efficient way of doing this has been found, and we will 
*have* to support it-- which would mean *two* more formats to support by 
implementers, because the format introduced now would also have to be 
supported.

> > Would it, btw, be legal to provide a download script for the papers, 
> > like Debian provided for Microsoft's web fonts?
> >    
>
> Probably. But some of them are of course only available e.g. at universities
> with deals with ACM.

Hm, we should probably not use those in the public demo.

> > > > Let me put it like this: It has broad enough applicability that I'm 
> > > > thinking this *might* just be good enough to warrant the burden on 
> > > > future implementations.
> > > >  
> > > >
> > > >        
> > > Great, let's PEG it.
> > >
> > >      
> > Ok. Who's going to?
> >    
>
> For the chunk thing, you're probably the best.

Scheduled for alpha5.
- Benja