Re: [Bug-wget] GSoC Project | Design and Implementation of a Framework for Plugins

[Didik Setiawan]

On Mon, Mar 20, 2017 at 04:15:44PM +0100, Tim Ruehsen wrote:
> 
> Welcome, Didik !

Thanks, Tim.

> One way to implement plugins is via libdl (dlopen(), ...), and that is what I 
> have in mind. That is not perfectly portable, but our first goal will be 
> systems that support dlopen().

So, should I continue to use dlopen() or there is another better method? In 
case 
we need more portability.

> Since we can't (or want) to make everything pluggable, we should first 
> address 
> the already requested functionality. There should be (some ?) requests in the 
> wget1.x issue tracker that ask for plugin / external program for some good 
> reasons. You should review these, pick the parts that seems reasonable to you 
> and implement these first.
>
> E.g. I remember someone asking for some customisable 'filter' for URLs to 
> follow. In terms of a loadable module, that would be a function that takes a 
> URL and returns 1|0 (yes|no). Or a bit more general: takes a URL plus extra 
> information, e.g. referer, host, mimetype, ... and maybe returns a list of 
> URLs to fetch. Whatever makes sense.

Do you talking about this [1]? Then I refer to one of the three idea that says
call an external program which returns 0 or 1 to signal acceptance or 
rejectance 
of each proposed URI. Wget shall wait for the program to return. Any other exit
status shall cause wget to terminate.
I will try to implement this. 

> One goal would be to make up data structures and an API that can be extended 
> without breaking compatibility between  wget2 and the plugin in the future. 
> E.g. a newer wget2 should still be able to work with an older plugin and vice 
> versa. 

I will note this.

[1] https://savannah.gnu.org/bugs/?45803