[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: LYNX-DEV Since Lynx won't, what will?
From: |
Jim Dennis |
Subject: |
Re: LYNX-DEV Since Lynx won't, what will? |
Date: |
Sun, 10 Aug 1997 02:50:50 -0700 |
>> My hope is that a number of webmasters and sysadmins will use
>> Lynx as their prefered "web automation" tool and will
>> consequently put furhter pressure on their designers to have
>
> Please don't even think of marketing anything like this to this market
> unless it fully supports robots.txt (even if there are non-default
> options to defeat it). Your proposed market is not going bother
> with enforcing such restrictions manually.
As Fote points out -- my market is for the sysadmins and webmasters
hitting their own site (from their own LAN or even localhost).
One nice thing about using -traversal is that you get
"orphan detection" -- you can find any of your files that
have not links to them (with a little filtering and a diff).
Another nice thing is you can prepare a ToC.html (table
of contents) which is a list of every link that's anywhere
on your site -- and another one that lists ever link that
exits your site.
In any event, I agree that it should have support for
robots.txt by default -- with a command line switch to
over-ride that (and maybe a global configuration option
to over-ride *that* over-ride and mail a nastygram to
the SA when a user attempts to do so).
--
Jim Dennis (800) 938-4078 address@hidden
Proprietor, Starshine Technical Services: http://www.starshine.org
PGP 1024/2ABF03B1 Jim Dennis <address@hidden>
Key fingerprint = 2524E3FEF0922A84 A27BDEDB38EBB95A
;
; To UNSUBSCRIBE: Send a mail message to address@hidden
; with "unsubscribe lynx-dev" (without the
; quotation marks) on a line by itself.
;
- Re: LYNX-DEV Since Lynx won't, what will?, Jim Dennis, 1997/08/07
- Re: LYNX-DEV Since Lynx won't, what will?, Scott McGee (Personal), 1997/08/07
- Re: LYNX-DEV Since Lynx won't, what will?, Jim Dennis, 1997/08/07
- Re: LYNX-DEV Since Lynx won't, what will?, David Woolley, 1997/08/09
- Re: LYNX-DEV Since Lynx won't, what will?, Jonathan Sergent, 1997/08/20
- Re: LYNX-DEV Since Lynx won't, what will?, Larry W. Virden, x2487, 1997/08/20
- Re: LYNX-DEV Since Lynx won't, what will?, Foteos Macrides, 1997/08/09
- Re: LYNX-DEV Since Lynx won't, what will?,
Jim Dennis <=