Re: url-cache - (require 'url)

[Stefan Monnier]

>>> Maybe there's a better approach to the problem I'm trying to solve.  I want
>>> to add support for ETags to nnrss in Gnus.  To do that, nnrss needs access
>>> to HTTP headers.  nnrss currently uses mm-url-insert which calls
>>> url-insert-file-contents.  Those seem like the right functions to use but
>>> they don't provide access to the HTTP headers.
>> 
>> Why do they seem better to you than url-retrieve (or even url-http) given
>> the fact that they do not give you the info you need?

> Because url-insert-file-contents does some coding-system things that I
> assume are important for inserting URLs into buffers.  I don't know
> anything about coding-systems.  I could write a new function that uses
> url-retrieve or url-http but I'd probably have to duplicate the
> coding-system parts which seems wasteful.

> It also seems that other applications that want to insert the contents of a
> URL could benefit from having access to HTTP headers.

Good point.

I guess a good answer to that is simply to make url-insert-file-contents
"trivial" by moving most of its contents to a separate function.
Say url-insert, as in the patch below.  Does that provide the functionality
you're looking for?


        Stefan


--- url-handlers.el     03 jan 2006 11:16:30 -0500      1.16
+++ url-handlers.el     16 jan 2006 13:42:03 -0500      
@@ -197,33 +197,47 @@
     (url-copy-file url filename)
     filename))
 
+(defun url-insert (buffer &optional beg end)
+  "Insert the body of a URL object.
+BUFFER should be a complete URL buffer as returned by `url-retrieve'.
+If the headers specify a coding-system, it is applied to the body before it is 
inserted.
+Returns a list of the form (SIZE CHARSET), where SIZE is the size in bytes
+of the inserted text and CHARSET is the charset that was specified in the 
header,
+or nil if none was found.
+BEG and END can be used to only insert a subpart of the body.
+They count bytes from the beginning of the body."
+  (let* ((handle (with-current-buffer buffer (mm-dissect-buffer t)))
+         (data (with-current-buffer (mm-handle-buffer handle)
+                 (if beg
+                     (buffer-substring (+ (point-min) beg)
+                                       (if end (+ (point-min) end) 
(point-max)))
+                  (buffer-string))))
+         (charset (mail-content-type-get (mm-handle-type handle)
+                                          'charset)))
+    (mm-destroy-parts handle)
+    (if charset
+        (insert (mm-decode-string data (mm-charset-to-coding-system charset)))
+      (insert data))
+    (list (length data) charset)))
+
 ;;;###autoload
 (defun url-insert-file-contents (url &optional visit beg end replace)
-  (let ((buffer (url-retrieve-synchronously url))
-       (handle nil)
-       (charset nil)
-       (data nil))
+  (let ((buffer (url-retrieve-synchronously url)))
     (if (not buffer)
        (error "Opening input file: No such file or directory, %s" url))
     (if visit (setq buffer-file-name url))
-    (with-current-buffer buffer
-      (setq handle (mm-dissect-buffer t))
-      (set-buffer (mm-handle-buffer handle))
-      (setq data (if beg (buffer-substring beg end)
-                  (buffer-string))))
-    (kill-buffer buffer)
-    (mm-destroy-parts handle)
-    (if replace (delete-region (point-min) (point-max)))
     (save-excursion
-      (setq charset (mail-content-type-get (mm-handle-type handle)
-                                            'charset))
-      (let ((start (point)))
-       (if charset
-           (insert (mm-decode-string data (mm-charset-to-coding-system 
charset)))
-         (progn
-           (insert data)
-           (decode-coding-inserted-region start (point) url visit beg end 
replace)))))
-    (list url (length data))))
+      (let* ((start (point))
+             (size-and-charset (url-insert buffer beg end)))
+        (kill-buffer buffer)
+        (when replace
+          (delete-region (point-min) start)
+          (delete-region (point) (point-max)))
+        (unless (cadr size-and-charset)
+          ;; If the headers don't specify any particular charset, use the
+          ;; usual heuristic/rules that we apply to files.
+          (decode-coding-inserted-region start (point) url visit beg end 
replace))
+        (list url (car size-and-charset))))))
 
 (defun url-file-name-completion (url directory)
   (error "Unimplemented"))