[Scilab-Dev] SEP 88: New functions to get remote files

[Sylvestre Ledru]

Hello Samuel,

Many thanks for your feedback. Since it is not too late to perform
changes on the functions, I will see what I can do :)
I have to see the potential impact of merging the two but, a priori, I
don't see much issues here.

Merci,
Sylvestre
On 23/03/2013 13:58, Samuel Gougeon wrote:
> Hello Sylvestre,
> 
> Le 24/01/2013 18:14, Sylvestre Ledru a écrit :
>> Hello,
>>
>> The attachment describes the SEP #88, targeting Scilab 5.5.0.
>>
>> The goal of this SEP is to propose the introduction of three new functions:
>> * getURLcontent - Return the content of an URL (HTTP, HTTPS, FTP...)
>> Profiles:
>> filename = getURL(URL [, targetDir [, username [, password]]]]);
>> filename = getURL(URL [, targetFile [, username [, password]]]]);
>>
>> * getURL - Download an URL (HTTP, HTTPS, FTP...)
>> Profile:
>> output = getURLcontent(URL [, username, [, password]]);
>>
>> * splitURL - Split a URL (HTTP, HTTPS, FTP...)
>> Profile:
>> [proto, server, path, query, username, port, fragment] = splitURL(URL);
>>
>> Don't hesitate to comment it,
>> Sylvestre
> Sorry for the delay. The good thing is that now there is more
> information in the help
> of the beta-implementation available in the 5.5 branch; and it is
> possible to test them.
> 
> I developped some utilities under Scilab with curl ~3 years ago,  and i
> had a look
> on how to use these new functions instead of my owns. Here are some comments
> and suggestions about their present implementation.
> 
> *getURL() and getURLcontent()*:
> At the first glance, it is hard to understand the difference between
> getURL() and
> getURLcontent(). Actually, it is very slight. Since it is only a matter
> of output and
> that 90% of their respective jobs are the same, in my opinion, this does
> not
> deserve 2 distinct functions: It would be nicer to add an option
> specifying the
> output, or even more simply to have a second optionnal output argument:
> [filename [, content]] = getURL(...)
> It is always possible to specify a trash-file as filename.
> or
> output = getURL(...,"returnContent")
> At least, for the moment, a single help page merging the description of
> both
> functions would be nice.
> 
> _Present embarassing limitations w.r.t. curl features_:
> The most frustrating one is that these functions allow only queries in
> GET method.
> The POST method is not available,  whereas curl allows it with no
> problem and
> very great utility.
> => So: It would be very useful to add a curl_options string optionnal
> parameter,
> that would be passed to curl as is, allowing to post data with "-d
> param=value ...etc"
> and any other curl usage.
> By the way, specifying a file in which to dump the targetted content is
> possible
> in this way. Proxy parameters can also be specified.
> 
> Presently, it is not natural to use atomsSetConfig() to specify a proxy
> to use for
> getURL...().
> 
> In few words, it could be a good basic target to have a simple interface
> with curl,
> such as a curlScilab() function.
> The only problem i had when using curl through unix_#() functions was some
> encoding issues in the output content. getURL...() greatly fix this point.
> Otherwise, curl is so great that is is a big improvement to include it
> the Scilab
> package for all plateforms (not only Windows).
> 
> Other advanced features could be useful, such as allowing to follow
>  <META HTTP-EQUIV="Refresh" CONTENT="n ; url=new_target" >
> within a limited nesting depth ; and other ones.
> More elaborated functions could be posted to ATOMS or FileExchange.
> 
> *splitURL()*:
> works fine :-) IMO, its help page should be located in the "paths -
> filenames"
> subsection, together with fileparts() and other paths-processing functions.
> 
> Best regards
> Samuel
> 
> 
> 
> _______________________________________________
> dev mailing list
> dev at lists.scilab.org
> http://lists.scilab.org/mailman/listinfo/dev
>