<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#ffffff">
Le 16/03/2011 09:48, Michaël Baudin a écrit :
<blockquote cite="mid:4D807947.9090401@scilab.org" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
Hello,<br>
<br>
Thank you for providing your interesting comments. <br>
<br>
Do not forget that this is an English mailing list.<br>
<br>
The idea that you suggest on csv_stringtodouble is interesting. We did
not think about extending the current double function. The idea of
Allan on this topic was to mimic the str2double function in Matlab:<br>
<br>
<a moz-do-not-send="true"
href="http://www.mathworks.com/help/techdoc/ref/str2double.html">http://www.mathworks.com/help/techdoc/ref/str2double.html</a><br>
<br>
The main issue here is that the str2double function is extremely
flexible: the range of complex doubles strings that Matlab can process
is wide. It is rather difficult to entirely reproduce this. For
example, the imaginary number i can be before or after the imaginary
part, e.g. <span class="Apple-style-span"
style="border-collapse: separate; color: rgb(0, 0, 0); font-family: 'Times New Roman'; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; font-size: medium;"><span
class="Apple-style-span"
style="font-family: Verdana,Arial,Helvetica,sans-serif; font-size: 11px;"></span></span>'123
+ 45i' or '45i+123' or '45j+123' or 'j+123' or '-j-123' or '-45*j-123'
or '-45e+12*j-123e-3', etc... Octave has troubles to reproduce these
features. Part of the problem is that the Matlab function str2double
makes use of the sscanf function, which can manage flexible formats.<br>
</blockquote>
probably Matlab does something like cvstr (parse an expression and
evaluate it) . It is more flexible but less efficient. If you are
looking for efficiency you should restrict to real or integer numbers.
The Matlab str2double may be emulated using cvstr. the "45j" or "45i"
feature is the most difficult to handle, but it can be solved by a
preprocessing that changes the regular expressions [0-9]i and [0-9]j to
add a *.<br>
<blockquote cite="mid:4D807947.9090401@scilab.org" type="cite"> <br>
The compatibility with Excel and the backward compatibility were our
primary concern: be sure that we are well aware of these issues and
tried to solve it as much as possible. I think that the currrent
proposal is quite consistent and here is why.<br>
<br>
Notice the bug #8654 : In write_csv, the default separator and decimal
mark are non-standard. There are three problems:<br>
* The separator is the tab, while the Comma Separated Value data file
format is clear about the fact that the separator is the comma.<br>
* The decimal mark is the comma ",", while the standard is the dot ".".<br>
* Such a file will not be read correctly by the default settings of
read_csv.<br>
<br>
I emphasize that the decimal point is the comma "," only in France: in
most other countries (including U.S.), the decimal point is the dot
".". In Open Office for example, we can customize this with Tools -
Options - Language Settings - Languages. In the proposed csv_read
function, the decimal point is also customizable. A valid suggestion,
though, would be to customize the decimal point depending on the
locale. But this is beyond my technical knowledge. May be Sylvestre
could improve this.<br>
<br>
</blockquote>
I am not sure that relying on the locale should be a good solution. One
may want to read data generated under an other locale<br>
It seem better to make that customisable throught parameter as you
proposed.<br>
<blockquote cite="mid:4D807947.9090401@scilab.org" type="cite"> I
emphasize that the columns of a Comma Separated Value data files are
separated by commas ",", as suggested by the name of this "format". If
the separator was the Tab, this would be called a "TAB Separated Value".<br>
<br>
There is a link between the separator and the decimal point in the
current function. The reason why the TAB was chosen in the current
read_csv function, is because the comma "," was chosen as the decimal
point. If the comma was also chosen to be the separator, we could not
read the file anymore, because of the confusion with the decimal point.
This is the reason why the current default for the separator is the
TAB. <br>
<br>
The backward compatibility is a real concern. <br>
* The backward compatibility can be broken, since the current state is
inconsistent. What is the point in maintaining the current state, where
a file written by write_csv cannot be read by read_csv ?<br>
* The "forward" compatibility is improved: the comma is the default
separator in Matlab and Octave.<br>
* The backward compatibility will probably be broken, by creating new
functions called "csv_read/csv_write", while the current functions are
"read_csv/write_csv". The current functions "read_csv/write_csv" will
be tagged as obsolete, and then removed in a later release. <br>
<br>
</blockquote>
Ok! I did not realize that you proposed a name change. Good solution!<br>
<br>
Serge Steer<br>
<blockquote cite="mid:4D807947.9090401@scilab.org" type="cite"> Best
regards,<br>
<br>
Michaël Baudin<br>
<br>
<br>
Le 15/03/2011 16:53, Serge Steer a écrit :
<blockquote cite="mid:4D7F8B97.8010604@scilab.org" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
Le 15/03/2011 15:31, Michaël Baudin a écrit :
<blockquote cite="mid:4D7F782C.8000105@scilab.org" type="cite">
<meta http-equiv="content-type"
content="text/html; charset=ISO-8859-1">
Hello,<br>
<br>
This SEP describes the<span style="font-weight: normal;"> definition
and the improvement of CSV import and export functions in Scilab.</span>
<br>
<br>
This is a work with Allan Cornet, who is the leader of this project.<br>
<br>
Best regards,<br>
<br>
Michaël Baudin<br>
<br>
<br>
<pre class="moz-signature" cols="72">--
Michaël Baudin
Ingénieur de développement
<a moz-do-not-send="true" class="moz-txt-link-abbreviated"
href="mailto:michael.baudin@scilab.org">michael.baudin@scilab.org</a>
-------------------------
Consortium Scilab - Digiteo
Domaine de Voluceau - Rocquencourt
B.P. 105 - 78153 Le Chesnay Cedex
Tel. : 01 39 63 56 87 - Fax : 01 39 63 55 94
</pre>
</blockquote>
J'ai un petit souci avec ce SEP. le format csv était a priori prévus
pour les échanges avec excel d'où l'utilisation de la virgule au lieu
du point décimal et par conséquent de tab comme séparateur.... Il me
semble que cette spec permet de gerer ces deux choses, mais ce n'est
pas dit explicitement, de même que le défaut n'est pas précisé. Pour la
compatilibilité ascendante il serait bon que le défaut corresponde a la
situation actuelle...<br>
<br>
Par ailleurs je ne comprend pas pourquoi la fonction
<meta http-equiv="CONTENT-TYPE"
content="text/html; charset=ISO-8859-1">
<title></title>
<meta name="GENERATOR" content="OpenOffice.org 3.1 (Linux)">
<style type="text/css">
<!--
@page { margin: 2cm }
P { margin-bottom: 0.21cm }
--</style>csv_stringtodouble
est prefixée par csv, si je comprend bien elle n'a rien a voir avec le
format csv. Pourquoi ne pas l'appeler stringtodouble ou tout simplement
completer la semantique de double, int8,int16,int32,....) (sachant que
string fait l'inverse)<br>
<br>
Serge<br>
Serge<br>
<br>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Michaël Baudin
Ingénieur de développement
michael.baudin@scilab.org
-------------------------
Consortium Scilab - Digiteo
Domaine de Voluceau - Rocquencourt
B.P. 105 - 78153 Le Chesnay Cedex
Tel. : 01 39 63 56 87 - Fax : 01 39 63 55 94
</pre>
</blockquote>
<br>
</body>
</html>