<html>
<head>
<base href="https://bugzilla.scilab.org/" />
</head>
<body><table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Reporter</th>
<td>Ahab1983@freenet.de</td>
</tr>
<tr>
<th>Bug ID</th>
<td><a class="bz_bug_link
bz_status_NEW "
title="NEW - csvWrite is done in 20sec, csvRead needs about 50min"
href="https://bugzilla.scilab.org/show_bug.cgi?id=15491">15491</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>csvWrite is done in 20sec, csvRead needs about 50min
</td>
</tr>
<tr>
<th>Product</th>
<td>Scilab software
</td>
</tr>
<tr>
<th>Version</th>
<td>6.0.0
</td>
</tr>
<tr>
<th>OS</th>
<td>Windows 10
</td>
</tr>
<tr>
<th>Status</th>
<td>NEW
</td>
</tr>
<tr>
<th>Severity</th>
<td>Medium
</td>
</tr>
<tr>
<th>Component</th>
<td>Data import/export
</td>
</tr>
<tr>
<th>Assignee</th>
<td>bugzilla.admin@scilab.org
</td>
</tr>
</table>
<p>
<div>
<pre>BUG DESCRIPTION:
----------------
I got a csv-dataset with 2e7 entries from a MSO4104(~700MByte).
ERROR LOG:
----------
Runs without error msg
HOW TO REPRODUCE THE BUG:
-------------------------
I created a Matrix with a timestamp:
function [m]=create_data(count,timeres,row,timestart)
if count>0 & row>0 then
time=(1:count)';
time=time*timeres+timestart;
m=ones(count,row);
m(:,1)=time;
end
endfunction
k=create_data(2e7,2e-10,5,-2e-3); //needs ~2 sec
csvWrite(k,"test2.csv",",","."); //needs ~8 sec
u=csvRead("test2.csv",",",[],"double"); //needs ~50 min
OTHER INFORMATION:
------------------
I often used datasets of 1e4 entries from the same source/kind and there was no problem to get it in some secs via csvRead. For example Notepad++(64
Bit) can open the whole dataset in ~10s. To make progress I had to fragment into subsets of 1e6 entries to be loadable in excel.
(Trying to have a plot from the whole set or even one of these subsets caused a shutdown of Scilab without any error msg. Giving Java a heap of
2560MB, restart...reloading solved that.)
hint? On the task-manager the memory consumption seemed not be continuous growing.
memory management issue?</pre>
</div>
</p>
<hr>
<span>You are receiving this mail because:</span>
<ul>
<li>You are watching all bug changes.</li>
</ul>
</body>
</html>