[Scilab-users] Pairwise distance of a huge amount of points

[Dang, Christophe]

Hello,

I need to compute the pairwise distance of a huge amount of points, namely n = 49545.

I re-read a previous discussion
http://mailinglists.scilab.org/Plot-overlays-on-images-tp2617675p4030599.html

So, I'm trying yo avoid loops,
but I need then at least to have a vectors, or sparse matrices, containing n*(n+1)/2 elements,
i.e. 1.227D+09, which of course exceeds the maximum stacksize.

Any hint?

FYI, three vectors of size n called X0, Y0 and Z0,
the following procedure works well for small vectors
(I tried with n = 1000)

// ********** 
n = size(X0, "r"); // amount of points

uns = sparse(triu(ones(n, n))); // sparse upper triangle matrix of ones

Xd = meshgrid(X0).*uns;
Yd = meshgrid(Y0).*uns;
Zd = meshgrid(Z0).*uns;
Xf = meshgrid(X0)'.*uns;
Yf = meshgrid(Y0)'.*uns;
Zf = meshgrid(Z0)'.*uns;
D = sqrt((Xf - Xd).^2 + (Yf - Yd).^2 + (Zf - Zd).^2);
// **********

in my case, it stops at step 2.

The limiting operation is ones(n, n), which requires a n*n matrix.
Using directly sparse() also requires a n^2 vector.

It might be possible to work with higher values of n by generating a sparse matrix without generating a n*n matrix (or a n^2 vector), as there are only n*(n+1)/2 non-zeroes values -- Is it possible?

(Anyway, in my case, it is still too big).

-- 
Christophe Dang Ngoc Chan
Mechanical calculation engineer

______________________________________________________________________

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error), please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.
______________________________________________________________________