Fast Row Operations With Python Sparse Matrices

I have a large sparse matrix in python, and would like to perform many elementary row operations on it. That is, adding a multiple of one row to another row. What is the most effic

Solution 1:

First, a MCVexample would help the question a lot. I can only speculate about your row operations.

A fundamental question - do the rows differ in their sparsity structure? Take the lil case. If 2 rows have the same rows lists, then your math can work with the data lists directly. If rows differ than math becomes much more complicated, since you have to change both lists.

Both lil and csr support indexing by row

In [7]: M=sparse.rand(10,10,.3)
In [8]: Mr=M.tocsr()
In [9]: Ml=M.tolil()

Yes, csr gives a warning if you change a row by adding another:

In [17]: Ml[2,:] += Ml[1,:]
In [18]: Mr[2,:] += Mr[1,:]
...
  SparseEfficiencyWarning)

But the lil math actually uses a csr intermediary. lil rows are represented as lists, not arrays.

In [14]: Ml[1,:]+Ml[2,:]
Out[14]: 
<1x10 sparse matrix of type'<class 'numpy.float64'>'with5 stored elements in Compressed Sparse Row format>

Indexed matrix operations are slow, especially compared to the dense array equivalents. But they take care of a lot of little details for you.

I've explored row operations in other SO answers. When I have a better idea of what you are trying to do, I search those.

Overall, there isn't a magic bullet, especially if you are changing sparsity. scipy sparse isn't the best tool for fast row calculations.

scipy: Adding a sparse vector to a specific row of a sparse matrix - this one is close enough that I'm tempted to flag this question as a duplicate.

Extremely slow sum row operation in Sparse LIL matrix in Python

(more in a SO search on 'user:901925 [scipy] rows')