Generate Random Binary Matrix

I wish to generate 10,000 random binary matrices which have the same number of 1s per row and per column as a given binary matrix.

The matrix is ~500 x ~10,000. There are about 2,000,000 1s. There are no zero rows or columns.

My current method converts the binary matrix into a bipartite adjacency matrix, and performs 1,000,000 random edge switches to guarantee randomness. This takes 13,000 seconds for 1 matrix. I'm coding in python, using a modified version of networkx's double_edge_swap function.

Is there a more efficient way to generate such matrices?