Random shuffle of array, but keep diagonal fixed

I have defined an array that looks like this:

A = np.array([[1,2,3], [4,1,5], [6,7,1]])

I would like to randomly shuffle it, but keep the 1's in the diagonal fixed. When I do

B = [0,1,2]
np.random.shuffle(B)

all elements are shuffled, including the 1's in the diagonal.

Does anyone know a solution to do this?

Thank you!