generate a random number in powershell from a range but exclude 1

I want to generate a random number from a range (lets say 1 to 3) but exclude from that range specific number in variable X

So I got this code

get-random -inputobject (1..4) | where {$_ -notin $X}

this indeed generate a random number between 1 and 3 but when the generate number is equal to the number in X, it returns nothing.

How can I force it to run again when it happens so I always get a number, don't matter how many tries it takes?