Ordered random numbers in Matlab

I am trying to generate random numbers between 1 and 5 using Matlab's randperm and calling randperm = 5.

Each time this gives me a different array let's say for example:

x = randperm(5)
x = [3 2 4 1 5]

I need the vector to be arranged such that 4 and 5 are always next to each other and 2 is always between 1 and 3... so for e.g. [3 2 1 4 5] or [4 5 1 2 3].

So essentially I have two "blocks" of unequal length - 1 2 3 and 4 5. The order of the blocks is not so important, just that 4 & 5 end up together and 2 in between 1 and 3.

Does anyone know how I can do this?

Thanks