How to sample rows without replacement within (multiple) subgroups in R

I have a randomisation/sampling problem I can't get my head around. Let's say I want to create a randomised order for presentation of stimuli in an experiment with two variables, colour and item, each with three levels.

library(dplyr)
set.seed(42)

participant_id <- 1:12
colour <- c("blue", "green", "red")
item <- c("apple", "banana", "pear")

I want each participant to see a random pairing of colour and item, in a random order in the list, but never seeing any individual value more than once.

I can get close with the code below - each participant sees each colour once, in random order - but can't think how to ensure that the item paired with each colour also appears without repetition.

# dataframe of all possible combinations
all_permutations <- crossing(participant_id, colour, item) 

my_list <- all_permutations %>%
  group_by(participant_id, colour) %>%
  # randomly sample one row of item, per colour and participant  
  sample_n(1) %>% 
  group_by(participant_id) %>% 
  # randomly re-order within grouping
  sample_frac()

head(my_list, 15) 

# A tibble: 15 x 3
# Groups:   participant_id [5]
   participant_id colour item  
            <int> <chr>  <chr> 
 1              1 blue   pear  
 2              1 red    apple 
 3              1 green  pear  
 4              2 green  banana
 5              2 blue   pear  
 6              2 red    banana
 7              3 blue   pear  
 8              3 green  apple 
 9              3 red    banana
10              4 red    pear  
11              4 green  banana
12              4 blue   pear  
13              5 red    banana
14              5 green  apple 
15              5 blue   pear  

Any suggestions gratefully received! Thanks in advance.