How to perform a JButton's action listener a certain number of times

I have some code where I have code with 5 JButtons and I have a seperate array that generates a random number between 1-6 (dice roll) and inside the action listener I have just the part that adds my button to my panel (panel.add(roll1)) and before the action listener I have an array which generates the random number and then I have a switch saying if the random number is 1 then set an image to dice1 if array is 2 set image to dice2. So I have all this working now since I'm making the game of yahtzee I need to know how to make this runnable at a max of 3 times per move. Right now the JButton is only clicked once and it outputs other JButtons with random dice images, but when I click the roll button button again the dice does not roll it stays the same. How would you do this?