jQuery - How to randomize within an array of arrays?

I'm building a game with answers in a 4x4 grid wall, the answers will initially be in groups of four which need to be randomized at the start of each game. The player then has to re-arrange the answers into groups (in case that doesn't make sense, http://ift.tt/1TtmwNW, shows examples of what I'm trying to achieve).

I have set up a grid with arrays of answers as follows:

var grid = [
  ['oak','cedar','fir','pine'],
  ['red','blue','green','yellow'],
  ['villa','spurs','city','united'],
  ['table','chair','door','stool']
];

Using the Fisher-Yates shuffle as follows I can get the grid to randomize the arrays:

function shuffle(grid){

var i = grid.length;
var j;
var temp;

while(--i>0){
    j = Math.floor(Math.random()*(i+1));
    temp = grid[j];
    grid[j] = grid[i];
    grid[i] = temp;
}
return grid;
}

The problem is that it only shuffles the whole lines, rather than the items within each array so that, for example, 'chair' will never be next to 'city'. Does anyone have any suggestions as to how this might be done?