Get Random and Remove from Array

Beginner at Powershell scripting, so this is probably painfully easy to most:

Looking to make an array of Tasks and then remove them from the array as they are assigned.

So far this works in getting the random task:

$Tasks = "a", "b", "c", "d"
$RandomTask = Get-Random $Tasks

But when I try to remove it, I get errors:

$Tasks = "a", "b", "c", "d"
$RandomTask = Get-Random $Tasks

#Something will assign the task here

$Tasks.Remove("$RandomTask")

How do I remove the randomly chosen string from the array? I keep getting this:

Exception calling "Remove" with "1" argument(s): "Collection was of a fixed size."

Thanks in advance.