How do I randomise the order of buttons for a Quiz in VB.NET?

So I am currently working on a little project where I am making a quiz in VB. I'm still fairly intermediate so this has been a good project for me to work on so far. I am trying to make it so that the answers (which are written on buttons) are randomised each time. So let's say, the third button isn't always the correct answer.

Currently, my code works as follows. Below is a function to generate a random number from 1-4 in an array and checks there are no repeats. So the array may read arraytofill(3,2,4,1) etc.

 Function RandomNumbers()
    Dim r As New Random
    Dim numfilled As Integer = 1
    Dim arraytofill(4) As Integer
    Dim rnumber As Integer = r.Next(1, 5)
    Dim found = False

    While numfilled < 5 'creating 5 numbers
        rnumber = r.Next(1, 5) 'generate numbers
        For i = 1 To numfilled

            If arraytofill(i) = rnumber Then
                found = True 'there is a repeated number

            End If
        Next

        If found = False Then
            arraytofill(numfilled) = rnumber
            numfilled = numfilled + 1 'add to array and go to next cycle
        Else
            found = False
        End If
    End While

    Return arraytofill
End Function

Then there is the code to ask the questions. The user can pick from as many as 14 categories and each category has 5 questions inside of it.

 Sub NextQuestion()
Dim rnumbers() As Integer
rnumbers() = RandomNumbers()
    buttons = {Answer1Btn, Answer1Btn, Answer2Btn, Answer3Btn, Answer4Btn}

 If questionsincategory <> 5 Then
questionsincategory = questionsincategory + 1
    nQuestion = nQuestion + 1

    QuestionTxtBox.Text = TestQuestions(nQuestion).question
    ButtonArray(rnumbers(1)).Text = TestQuestions(nQuestion).answer1
    ButtonArray(rnumbers(2)).Text = TestQuestions(nQuestion).answer2
    ButtonArray(rnumbers(3)).Text = TestQuestions(nQuestion).answer3
    ButtonArray(rnumbers(4)).Text = TestQuestions(nQuestion).correct
    CurrentCatBox.Text = "The Current Category is: " & TestQuestions(nQuestion).category

    t = t + 1
    QuestionNumTxtBox.Text = "Question " & t & "/" & num * 5
 Else

    ' are there anymore categories?
    questionsincategory = 1
    categorycounter = categorycounter + 1
    If categorycounter < num Then
        'there is at least 1 more category

        For j = nQuestion To 65 'I have 65 questions loaded into the program from a file


            If TestQuestions(j).category = choices(categorycounter) Then
                'outputs question here
                Dim rnumbers() As Integer = RandomNumbers()
                Dim ButtonArray() As Control = {Answer1Btn, Answer1Btn, Answer2Btn, Answer3Btn, Answer4Btn}

                QuestionTxtBox.Text = TestQuestions(j).question
                ButtonArray(rnumbers(1)).Text = TestQuestions(j).answer1
                ButtonArray(rnumbers(2)).Text = TestQuestions(j).answer2
                ButtonArray(rnumbers(3)).Text = TestQuestions(j).answer3
                ButtonArray(rnumbers(4)).Text = TestQuestions(j).correct
                nQuestion = j

                CurrentCatBox.Text = "The Current Category is: " & TestQuestions(j).category
                t = t + 1
                QuestionNumTxtBox.Text = "Question " & t & "/" & num * 5

                'leave for loop to prevent further questions appearing
                Exit For
            End If
        Next
    Else
        'output score and close form
        MessageBox.Show("You scored " & QuizPoints & " points out of " & num * 5)
        MyConnection() 'connect to database and email results
        EmailResults()
    End If
    End If
End Sub

At the moment, when the code runs, it does not randomise the order of the buttons- despite the arraytofill() being correctly randomly ordered when I step through it. The code works like it did before I implemented the random sorting, which is strange to me. Is it not identifying the index correctly?

This is my first time posting on here so I hope I have made myself clear and understandable. Your guidance is very much appreciated.