Securely generating PKCE code for OAuth public client with PowerShell

I'm trying to generate a 43-octet (Byte) string to use as code verifier for OAuth authentication with PowerShell as described in RFC7636.

4. Protocol

4.1. Client Creates a Code Verifier

The client first creates a code verifier, "code_verifier", for each OAuth 2.0 [RFC6749] Authorization Request, in the following manner:

code_verifier = high-entropy cryptographic random STRING using the unreserved characters [A-Z] / [a-z] / [0-9] / "-" / "." / "_" / "~"
from Section 2.3 of [RFC3986], with a minimum length of 43 characters and a maximum length of 128 characters.

ABNF for "code_verifier" is as follows.

code-verifier = 43*128unreserved unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" ALPHA = %x41-5A / %x61-7A DIGIT = %x30-39

NOTE: The code verifier SHOULD have enough entropy to make it
impractical to guess the value. It is RECOMMENDED that the output of a suitable random number generator be used to create a 32-octet
sequence. The octet sequence is then base64url-encoded to produce a
43-octet URL safe string to use as the code verifier.

I found that using the RNGCryptoServiceProvider class is more likely to generate a high-entropy random number but when I convert it into a base64 string, it has undesired characters.

$RandomNumberGenerator = New-Object System.Security.Cryptography.RNGCryptoServiceProvider
    $Bytes = New-Object Byte[] 32
    $RandomNumberGenerator.GetBytes($Bytes)
    [System.Convert]::ToBase64String($bytes)

How can I generate a compliant base64 string using the RNGCryptoServiceProvider?