In .Net, substitutions in regular expression replacement patterns are preceeded with the dollar symbol. Powershell also uses the dollar sign for variables. And, since Powershell attempts to expand variable when they are in double quoted strings, when passing a replacement pattern to a regular expression based operator or function, you must either use single quotes or escape the dollar symbol. Otherwise, Powershell will attempt the expand the regular expression substitution as if it were a powershell variable.

So, as an example, if we wanted to change the format of a phone number, we could run this:

"(555) 555-5555" -replace "^\((\d{3})\) (\d{3})-(\d{4})",'$1-$2-$3'

Which would output:

555-555-555

But, if we used double quotes like so:

"(555) 555-5555" -replace "^\((\d{3})\) (\d{3})-(\d{4})","$1-$2-$3"

Then, Powershell would look for variables called 1,2, and 3.
If they weren't defined, the output would be:

--