fix: panic in `regexp.MustCompile` when building wildcarddirectories with non ascii characters #1947

camc314 · 2025-10-24T13:47:12Z

No description provided.

Copilot

Pull Request Overview

This PR fixes a panic in regexp.MustCompile when building wildcard directories with non-ASCII characters in file paths. The issue was caused by the regex pattern using \w (which only matches ASCII word characters in Go's regexp package), making it incompatible with Unicode characters commonly found in international file paths.

Key Changes:

Updated the reserved character pattern to explicitly escape only regex metacharacters instead of using negated character classes
Replaced Go-incompatible \w pattern with explicit list of special regex characters
Added comprehensive test coverage for non-ASCII characters in wildcard directory paths

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
internal/vfs/utilities.go	Fixed regex pattern to escape only actual regex metacharacters, removing unsupported `\w` pattern that caused panics with Unicode
internal/tsoptions/wildcarddirectories_test.go	Added new test cases covering Norwegian, Japanese, Chinese, and other Unicode characters in file paths

Copilot · 2025-10-24T13:47:50Z

internal/vfs/utilities.go

+// so we only escape characters that have special meaning in regex.
 var (
-	reservedCharacterPattern *regexp.Regexp = regexp.MustCompile(`[^\w\s/]`)
+	reservedCharacterPattern *regexp.Regexp = regexp.MustCompile(`[\\.\+*?()\[\]{}^$|#]`)


The backslash in the character class should not be escaped. In Go regex, within a character class [], a literal backslash only needs to appear once. The pattern [\\.\+*?()\[\]{}^$|#] attempts to escape the backslash itself, which may not match literal backslashes correctly. Change to [\\.+*?()\[\]{}^$|#] (single backslash, no escape for +).

Suggested change

reservedCharacterPattern *regexp.Regexp = regexp.MustCompile(`[\\.\+*?()\[\]{}^$|#]`)

reservedCharacterPattern *regexp.Regexp = regexp.MustCompile(`[\\.+*?()\[\]{}^$|#]`)

Copilot · 2025-10-24T13:47:51Z

internal/vfs/utilities.go

+// so we only escape characters that have special meaning in regex.
 var (
-	reservedCharacterPattern *regexp.Regexp = regexp.MustCompile(`[^\w\s/]`)
+	reservedCharacterPattern *regexp.Regexp = regexp.MustCompile(`[\\.\+*?()\[\]{}^$|#]`)


The + character does not need to be escaped inside a character class. In Go regex character classes, + is treated as a literal character. Change \+ to + for consistency and correctness.

Suggested change

reservedCharacterPattern *regexp.Regexp = regexp.MustCompile(`[\\.\+*?()\[\]{}^$|#]`)

reservedCharacterPattern *regexp.Regexp = regexp.MustCompile(`[\\.+*?()\[\]{}^$|#]`)

jakebailey · 2025-10-24T15:30:10Z

I'm not sure I understand; if this were panicking wouldn't everything be broken right now?

camc314 · 2025-10-24T15:35:13Z

I'm not sure I understand; if this were panicking wouldn't everything be broken right now?

sorry should've clarified in the PR description, apparently this is only an issue when you've got non-ascii characters inside the path on windows machines.

panic: regexp: Compile(`(?i)^((C\:/Users/TobiasL\ægreid/dev/app/frontend/packages/react/src(/.+?)?/[^/]*\.test\.ts[^/]*)|(C\:/Users/TobiasL\ægreid/dev/app/frontend/packages/react/src(/.+?)?/[^/]*\.stories\.ts[^/]*)|(C\:/Users/TobiasL\ægreid/dev/app/frontend/packages/react/src(/.+?)?/[^/]*\.mdx))($|/)`): error parsing regexp: invalid escape sequence: `\æ`
goroutine 299 [running]:
regexp.MustCompile({0xc0000b0840, 0x151})
        /opt/hostedtoolcache/go/1.25.0/x64/src/regexp/regexp.go:313 +0xb4
github.com/microsoft/typescript-go/internal/tsoptions.getWildcardDirectories({0xc000500f70, 0x1, 0x70?}, {0xc0000a9840?, 0xc000118900?, 0x70?}, {0x28?, {0xc000118c60?, 0x7ff62e770331?}})

jakebailey · 2025-10-24T15:55:31Z

Ah, this is oxc-project/tsgolint#318. I didn't realize you meant a knock-on effect from a later regex call.

jakebailey · 2025-10-24T15:56:10Z

internal/vfs/utilities.go

-// It may be inefficient (we could just match (/[-[\]{}()*+?.,\\^$|#\s]/g), but this is future
-// proof.
+// Reserved characters - only escape actual regex metacharacters.
+// Go's regexp doesn't support \x escape sequences for arbitrary characters,


We shouldn't be using regexp at all; does switching to regexp2 fix this?

Yeah, I don't understand why getWildcardDirectories uses regexp instead of regexp2.

(I would rather we not use regexp/regexp2 at all in the codebase but that's another problem.)

i would guess it's still a problem with regexp2.

I remove this regex in favor of a simple for loop though 6bf5543

It shouldn't be, given regexp2 has a mode that emulates ECMAScript regexes?

If we are going to use regexp and not regexp2, then we can probably just use regexp.QuoteMeta.

jakebailey · 2025-10-24T17:56:54Z

The latest commit is not exactly what I mean; I wasn't trying to say to continue to change the escaping, but rather fix getWildcardDirectories to not use Go's regexp package, given the original code was more or less trying to emulate ECMAScript regexes which can be done with regexp2.

I feel somewhat less confident about hand escaping regexes like this unless we can be sure that those characters are definitely all that matter?

Otherwise, we'd just use https://pkg.go.dev/regexp#QuoteMeta, right?

camc314 · 2025-10-25T11:52:16Z

The latest commit is not exactly what I mean; I wasn't trying to say to continue to change the escaping, but rather fix getWildcardDirectories to not use Go's regexp package, given the original code was more or less trying to emulate ECMAScript regexes which can be done with regexp2.

I feel somewhat less confident about hand escaping regexes like this unless we can be sure that those characters are definitely all that matter?

Otherwise, we'd just use https://pkg.go.dev/regexp#QuoteMeta, right?

Ah i see. i reverted that commit. The problem is that GetRegularExpressionForWildcard combines multiple strings, so it has to be escaped, hence we can't just omit the regex escaping change. Regardless, i changed wildcarddirectories to use regexp2.

Hopefully I understood that correctly 🙂

jakebailey · 2025-10-25T16:19:37Z

Why does Strada not need the escaping change, though?

jakebailey · 2025-10-28T18:49:03Z

internal/vfs/utilities.go

+// so we only escape characters that have special meaning in regex.
 var (
-	reservedCharacterPattern *regexp.Regexp = regexp.MustCompile(`[^\w\s/]`)
+	reservedCharacterPattern *regexp.Regexp = regexp.MustCompile(`[\\.\+*?()\[\]{}^$|#]`)


The goal of suggesting regex2 was to avoid making this particular change. If we still want to restrict the characters, it makes me think we should just use QuoteMeta and not change anything else.

fix: panic in regexp.MustCompile when building wildcarddirectories

ce774d0

Copilot AI review requested due to automatic review settings October 24, 2025 13:47

Copilot AI reviewed Oct 24, 2025

View reviewed changes

camc314 changed the title ~~fix: panic in regexp.MustCompile when building wildcarddirectories~~ fix: panic in regexp.MustCompile when building wildcarddirectories with non ascii characters Oct 24, 2025

jakebailey reviewed Oct 24, 2025

View reviewed changes

remove use of regex

6bf5543

u

853ac9d

camc314 force-pushed the c/fix-panic-when-compiling-regex branch from 136d7fe to 853ac9d Compare October 25, 2025 11:49

jakebailey reviewed Oct 28, 2025

View reviewed changes

	reservedCharacterPattern regexp.Regexp = regexp.MustCompile(`[\\.\+?()\[\]{}^$\|#]`)
	reservedCharacterPattern regexp.Regexp = regexp.MustCompile(`[\\.+?()\[\]{}^$\|#]`)

fix: panic in regexp.MustCompile when building wildcarddirectories with non ascii characters #1947

Are you sure you want to change the base?

fix: panic in regexp.MustCompile when building wildcarddirectories with non ascii characters #1947

Uh oh!

Conversation

camc314 commented Oct 24, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

jakebailey commented Oct 24, 2025

Uh oh!

camc314 commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jakebailey commented Oct 24, 2025

Uh oh!

jakebailey Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jakebailey Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

camc314 Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

jakebailey Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

jakebailey commented Oct 24, 2025

Uh oh!

camc314 commented Oct 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jakebailey commented Oct 25, 2025

Uh oh!

jakebailey Oct 28, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

fix: panic in `regexp.MustCompile` when building wildcarddirectories with non ascii characters #1947

fix: panic in `regexp.MustCompile` when building wildcarddirectories with non ascii characters #1947

camc314 commented Oct 24, 2025 •

edited

Loading

jakebailey Oct 24, 2025 •

edited

Loading

camc314 commented Oct 25, 2025 •

edited

Loading