SIP-72: dedented triple-quoted string literals

[lihaoyi]

Implements scala/improvement-proposals#112

• The initial Lexing/Scanning is lenient, only looking for the opening ''''* and equivalent closing delimiter. This matches how we can expect this to be implemented in other tools that have more restricted lexing frameworks (IntelliJ w/ JFlex, VSCode w/ TextMate Grammars, NeoVim w/ TreeSitter)

• All other validation (opening delimiter must be followed by newline, closing delimiter must be preceded by whitespace only) and de-denting is left to the parsing phase, which is the only time we have a complete string "literal" when an interpolator is present, and thus are able to look at the trailing delimiter's preceding indent whitespace and trim it from all earlier STRINGLIT/STRINGPART tokens

• def interpolatedString needed to be refactored to support dedenting: rather than constructing the trees immediately, we first assemble all the strings parts, then use the last string part to compute the dedent that we apply to all other parts, and only then do we construct the trees

• Covered by neg/ tests and run/ tests for all the major features and edge cases I could think of:

  • All indentation removed
  • Some indentation preserved
  • Empty strings
  • Single-line strings
  • Blank lines in the string
  • Leading and trailing blank lines
  • Varying indentation
  • Extensible delimiters with 4 and 5 quotes
  • Funky operator and unicode characters in the string
  • Tab-based indentation
  • Interpolation with s and f
  • Single- and Multi-line pattern matching with and without interpolation
  • In larger expressions: lists, infix operators, etc.
  • As singleton-type ascriptions and singleton-type parameters
  • As literals passed to @compileTimeOnly

• I haven't managed to reliably run tests for some reason, I think I'm bumping into https://contributors.scala-lang.org/t/current-testcompilation/7256. But I tested it manually by copy-pasting the run/neg test files into the bin/scala REPL and compared the output manually with the .check files on disk, and the output is identical

[odersky]

That was fast!

[lihaoyi]

Not ready to review yet! Still need a bit more vibing haha

[Gedochao]

Not ready to review yet! Still need a bit more vibing haha

Ah right, I'll convert it to draft then

[He-Pin]

Should be able to detect this in one loop

[lihaoyi]

@odersky @sjrd looks like most of the tests are green, this should be ready for review. I couldn't find any error message in the failing job's logs, might need help from someone more familiar with the CI setup to help take a look

[lihaoyi]

The last failure turned out to be due to error message positioning issues, fixed that and it's all green now

[lihaoyi]

@noti0na1 resolved your comments

[lihaoyi]

also wired up the REPL incomplete literal detection to work for ''' strings