[horhulenko]

Resolves #660

A naïve regex for comments, #[^\n]*, matched inside strings too, and a """ inside a comment opened a docstring block that swallowed the rest of the file.

Now a # inside a string and quotes inside a comment no longer break the highlighting.

fix/python-syntax-highlighter-context-reworked replaces the regexes with a small scanner and passes every case below, but the diff is much larger. Your call whether it's worth it.

Checked against the file below, lines highlighted correctly:

test case	before	this PR	reworked
escaped quotes	0/8 (0%)	8/8 (100%)	8/8 (100%)
raw strings	0/5 (0%)	5/5 (100%)	5/5 (100%)
prefixes	0/4 (0%)	4/4 (100%)	4/4 (100%)
quote clusters	5/15 (33%)	15/15 (100%)	15/15 (100%)
multi-line triple strings¹	0/19 (0%)	1/19 (5%)	19/19 (100%)
line continuation in strings	0/5 (0%)	2/5 (40%)	5/5 (100%)
f-strings	0/14 (0%)	9/14 (64%)	14/14 (100%)
comments containing quotes	2/4 (50%)	4/4 (100%)	4/4 (100%)
no space / weird spacing	4/8 (50%)	8/8 (100%)	8/8 (100%)
# in other syntax	3/7 (43%)	7/7 (100%)	7/7 (100%)
unicode / escapes	2/4 (50%)	4/4 (100%)	4/4 (100%)
string as statement	1/2 (50%)	2/2 (100%)	2/2 (100%)
tricky endings	4/4 (100%)	4/4 (100%)	4/4 (100%)
total	21/99 (21%)	73/99 (74%)	99/99 (100%)
vs before	—	+248%	+371%
vs this PR	−71%	—	+36%
vs reworked	−79%	−26%	—

No regressions.

¹ before and this PR color multi-line """ strings like comments, and an escaped \""" inside one closes it early, which shifts every block after it. The reworked branch fixes both and uses the string color.

test file

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# Torture test for # and string/""" highlighting. Requires Python 3.12+ (PEP 701 f-strings).
# Every line marked "# OK" must have exactly that trailing part as a comment.

# ---------- 1. escaped quotes ----------
s1 = "a \" # b"                 # OK  (your screenshot case)
s2 = 'a \' # b'                 # OK
s3 = "\\"                       # OK  escaped backslash, string ends here
s4 = "\\\" # still string"      # OK  \\ then \" -> string continues
s5 = "\\\\" # OK
s6 = '\\' + "#" + '\\\''        # OK
s7 = "it's # fine"              # OK  other quote type inside
s8 = 'say "hi" # there'         # OK

# ---------- 2. raw strings (backslash still blocks the quote!) ----------
r1 = r"\"" # OK   content is \"
r2 = r'\'' # OK
r3 = r"\\" # OK
r4 = R"C:\path\#notcomment" # OK
r5 = rb'\'#' + Rb"#" + bR"\"#" + BR'#' # OK

# ---------- 3. prefixes ----------
p1 = b"#" + B'#'    # OK
p2 = u"#" + U'#'    # OK
p3 = f"#" + F'#'    # OK
p4 = rf"\{{#}}" + fR'#' + Rf"#" # OK

# ---------- 4. quote clusters ----------
e1 = ""     # OK  empty
e2 = ''''''  # OK  empty triple
e3 = """""" # OK  empty triple
e4 = "a"""  # OK  "a" followed by "" (NOT a triple quote!)
e5 = "" "" ""   # OK  three empty strings
e6 = """a\"""" # OK  content: a"
e7 = """a""" "#" # OK
e8 = '"""' # OK  triple quote inside single-quoted string
e9 = "'''" + '"' + "'" # OK
e10 = """ " "" ' '' ''' # """ # OK
e11 = ''' " "" """ ' '' # ''' # OK
e12 = """\\""" # OK  escaped backslash, then closing """
e13 = """abc\"""" # OK
e14 = r"""a\"""" # OK  raw: content a\"
e15 = '''''' + """""" + "" + '' # OK

# ---------- 5. multiline triple strings with fake comments ----------
m1 = """
# not a comment
    # still not a comment
''' still inside
" single dq
"" two dq
\""" escaped triple, still inside
"""  # OK

m2 = '''# line 1
""" #
''' # OK

def func():
    """Docstring with # hash and 'quotes' and "quotes".

    >>> func()  # doctest comment, still string
    """
    return 1  # OK

def func2(): '''one-liner # docstring'''; return 2  # OK

class C:
    """""" # OK
    x = """
    # """ # OK

# ---------- 6. line continuation inside strings ----------
c1 = "abc\
# not a comment, continued string\
" # OK
c2 = 'x\
' # OK

# ---------- 7. f-strings (3.12+) ----------
x = 255
f1 = f"{x:#x}"                    # OK  format spec #, not a comment
f2 = f"{'#'}"                     # OK
f3 = f"{"#"}"                     # OK  same quotes nested (3.12)
f4 = f"{f"{f"{'#'}"}"}"           # OK  deep nesting
f5 = f"{ {'a': '#'}['a'] }"       # OK
f6 = f"{{#}}"                     # OK  literal braces
f7 = f'{x!r:#>10}'                # OK
f8 = f"""{
    x  # REAL comment inside f-string replacement field
}"""                              # OK
f9 = f"{"\"#"}"                   # OK
f10 = f"{'''#'''}" + f'{"""#"""}' # OK
f11 = rf"{x}\{{#" # OK
f12 = f"{x:{'#'}>{10}}" # OK  nested format spec

# ---------- 8. comments containing quotes ----------
y = 1  # it's a "comment" with """ and ''' -- must NOT open a string
z = 2  # """
w = 3  # '
# """ this whole line is a comment, next line is code
v = 4  # OK

# ---------- 9. no space / weird spacing ----------
a=1#OK
b="#"#OK
c='#'#'OK'
d	=	5	# OK (tabs)
e = [ "#", '#',  # OK
      "#" ]  # OK
#OK
	#OK (tab-indented comment line)

# ---------- 10. # in other syntax ----------
def g(a: "Type#1" = "#", *, b='#') -> "R#":  # OK
    return {"#": '#'}["#"]  # OK

@staticmethod  # OK
def h(): pass  # OK

lam = lambda q="#": q  # OK
cond = "#" if "#" in "##" else '#'  # OK
k = 1 if True else 2  # OK

# ---------- 11. unicode / escapes ----------
u1 = "\N{NUMBER SIGN}"  # OK
u2 = "\x23\u0023\43"    # OK  all are '#'
u3 = "é # ü"            # OK
u4 = "😀 # 😀" # OK

# ---------- 12. string as statement, then code ----------
"""standalone string"""; s_after = 1  # OK
'#'; "#"; '''#'''; """#"""  # OK

# ---------- 13. tricky endings ----------
t1 = "ends with backslash-quote \\" + '\\' # OK
t2 = """ends with quote" """ # OK
t3 = """ends with two quotes"" """ # OK
t4 = '''it''s''' # OK  'it' adjacent? no: one triple string containing it''s

# last line: comment without trailing newline
# EOF