Python Regular Expressions: Patterns, Groups, Lookarounds, and the re Module

A regular expression describes a set of text patterns. Python’s re module can use a pattern to locate text, extract structured fields, split input, or replace matching fragments. A regular-expression operation has three distinct inputs: the source text, a pattern written as a Python string, and the re engine that interprets the pattern.

Keeping those layers separate prevents the most common beginner error. Python parses the string literal first; the regular-expression engine then parses the resulting string as a pattern. A backslash may therefore have meaning in both layers.

1. Python Strings and Regular-Expression Patterns

1.1 Python escape sequences

Inside an ordinary Python string, a backslash introduces an escape sequence. Common examples include \n for a newline, \r for a carriage return, \t for a tab, \\ for a literal backslash, \x41 for a hexadecimal byte value, and \u4e2d for a Unicode code point. Python treats the resulting value as a character, so len("\n") is 1 even though the source literal contains two visible symbols.

Quotes only require escaping when they would otherwise terminate the string. Both of the following values contain a single double-quote character:

quote_a = '"'
quote_b = "\""
assert quote_a == quote_b

A normal string cannot end with an unescaped backslash because the backslash would escape the closing quote. For the same reason, a raw string cannot end with an odd number of backslashes.

1.2 Raw strings remove one escaping layer

A raw-string prefix tells Python to preserve backslashes instead of interpreting most escape sequences. For example, r"\n" has length 2, whereas "\n" has length 1. Regular-expression patterns should normally use raw strings because regex syntax also relies heavily on backslashes:

ordinary_pattern = "\\d+"
raw_pattern = r"\d+"
assert ordinary_pattern == raw_pattern

Both values contain the pattern \d+, but the raw form is easier to read. Raw strings do not disable the regular-expression parser; r"\d" still means “a decimal digit” when re compiles it.

Matching a literal backslash demonstrates the two parsing layers. The regex engine needs \\ to represent a literal backslash, so the recommended Python pattern is r"\\". Without a raw string, the same pattern would be written as "\\\\".

2. Matching with the re Module

2.1 Choosing the right matching function

Import the standard-library module with import re. The main matching functions differ in where and how many matches they inspect:

Function	Behavior	Typical result
`re.match()`	Tries only at the start of the string	`Match` or `None`
`re.fullmatch()`	Requires the entire string to match	`Match` or `None`
`re.search()`	Finds the first match anywhere	`Match` or `None`
`re.findall()`	Returns all matches as strings or tuples	`list`
`re.finditer()`	Iterates over all `Match` objects	iterator
`re.split()`	Splits at every matching delimiter	`list`
`re.sub()`	Replaces matching text	`str`
`re.subn()`	Replaces text and reports the count	`(str, int)`

Use fullmatch for validation, search for locating one occurrence, and finditer when every match needs positions or captured fields. The difference between match and search is visible in a short example:

import re

text = "prefix abc suffix"
assert re.match(r"abc", text) is None
found = re.search(r"abc", text)
assert found is not None
assert found.span() == (7, 10)

A successful operation returns a re.Match object. start() and end() return the half-open match boundaries, span() returns both boundaries as a tuple, and group() returns matched text. group(0) always means the complete match; numbered and named groups expose captured subpatterns.

match = re.search(r"(?P<name>[a-z]+)=(\d+)", "size=42")
assert match is not None
assert match.group() == "size=42"
assert match.group(1) == "size"
assert match.group(2) == "42"
assert match.group("name") == "size"
assert match.groupdict() == {"name": "size"}

Always handle the possibility of None before calling match methods. An unchecked re.search(...).group() raises AttributeError when the text does not match.

2.2 Compiling patterns and applying flags

re.compile() creates a reusable pattern object. Compilation makes intent clear when one pattern is applied repeatedly, and methods on the object no longer need the pattern argument:

assignment = re.compile(r"(?P<name>[a-z_]+)=(?P<value>\d+)", re.IGNORECASE)

for text in ["size=42", "COUNT=7"]:
    match = assignment.fullmatch(text)
    if match:
        print(match.groupdict())

Python caches a limited number of recently used patterns, so explicit compilation is mainly useful for reuse and readability rather than as a universal performance trick. Common flags are re.IGNORECASE for case-insensitive matching, re.DOTALL to let . match newlines, re.MULTILINE to make ^ and $ operate at each line boundary, and re.VERBOSE to allow whitespace and comments inside a complex pattern.

3. Core Pattern Syntax

3.1 Literals, character classes, and shorthand classes

Most characters match themselves. The pattern r"abc" finds the exact sequence abc, and matching is case-sensitive unless re.IGNORECASE is enabled. A dot matches one character except a newline by default; re.DOTALL includes newlines.

Square brackets define a character class that consumes exactly one character. r"[1234]" matches one of four digits, r"[0-9]" matches one ASCII digit, and r"[^0-9]" matches one character outside that range. A caret negates the class only when it appears immediately after [. A hyphen defines a range between suitable endpoints; place it first or last, or escape it, when a literal hyphen is required.

Most metacharacters lose their special role inside a class. For example, r"[.]" matches a literal dot. A closing bracket and backslash are clearer when escaped, as in r"[\]\[]" for either bracket and r"[\\]" for a backslash.

The engine provides shorthand classes:

Pattern	Meaning for Python `str` patterns
`\d` / `\D`	Unicode decimal digit / not a decimal digit
`\w` / `\W`	Unicode alphanumeric or underscore / its complement
`\s` / `\S`	Unicode whitespace / not whitespace

Uppercase shorthand classes are complements of their lowercase forms. Python’s Unicode-aware defaults are broader than ASCII: \d can match decimal digits outside 0-9, and \w can match letters from many scripts. Use re.ASCII when a protocol or file format specifically requires ASCII semantics, or write an explicit range such as [0-9].

3.2 Concatenation, alternation, and quantifiers

Adjacent pattern elements match adjacent text. r"\d\w" consumes a digit followed by a word character. The alternation operator | chooses between alternatives, and it has low precedence: r"ab|c" means either ab or c, while r"a(?:b|c)" means ab or ac.

Quantifiers repeat the preceding atom or group:

Quantifier	Number of repetitions
`?`	zero or one
`*`	zero or more
`+`	one or more
`{m}`	exactly `m`
`{m,n}`	from `m` through `n`
`{m,}`	at least `m`
`{,n}`	at most `n`

Quantifiers are greedy by default. re.search(r"m{2,3}", "mmm") consumes three characters, whereas re.search(r"m{2,3}?", "mmm") consumes two. Adding ? after *, +, ?, or a brace quantifier selects the shortest match that still lets the complete pattern succeed.

Grouping determines what a quantifier repeats. r"(?:ab)+" matches ab, abab, and longer repetitions of the pair; r"ab+" matches one a followed by one or more b characters.

4. Groups and Backreferences

4.1 Capturing and non-capturing groups

Parentheses control precedence and normally create a capturing group. Groups are numbered by the position of their opening parenthesis, from left to right:

match = re.search(r"((\d)\w(\d))", "a1b3x")
assert match is not None
assert match.group(0) == "1b3"
assert match.group(1) == "1b3"
assert match.group(2) == "1"
assert match.group(3) == "3"

When a group is needed only for precedence or repetition, write (?:...). A non-capturing group does not receive a number and does not alter the numbering of surrounding captures. r"([a-c])(?:[d-f](<[g-h]>))", for example, has two captured groups even though the pattern contains three pairs of parentheses.

4.2 Named groups and backreferences

A named group uses (?P<name>...). Names make extraction code easier to maintain than positional indexes:

line_pattern = re.compile(
    r"^(?P<level>INFO|WARN|ERROR)\s+"
    r"(?P<date>\d{4}-\d{2}-\d{2})\s+"
    r"(?P<message>.+)$"
)

match = line_pattern.fullmatch("ERROR 2026-09-10 disk full")
assert match is not None
assert match.groupdict()["level"] == "ERROR"

A backreference requires later text to equal text already captured. Numbered backreferences use \1, \2, and so on. Named backreferences use (?P=name). The pattern r"\b(?P<word>\w+)\s+(?P=word)\b" finds an adjacent repeated word such as the the. A backreference compares text; a quantifier merely repeats the group’s pattern and may match different text each time.

Capturing groups also affect findall and split. If a findall pattern has one capture, the result contains that group rather than the complete match; multiple captures produce tuples. Use (?:...) when the caller needs complete matches but grouping is required inside the pattern.

5. Positions, Boundaries, and Lookarounds

5.1 Anchors, boundaries, and lookarounds

Anchors assert a position without consuming characters. ^ matches the start of the string, and $ matches the end or the position before a final newline. With re.MULTILINE, the same anchors also apply at line boundaries. \A and \Z retain whole-string behavior regardless of multiline mode. For strict validation, fullmatch is usually clearer than manually adding start and end anchors.

\b asserts a boundary between a word character and a non-word character, while \B asserts the opposite. Word boundaries follow the engine’s definition of \w; under the default Unicode behavior, letters outside ASCII count as word characters.

Lookarounds assert surrounding text without including the asserted text in the match:

Form	Requirement
`(?=...)`	Positive lookahead: following text must match
`(?!...)`	Negative lookahead: following text must not match
`(?<=...)`	Positive lookbehind: preceding text must match
`(?<!...)`	Negative lookbehind: preceding text must not match

The pattern r"(?<!\d)abc(?=\d)" finds abc only when no digit appears immediately before it and a digit appears immediately after it. The returned match still contains only abc.

Python’s built-in re engine requires the inside of a lookbehind to have a fixed width. (?<=ab) and (?<=\d{3}) are valid; a variable-length lookbehind such as (?<=a+) is rejected. When a requirement can be expressed by consuming text and capturing the desired field, that structure is often easier to understand than several stacked assertions.

5.2 Precedence and readable complex patterns

The practical precedence order is escape sequences and character classes first, quantifiers next, concatenation after that, and alternation last. Parentheses make intended grouping explicit. For a long pattern, re.VERBOSE permits indentation and comments without turning layout spaces into matched spaces:

date_pattern = re.compile(
    r"""
    (?P<year>\d{4}) -
    (?P<month>0[1-9]|1[0-2]) -
    (?P<day>0[1-9]|[12]\d|3[01])
    """,
    re.VERBOSE,
)

The date_pattern example validates the numeric shape and broad ranges of a date, but the expression does not know that February 30 is invalid. Regular expressions are well suited to lexical structure; domain rules may still need ordinary Python code or a parser such as datetime.

6. Finding, Splitting, and Replacing Text

6.1 Collecting matches and splitting text

findall is convenient when only matched values are required. With no capturing groups it returns complete matches; with groups it returns captured strings or tuples:

text = "aAxxxxbB"
assert re.findall(r"[a-z][A-Z]", text) == ["aA", "bB"]
assert re.findall(r"([a-z])([A-Z])", text) == [("a", "A"), ("b", "B")]

finditer returns Match objects lazily and is preferable when the input is large or positions are needed:

for match in re.finditer(r"\d+", "ports: 80, 443"):
    print(match.group(), match.span())

re.split(pattern, text, maxsplit=0) splits on every match unless maxsplit limits the count. Consecutive delimiters and delimiters at an endpoint can produce empty strings. Capturing the delimiter inserts it into the result, which is useful when separators must be retained.

assert re.split(r"[,;]\s*", "alpha, beta;gamma") == [
    "alpha",
    "beta",
    "gamma",
]
assert re.split(r"([,;])\s*", "alpha, beta;gamma") == [
    "alpha",
    ",",
    "beta",
    ";",
    "gamma",
]

Use str.split() when a fixed literal separator is sufficient. A regular expression becomes useful when several delimiters, variable whitespace, or contextual rules must be recognized.

6.2 Replacing matches with strings or functions

re.sub(pattern, replacement, text, count=0) replaces all matches by default. A replacement string can refer to captured groups with \g<name> or \g<1>. A callable replacement receives a Match object and computes each new value:

text = "Ada:5 Linus:8"
normalized = re.sub(
    r"(?P<name>[A-Za-z]+):(?P<score>\d+)",
    r"\g<name>=\g<score>",
    text,
)

incremented = re.sub(
    r"\d+",
    lambda match: str(int(match.group()) + 1),
    text,
)

assert normalized == "Ada=5 Linus=8"
assert incremented == "Ada:6 Linus:9"

re.subn() performs the same replacement and returns (new_text, replacement_count). The count is valuable in migrations and cleanup scripts because an unexpected value can reveal that the input format changed.

7. Designing and Testing Reliable Patterns

Start with representative examples and decide whether the task requires locating, extracting, splitting, replacing, or validating. Write the smallest pattern that captures the actual structure, compile it when reused, and test both expected matches and expected failures. Boundary cases should include empty text, Unicode characters, newlines, repeated delimiters, and unusually long input.

A pattern can be syntactically correct yet produce excessive backtracking. Nested ambiguous quantifiers such as (a+)+ are dangerous on adversarial input. Prefer explicit delimiters, bounded repetitions, and simpler parsing logic when the input has recursive or deeply nested structure. Regular expressions work well for tokens and flat records; formats such as HTML, JSON, and programming languages already have dedicated parsers.

The authoritative references are the Python documentation for re and the Regular Expression HOWTO. When behavior matters across Python versions, test against the version used in production because supported syntax and Unicode tables can change.