Python Regex

A regular expression (regex, or regexp) is a mini pattern language for describing shapes of text: “three digits”, “anything that looks like an email address”, “a capital letter right after whitespace”. Python exposes this power through the built-in re module, which turns your pattern string into an internal matching engine and runs it against text to search, validate, extract, or rewrite content. Regex syntax looks dense at first, but it is extremely reusable — nearly the same syntax works in JavaScript, grep, editors, and dozens of other languages, so the time you invest here pays off everywhere.

Overview: How Regular Expressions Work

The re module does not reinterpret your pattern character-by-character every time you match. Calling a function like re.search() or re.compile() parses the pattern string once into a Pattern object — internally a small program that knows how to walk through text one character at a time, branching whenever the pattern offers alternatives (like | or a quantifier such as *).

When that program runs against a string, the engine tries to align the pattern starting at each position (for search()) or only at position 0 (for match()). Most parts of a pattern are greedy by default: a quantifier like + or * first tries to consume as much text as possible, then backtracks — gives characters back one at a time — if the rest of the pattern fails to match what comes next. This backtracking is why complex patterns can occasionally be slow on adversarial input, and why greedy vs. lazy (non-greedy) quantifiers matter, as you’ll see in Common Mistakes below.

A successful match produces a Match object, not just a string. That object remembers where the match started and ended (.start(), .end(), .span()), the full matched text (.group() or .group(0)), and the text captured by any parenthesized groups inside the pattern (.group(1), .groups(), or named groups via .groupdict()). If nothing matches, functions like re.search() and re.match() return None — always check for that before calling .group(), or you will get an AttributeError.

One detail trips up almost everyone eventually: regex patterns rely heavily on backslash sequences like \d, \w, and \b. Python’s own string literals also use backslashes for escapes (\n, \t, and so on), so a plain string like "\b" is silently interpreted by Python before the regex engine ever sees it, turning it into a backspace character instead of the regex “word boundary” token. That is why regex patterns should almost always be written as raw strings, prefixed with r, e.g. r"\d+\s\w+". A raw string tells Python “don’t process backslash escapes, hand the characters straight through” — exactly what the re module needs.

Syntax

The general shape of using the module is: write a pattern string, then call one of its matching functions against a target string.

import re

pattern = r"regex-pattern-here"
result = re.search(pattern, "some text to search")
print(result)

The table below lists the most common top-level functions. Each is also available as a method on a compiled re.Pattern object returned by re.compile(), which is faster when you reuse the same pattern many times:

Function	What it does
re.search(pattern, s)	Scans the whole string, returns a Match for the first location that matches, or None.
re.match(pattern, s)	Tries to match only at the start of the string.
re.fullmatch(pattern, s)	Matches only if the pattern accounts for the entire string.
re.findall(pattern, s)	Returns a list of all non-overlapping matches (or tuples of groups, if the pattern has groups).
re.finditer(pattern, s)	Like findall, but returns an iterator of Match objects, preserving position and group info.
re.sub(pattern, repl, s)	Returns a new string with matches replaced by repl.
re.subn(pattern, repl, s)	Like sub, but also returns the number of substitutions made.
re.split(pattern, s)	Splits the string wherever the pattern matches.
re.compile(pattern)	Pre-compiles a pattern into a reusable Pattern object.

Inside the pattern string itself, a small set of metacharacters and special sequences do almost all the work:

Syntax	Meaning
.	Any character except newline (unless re.DOTALL is set)
\d / \D	A digit / a non-digit
\w / \W	A “word” character (letter, digit, underscore) / not one
\s / \S	Whitespace / non-whitespace
\b / \B	Word boundary / not a word boundary
^ / $	Start / end of string (or line, with re.MULTILINE)
*, +, ?	0-or-more, 1-or-more, 0-or-1 of the preceding element (all greedy)
{m,n}	Between m and n repetitions
*?, +?, ??	Lazy (non-greedy) versions of the quantifiers above
[...]	A character class, e.g. [a-zA-Z0-9_]
(...)	A capturing group
(?:...)	A non-capturing group
(?P<name>...)	A named capturing group
|	Alternation (“or”)
(?=...) / (?!...)	Lookahead: matches here only if followed / not followed by …

Most functions also accept an optional flags argument, most commonly re.IGNORECASE (case-insensitive matching), re.MULTILINE (make ^ and $ match at each line break, not just the string edges), and re.DOTALL (make . match newlines too).

Examples

Example 1: search(), findall(), and match()

import re

text = "The quick brown fox jumps over 3 lazy dogs, and 42 cats watched."

# Find all sequences of digits
numbers = re.findall(r"\d+", text)
print("Numbers found:", numbers)

# Search for the first word starting with 'q'
match = re.search(r"\bq\w*", text)
if match:
    print("First 'q' word:", match.group())
    print("Match span:", match.span())

# Check if the string starts with "The"
if re.match(r"The", text):
    print("Text starts with 'The'")

Output:

Numbers found: ['3', '42']
First 'q' word: quick
Match span: (4, 9)
Text starts with 'The'

re.findall(r"\d+", text) scans the whole string and returns every run of digits, as a list of strings — \d+ means “one or more digits”, so “3” and “42” are returned separately even though they’re different lengths. re.search() returns a single Match object for the first place the pattern matches anywhere in the string; \bq\w* means “a word boundary, then a literal q, then zero or more word characters”, which captures the whole word “quick”. Finally, re.match() only checks the very start of the string, so it succeeds because the text literally begins with “The”.

Example 2: capturing groups and named groups

import re

log_line = "2026-07-19 14:32:07 ERROR Failed to connect to database"

pattern = r"(?P<date>\d{4}-\d{2}-\d{2}) (?P<time>\d{2}:\d{2}:\d{2}) (?P<level>[A-Z]+) (?P<message>.+)"
match = re.match(pattern, log_line)

if match:
    print("Date:", match.group("date"))
    print("Time:", match.group("time"))
    print("Level:", match.group("level"))
    print("Message:", match.group("message"))
    print("All groups:", match.groups())
    print("As dict:", match.groupdict())

Output:

Date: 2026-07-19
Time: 14:32:07
Level: ERROR
Message: Failed to connect to database
All groups: ('2026-07-19', '14:32:07', 'ERROR', 'Failed to connect to database')
As dict: {'date': '2026-07-19', 'time': '14:32:07', 'level': 'ERROR', 'message': 'Failed to connect to database'}

Each (?P<name>...) block is a named capturing group. Instead of remembering that “group 3 is the log level”, you can ask for match.group("level") directly, which makes patterns with many groups far more readable. match.groups() returns all captured text as a plain tuple in order, and match.groupdict() returns the same data as a dictionary keyed by group name — handy for feeding straight into something like a dataclass or a database row.

Example 3: extracting and redacting with finditer() and sub()

import re

text = """Contact us at support@example.com or sales@example.org.
Call (555) 123-4567 for urgent issues."""

email_pattern = re.compile(r"[\w.+-]+@[\w-]+\.[a-zA-Z]{2,}")

emails = email_pattern.findall(text)
print("Emails:", emails)

for m in email_pattern.finditer(text):
    print(f"Found '{m.group()}' at position {m.start()}-{m.end()}")

masked = email_pattern.sub("[REDACTED]", text)
print(masked)

phone_pattern = re.compile(r"\(\d{3}\)\s\d{3}-\d{4}")
phone_match = phone_pattern.search(text)
print("Phone:", phone_match.group() if phone_match else "Not found")

Output:

Emails: ['support@example.com', 'sales@example.org']
Found 'support@example.com' at position 14-33
Found 'sales@example.org' at position 37-54
Contact us at [REDACTED] or [REDACTED].
Call (555) 123-4567 for urgent issues.
Phone: (555) 123-4567

This example compiles the pattern once with re.compile() and reuses it three ways: findall() to list every email, finditer() to get each match’s exact position, and sub() to build a new string with every match replaced by "[REDACTED]". Notice the domain part of the pattern, [a-zA-Z]{2,}, deliberately excludes digits and punctuation so it stops at the trailing period after “org” instead of swallowing it — a small but important precision detail in real-world patterns.

Under the Hood: What Happens During a Match

When you call re.search(pattern, text), Python checks an internal cache of recently compiled patterns; if this exact pattern string hasn’t been compiled before, it parses it into a tree of operations (literal, character class, group, quantifier, alternation, and so on) and turns that tree into a small program the matching engine can execute — conceptually similar to a state machine, though CPython’s implementation is a backtracking engine rather than a true finite automaton.

To find a match, the engine picks a starting position in the text (position 0 for match(); every position left-to-right for search()) and walks the compiled program step by step. Each literal character or class either consumes one character of input or fails immediately. Quantifiers like * and + push a choice point: “try consuming another character, but remember this position in case the rest of the pattern needs some characters back.” That’s backtracking — if a later part of the pattern can’t match, the engine rewinds to the most recent choice point and tries a smaller amount of repetition.

This is also why lazy quantifiers (*?, +?) exist: they flip the default, trying the smallest possible repetition first and only growing if the rest of the pattern demands it. Once the engine finds a full match, or exhausts every possibility and gives up, it packages the result — matched text, start/end offsets, and any group captures — into the Match object you receive (or returns None if no starting position ever worked).

Common Mistakes

Mistake 1: Forgetting the raw string prefix

Because \b, \d, and friends are backslash sequences, it’s tempting to write patterns as ordinary strings. The problem is that Python’s string parser processes some backslash sequences — like \b, which means “backspace” in a normal string — before the re module ever sees the pattern.

import re

text = "foo bar"

# Wrong: "\b" without a raw string is a backspace character
wrong_pattern = "\bfoo"
print("Wrong result:", re.search(wrong_pattern, text))

# Correct: use a raw string so \b means "word boundary"
right_pattern = r"\bfoo"
print("Right result:", re.search(right_pattern, text))

Output:

Wrong result: None
Right result: <re.Match object; span=(0, 3), match='foo'>

The non-raw pattern is silently turned into a single backspace character followed by “foo”, so it never matches anything in ordinary text — re.search() returns None with no error or warning, which makes this bug easy to miss. The raw-string version preserves \b as the regex word-boundary assertion, so it correctly matches “foo” at the start of the string. Always prefix pattern strings with r.

Mistake 2: Assuming quantifiers stop at the nearest delimiter

Quantifiers like + and * are greedy by default — they try to match as much text as possible before backtracking. Applied to text where a delimiter appears more than twice, greedy matching can swallow far more than intended.

import re

html = "<p>Hello</p>"

greedy = re.findall(r"<.+>", html)
print("Greedy:", greedy)

non_greedy = re.findall(r"<.+?>", html)
print("Non-greedy:", non_greedy)

Output:

Greedy: ['<p>Hello</p>']
Non-greedy: ['<p>', '</p>']

<.+> starts at the first < and greedily consumes everything up to the last > in the string, merging both tags and the text between them into a single match. Adding a ? after the quantifier (<.+?>) makes it lazy, so it stops at the first > it finds, correctly isolating each tag. As a rule of thumb, prefer lazy quantifiers whenever a delimiter can repeat in your input — or, even better, use a character class that explicitly excludes the delimiter, like <[^>]+>.

Best Practices

• Always write pattern strings as raw strings (r"...") to avoid clashes between Python’s string escapes and regex escapes.
• Call re.compile() once and reuse the resulting Pattern object when matching the same pattern repeatedly, especially inside loops.
• Prefer specific character classes ([0-9], \w) over a bare . to avoid accidentally matching characters you didn’t intend to include.
• Use non-capturing groups (?:...) when you need grouping or alternation but don’t need the captured text, keeping group numbering predictable.
• Use named groups (?P<name>...) once a pattern has more than two or three groups — match.group("level") is far more readable than match.group(3).
• Anchor with ^/$ or use re.fullmatch() when validating that an entire string matches a format, not just some substring inside it.
• Be cautious with nested or overlapping quantifiers (like (a+)+) on untrusted input — they can cause catastrophic backtracking and hang your program.
• For long, complex patterns, use re.VERBOSE so you can add whitespace and # comments inside the pattern itself.
• Don’t reach for regex to parse deeply nested or recursive structures like full HTML or JSON — use a proper parser for those instead.

Practice Exercises

• Write a function that extracts every hashtag (like #python or #100DaysOfCode) from a tweet-style string, ignoring a bare # that isn’t followed by any word characters.
• Given the string "3, 14, 159,7" (inconsistent spacing around the commas), use re.split() to turn it into the list of integers [3, 14, 159, 7].
• Write a pattern with a named group called area_code that validates and extracts the area code from a US-style phone number formatted as (555) 123-4567.

Summary

• The re module compiles pattern strings into a matching engine that scans text for matches.
• search(), match(), and fullmatch() locate a single match; findall() and finditer() enumerate all matches; sub(), subn(), and split() transform text.
• A Match object exposes .group(), .groups(), .groupdict(), .span(), .start(), and .end().
• Always use raw strings (r"...") for patterns to avoid Python pre-processing backslash escapes.
• Quantifiers are greedy by default and backtrack when needed; append ? for lazy matching.
• Named groups ((?P<name>...)) and non-capturing groups ((?:...)) keep complex patterns organized and readable.
• Compile a pattern once with re.compile() and reuse it when matching repeatedly for better performance.