Text Processing and Regular Expressions

By the end of this lesson you'll be able to:

• Write basic regular expressions to match patterns in text
• Use re.search(), re.findall(), re.split(), and re.sub() to search and transform text
• Build patterns using character classes, quantifiers, anchors, and groups
• Use raw strings (r"...") to avoid backslash confusion

Regular expressions (regex) are a mini-language for describing text patterns. With a single regex, you can find phone numbers, validate email addresses, or extract data from thousands of lines of text.

Welcome to Chapter 29!

Regular expressions look scary at first — but once you learn the building blocks, they're incredibly powerful. One line of regex can do what would take 20 lines of string methods. Let's decode the mystery! Let's code it together!

The re Module

Import Python's regex module with import re.

The core functions are:

Function	Returns	Use when
re.search(pattern, text)	Match object or None	Check if pattern appears anywhere
re.findall(pattern, text)	List of all matches	Get every occurrence
re.split(pattern, text)	List of pieces	Split text by a pattern
re.sub(pattern, replacement, text)	New string	Replace matches

Raw Strings

Regex patterns use backslashes a lot — \d, \w, \s. In regular Python strings, \d is treated as a special escape sequence. To avoid confusion, always write regex patterns as raw strings by putting r before the quote: r"\d".

# Without raw string: Python interprets \n as newline
pattern = "\n"   # newline character

# With raw string: Python passes \n literally to the regex engine
pattern = r"\n"  # the two characters backslash + n

Rule: always use r"..." for regex patterns.

Basic Pattern Elements

Before we write real patterns, here are the building blocks:

Pattern	Matches
.	Any single character (except newline)
\d	A digit (0–9)
\w	A word character (letter, digit, underscore)
\s	A whitespace character (space, tab, newline)
\D	Not a digit
\W	Not a word character
\S	Not whitespace
^	Start of the string
$	End of the string

Quantifiers

Quantifiers control how many times a pattern element repeats:

Quantifier	Meaning	Example
*	0 or more times	\d* — zero or more digits
+	1 or more times	\d+ — one or more digits
?	0 or 1 times	\d? — optional digit
{n}	Exactly n times	\d{3} — exactly 3 digits
{n,m}	Between n and m times	\d{2,4} — 2 to 4 digits

re.search() — Find First Match

re.search(pattern, text) scans through text looking for the first location where the pattern matches. It returns a match object if found, or None if not.

What Do You Think Will Happen?

The code below searches for a phone number pattern in different strings. Which strings do you think will match the pattern \d{3}-\d{4}? Make your guess — then run it!

▶ Run ↻ Reset

match.group() returns the text that was matched.

re.findall() — Find All Matches

re.findall(pattern, text) returns a list of every non-overlapping match.

▶ Run ↻ Reset

See It: The Regex Match Lab

What Do You Think Will Happen?

The lab below starts with the pattern \d{3}-\d{4}. Read the chips under the pattern box, then predict: which pieces of the sample text will light up — and how many matches will findall report?

Explore the Regex Match Lab MicroSim

Now experiment: delete the - from the pattern and watch the matches change instantly. Try each preset button, then build your own pattern one character at a time. The chip strip always translates your pattern into plain English — regex stops being magic once you can read it aloud.

Character Classes

A character class is a set of characters in square brackets. It matches any single character in the set.

Pattern	Matches
[aeiou]	Any lowercase vowel
[A-Z]	Any uppercase letter
[a-z]	Any lowercase letter
[0-9]	Any digit (same as \d)
[^aeiou]	Any character that is NOT a vowel

re.split() — Split by Pattern

re.split(pattern, text) splits text wherever the pattern matches — like str.split() but with regex power.

▶ Run ↻ Reset

re.sub() — Substitute Matches

re.sub(pattern, replacement, text) replaces every match with the replacement string.

import re

# Remove all digits from a string:
cleaned = re.sub(r"\d", "", "Hello123World456")
print(cleaned)   # "HelloWorld"

# Replace multiple spaces with one:
normalized = re.sub(r"\s+", " ", "too   many    spaces")
print(normalized)   # "too many spaces"

Anchors: Start and End

^ matches the start of the string. $ matches the end.

# Matches only if string starts with "Hello":
if re.search(r"^Hello", text):
    print("Starts with Hello")

# Matches only if string ends with a digit:
if re.search(r"\d$", text):
    print("Ends with a digit")

Alternation with |

| means "or" — match either the left or the right pattern.

import re

pattern = r"cat|dog|rabbit"
matches = re.findall(pattern, "I have a cat, a dog, and a rabbit.")
print(matches)   # ['cat', 'dog', 'rabbit']

Groups with Parentheses

Parentheses () create a group — they let you capture specific parts of a match.

import re

text = "Born: 2010-06-15"
match = re.search(r"(\d{4})-(\d{2})-(\d{2})", text)
if match:
    year, month, day = match.group(1), match.group(2), match.group(3)
    print(f"Year={year}, Month={month}, Day={day}")

▶ Run ↻ Reset

Escaping Special Characters

Some characters have special meaning in regex: ., *, +, ?, (, ), [, ], {, }, ^, $, |, \. To match them literally, escape them with a backslash: \. matches a literal dot.

# Match a literal dot (not "any character"):
re.findall(r"\d+\.\d+", "3.14 and 2.71 and hello.world")
# → ['3.14', '2.71']

Compiled Regex Patterns

If you use the same pattern many times, compile it first for better performance:

import re

digit_pattern = re.compile(r"\d+")   # compile once

for text in many_strings:
    matches = digit_pattern.findall(text)   # use repeatedly

Learning Check

Your Turn — Extract Phone Numbers

The code below should extract all phone numbers from the text. The pattern only matches 3-digit numbers. Fix the pattern to match the full format NNN-NNN-NNNN!

▶ Run ↻ Reset

Experiments

Try these changes. Predict what will happen first, then run it to check!

1. Use re.findall(r"\b\w{5}\b", text) to find all 5-letter words in a sentence. \b is a word boundary. You'll know it worked when you see only 5-letter words.

2. Use re.sub(r"\b\d+\b", "NUM", "I have 3 cats and 12 birds") to replace all numbers with "NUM". You'll know it worked when you see "I have NUM cats and NUM birds".

3. Validate a password: at least 8 characters, at least one digit. Use re.search(r"^.{8,}$", pw) and re.search(r"\d", pw). You'll know it worked when "abc12345" passes and "abc123" fails (too short).

4. Use re.split(r"\s*,\s*", "Alice , Bob, Carol, Dave") to split a comma-separated list ignoring extra spaces. You'll know it worked when you see four clean names.

5. Compile the pattern r"\d{4}" and use it to find all 4-digit years in a sentence about history. You'll know it worked when you see a list of years.

Regex Wizard!

Regular expressions are a superpower — one pattern can search through millions of lines of text! They appear in web scraping, data validation, log analysis, and almost every text processing task. You've just added a serious tool to your programmer toolbox. Let's keep coding!

Take the Chapter Review Quiz

See Annotated References