Back in the word count chapter you built
word_count, char_count, and longest_word with simple for loops. Then you
used iterators to collapse those loops into one-liners. Now we'll expand on that
to write more complex text analysis tools.
Our implementation combines iterators, hash maps, and Option, as well as
max_by_key and HashMap::into_iter, which are two new iterator tools we'll
get to know in this chapter.
Both split_whitespace and split return iterators of &str. The first
handles any kind of whitespace and skips empties, which is usually what you want
for natural text:
for word in "hello world\nrust".split_whitespace() {
println!("{word}"); // hello, world, rust
}
max_by_key compares items using a value you derive from each one. Here we
choose a filename by its length rather than its alphabetical order:
let files = ["a.rs", "settings.rs", "main.rs"];
let longest = files.iter().max_by_key(|name| name.len());
assert_eq!(longest, Some(&"settings.rs"));
Add the lengths, convert the totals to f64, and only then divide so integer
truncation can't discard the fraction:
let total_chars: usize = words.iter().map(|w| w.chars().count()).sum();
let avg = total_chars as f64 / words.len() as f64;
Start by turning a string of text into a HashMap<String, usize> that records
how many times each word appears. Words are separated by whitespace and grouped
by their lowercase spelling ("Hello" and "hello" count as the same word).
Build the map by splitting on whitespace, lowercasing each piece, and bumping
its counter. The entry API handles the lookup and default insertion together,
then gives you the counter to update.
Useful from the Standard Library
str::split_whitespacesplits on any whitespace and skips empty pieces. That makes it a better default for natural text than splitting on one literal space.str::to_lowercasereturns a freshString. Use it as the map key soHelloandhellocollapse together.HashMap::entry+Entry::or_insertis the "look up; insert default; mutate" pattern you used with hashmaps.
use std::collections::HashMap;
/// Counts how many times each word appears in the text. Words are
/// whitespace-separated; counts use lowercase keys.
fn count_words(text: &str) -> HashMap<String, usize> {
let mut counts = HashMap::new();
for word in text.split_whitespace() {
*counts.entry(word.to_lowercase()).or_insert(0) += 1;
}
counts
}
#[test]
fn test_count_words() {
let text = "hello world hello rust world";
let counts = count_words(text);
assert_eq!(counts.get("hello"), Some(&2));
assert_eq!(counts.get("world"), Some(&2));
assert_eq!(counts.get("rust"), Some(&1));
}
#[test]
fn test_count_words_case_insensitive() {
let text = "Hello HELLO hello";
let counts = count_words(text);
assert_eq!(counts.get("hello"), Some(&3));
}
#[test]
fn test_count_words_unicode_and_whitespace() {
let counts = count_words("CAFÉ\tCafé\n🦀\u{2003}🦀");
assert_eq!(counts.get("café"), Some(&2));
assert_eq!(counts.get("🦀"), Some(&2));
assert_eq!(counts.len(), 2);
assert!(count_words("").is_empty());
assert!(count_words(" \t\n").is_empty());
}
Return the word with the largest count together with that count. Empty or
whitespace-only input should return None.
A working count_words is included so this step runs on its own. Reuse it and
implement only most_common_word.
Useful from the Standard Library
HashMap::into_iterconsumes the map and yields owned(K, V)pairs. In contrast,iteryields references to keys and values.Iterator::max_by_keyreturns the item with the largest derived key as anOption.
The return type owns its String. If the compiler reports a borrowed value,
check what your iterator yields and how long the map lives. You don't need to
keep the map after this function returns.
What should the comparison key measure: the word itself, or how often it appeared? Also check what your chosen iterator method returns for an empty map before adding a separate branch.
use std::collections::HashMap;
/// Counts how many times each word appears in the text. Words are
/// whitespace-separated; counts use lowercase keys.
fn count_words(text: &str) -> HashMap<String, usize> {
let mut counts = HashMap::new();
for word in text.split_whitespace() {
*counts.entry(word.to_lowercase()).or_insert(0) += 1;
}
counts
}
/// Finds the most common word in the text. Returns the word and its count, or
/// None if text is empty.
fn most_common_word(text: &str) -> Option<(String, usize)> {
count_words(text)
.into_iter()
.max_by_key(|(_, count)| *count)
}
#[test]
fn test_most_common_word() {
let text = "apple banana apple cherry apple";
let (word, count) = most_common_word(text).unwrap();
assert_eq!(word, "apple");
assert_eq!(count, 3);
assert_eq!(
most_common_word("CAFÉ\tcafé tea"),
Some(("café".to_string(), 2))
);
assert_eq!(most_common_word(""), None);
assert_eq!(most_common_word(" \t\n"), None);
}
Now you'll combine several small aggregations in one function. text_stats
returns three numbers about a piece of text: total word count, number of unique
words, and the average word length as an f64. You can compute all three from a
single pass over count_words's result, or split the work; either is fine.
Count every occurrence, including repeats: "a a bbbb" has three words, two
unique words, and an average length of 2.0. For empty or whitespace-only
input, return (0, 0, 0.0).
Use the lowercase words from count_words for both uniqueness and length.
Lengths count Unicode scalar values, not bytes or visible characters: "café"
has length 4, and "🦀" has length 1. Lowercasing can change that length: "İ"
becomes "i\u{307}" (an i and a combining dot), so its lowercase length is 2.
Words are separated by whitespace; punctuation stays attached, and no Unicode
normalization is required.
Watch the order of operations when you compute the average. Integer division
truncates, so cast to f64 before you divide. Converting an already-truncated
result won't bring the fraction back. The test compares the result against a
small tolerance because calculations with f64 can introduce rounding error.
count_words is provided here too. Reuse it and implement only text_stats.
Useful from the Standard Library
HashMap::valuesyields the occurrence counts;Iterator::sumadds them to give the total number of words.HashMap::lengives the number of unique words because each key appears once.HashMap::iteryields each word and its count.str::charscounts Unicode scalar values when combined withIterator::count;len()would count bytes.
use std::collections::HashMap;
/// Counts how many times each word appears in the text. Words are
/// whitespace-separated; counts use lowercase keys.
fn count_words(text: &str) -> HashMap<String, usize> {
let mut counts = HashMap::new();
for word in text.split_whitespace() {
*counts.entry(word.to_lowercase()).or_insert(0) += 1;
}
counts
}
/// Calculates basic text statistics. Returns (`total_words`, `unique_words`,
/// `average_word_length`). Lengths count Unicode scalar values in lowercase
/// words, including every occurrence. Empty or whitespace-only input returns
/// (0, 0, 0.0).
///
/// In real code you'd reach for a
/// `struct TextStats { total: usize, unique: usize, avg_len: f64 }` here; a
/// 3-tuple is hard to read at the call site. We're sticking with a tuple to
/// keep the focus on the iterator chain in the body.
fn text_stats(text: &str) -> (usize, usize, f64) {
let counts = count_words(text);
let total: usize = counts.values().sum();
let unique = counts.len();
let total_length: usize = counts
.iter()
.map(|(word, count)| word.chars().count() * count)
.sum();
let average_word_length = if total == 0 {
0.0
} else {
total_length as f64 / total as f64
};
(total, unique, average_word_length)
}
#[test]
fn test_text_stats() {
let text = "hello world rust";
let (total, unique, avg_len) = text_stats(text);
assert_eq!(total, 3);
assert_eq!(unique, 3);
assert!((avg_len - 4.66).abs() < 0.01); // Average length ≈ 4.66
// The expected 4.66 is only an approximation of 14/3 = 4.666..., so we
// compare the absolute difference to a tolerance. Floating-point values
// and arithmetic results can be rounded, but `==` compares stored values
// exactly; it does not allow for a small difference.
}
#[test]
fn test_text_stats_repeated_words() {
let (total, unique, avg_len) = text_stats("a A bbbb");
assert_eq!(total, 3);
assert_eq!(unique, 2);
assert!((avg_len - 2.0).abs() < 1e-10);
}
#[test]
fn test_text_stats_empty() {
assert_eq!(text_stats(""), (0, 0, 0.0));
assert_eq!(text_stats(" \t\n"), (0, 0, 0.0));
}
#[test]
fn test_text_stats_unicode() {
let (total, unique, avg_len) = text_stats("CAFÉ\tCafé\u{2003}🦀");
assert_eq!(total, 3);
assert_eq!(unique, 2);
assert!((avg_len - 3.0).abs() < 1e-10);
// Lowercase İ is i followed by a combining dot: two scalar values.
let (total, unique, avg_len) = text_stats("İ i\u{307}");
assert_eq!(total, 2);
assert_eq!(unique, 1);
assert!((avg_len - 2.0).abs() < 1e-10);
}
Your word counter now uses a HashMap keyed by lowercased words and
max_by_key to find the most common one. You used into_iter() to take
ownership of the result and combined counts and lengths into summary statistics.
What We Learned
split_whitespace()is the right default for word-splitting in natural text. It collapses runs of whitespace and skips empties.- Normalize keys as you build the map. Here, lowercasing before insertion makes
Helloandhellocontribute to the same count.into_itertransfers the keys and values out of aHashMap, which lets you return owned data without cloning it. In contrast,iteronly lends you references to entries that remain in the map.max_by_keyreturns anOption, so empty input naturally collapses toNonewithout a special-case branch.- Watch the integer-division trap when computing averages: divide after casting to
f64, not before. Tests for calculatedf64values usually compare a tolerance such as(a - b).abs() < epsinstead of using==.- Tuples like
(usize, usize, f64)work for tiny ad-hoc returns, but a named struct (TextStats { total, unique, avg_len }) reads better at the call site as the function grows.
You now have every tool you need to build a small program from scratch: structs,
enums, iterators, Option, Result, vectors, and strings. Want to put them
together? Build a password validator with your own
rules in the optional Creative Break project. Or try smart
pointers to learn how Box, Rc, and RefCell let you
store and share data. Both chapters are optional and do not count toward course
progress. You can try them now or keep going.
Extra practice to explore at your own pace. These chapters do not count toward course progress.