Strings & regex

Why strlen and mb_strlen disagree about one string

The mistake

You picture a string as a row of characters, and strlen as the count of them. For 'Laravel' that is exactly right: seven characters, seven bytes, and every string function does what it looks like it does.

A PHP string is not a row of characters. It is a row of bytes. The two only look the same while the text stays inside the English alphabet, which is why this bug never shows up in your tests and always shows up in production, the first time someone types their own name. From that moment there is no single answer to “how long is this string”. There are three, they are all correct, and PHP has a different function for each.

The machine

Simulator · string length

string

Every count and every cut runs on the tested reducer, checked against PHP 8.4 by npm run verify:php.

substr($s, 0, 5). The result is café. The whole string, 5 bytes.

Drive it

The panel starts on 'Laravel', where all three counts agree. Every scenario begins by leaving it.

Switch to the accented string and drag the cut to four bytes. substr stops in the middle of the é. What is left is not valid UTF-8, so the text becomes a replacement box and json_encode returns false instead of a string.
Switch to mb_substr, then pick the composed string. It looks identical to the one before it. Cut it at four and the encoding stays valid, and the accent quietly disappears. Nothing errors, nothing warns.
Pick the family emoji and read the three counts. Eighteen bytes, five code points, one picture. Cut it at two code points and you keep half a family and a dangling joiner.

The mechanism

UTF-8 stores one code point in one to four bytes. Anything in the original ASCII range keeps its single byte, which is what makes UTF-8 safe to adopt. Above that, the first byte announces how many bytes follow, and each of those continuation bytes is marked as a continuation. So é is C3 A9: a lead byte that says “two bytes”, then one that says “I am the rest of it”. The byte ruler in the panel draws continuation bytes with a dashed edge, because on their own they mean nothing.

strlen counts those bytes. It does not decode anything, which is why it is fast and why it answers a question you probably were not asking. mb_strlen decodes, and counts code points. For café that is 4 against 5, and the 4 is the number you meant.

Then the second surprise. A code point is still not a character. é can be written two ways: one code point U+00E9, or the letter e followed by a combining acute accent, U+0301. Both render identically. The second is two code points, so mb_strlen says 5 where your eye says 4. Emoji go further: the family in the panel is three people joined by two zero width joiners, five code points that draw one picture. What a reader calls a character is a grapheme cluster, and grapheme_strlen is the function that counts those.

Cutting fails in two different ways, and the difference matters more than the counts do.

substr cuts between bytes. When the cut lands inside a code point, the result is not valid UTF-8. That is a loud failure and you should be glad of it: json_encode refuses the string and returns false, most output shows a replacement box, and a strict database column rejects the write.

mb_substr cuts between code points, so it can never do that. It can still cut a combining accent off the letter it belongs to, or split a joined emoji, and that failure is silent. The string is valid, it renders, it saves. It just says something slightly different from what it said before. grapheme_substr is the only one of the three that cuts where a reader would say a character ends.

In your code

Truncating text for a teaser is where this reaches most people:

$teaser = substr($post->body, 0, 120);           // can produce invalid UTF-8
$teaser = mb_substr($post->body, 0, 120);        // always valid, can still eat an accent
$teaser = grapheme_substr($post->body, 0, 120);  // 120 characters, as a reader means it

The same split runs through the rest of the string functions. strtoupper('café') returns CAFé, because it only knows how to upper case bytes it recognises, and mb_strtoupper returns CAFÉ. str_pad pads to a byte count, so a column of accented words does not line up; mb_str_pad, added in PHP 8.3, does. In a pattern, /./ matches one byte and /./u matches one code point, which is what the u modifier is for.

If you reach for a framework helper to truncate user text, check which family it calls before you trust it.

The fine print

The mb_* functions need the mbstring extension and the grapheme_* functions need intl. Neither is guaranteed to be installed. Check before you depend on grapheme_substr in a package other people will run.
Everything here assumes UTF-8, which is what you want and not what you always get. In another encoding the byte patterns are different, and mb_* needs to be told which one.
Because é has two spellings, two strings that look identical can compare as different with ===. The fix is to normalise them first, with normalizer_normalize from intl. That is a page of its own and this one leaves it alone.
Grapheme clusters are defined by a Unicode table that changes between versions, so a very new emoji can count differently on two servers running different ICU builds. The five strings in the panel are old enough to be stable, and npm run verify:php re-measures them against a real PHP on every run.
A grapheme is still not a column of terminal width. Some characters are drawn double width and some are drawn as nothing at all. If you are aligning text in a terminal, that is a fourth question with a fourth answer.