WXR Import Fails With `PCDATA invalid Char value 18` and `Sequence ']]>' not allowed`? Sanitize the Bytes First

Migrating a client site from an old WordPress install to a fresh instance looked routine: grab the WXR export, run the importer, wait. The export was 10MB and held 1758 items: posts, pages, and attachments. I uploaded it, clicked run, and the importer died in the first second. Not a timeout, not a white screen, but the XML parser threw before a single post made it in.

The log was full of red lines from libxml:

PCDATA invalid Char value 18
PCDATA invalid Char value 27
Sequence ']]>' not allowed in content

Two different families of error in one file. The first fired repeatedly for char values in the 18 to 29 range. The second showed up only once, but that was enough to bring the whole parse down.

Dead ends first

My first reflex was wrong. I assumed the file got corrupted in transfer, so I re-exported it from the old site. Identical result. I opened the 10MB file in an editor and scanned it: thousands of tidy <item> lines, all normal. My eyes were never going to find the bad byte; it is not something you can see. I also bumped memory_limit, guessing the parser ran out of memory. Not that either. The parser was healthy and doing its job: rejecting XML that really was invalid.

Why the parser refused

Two distinct root causes, and both come from how WordPress content gets authored.

First, control bytes. The editor stores whatever an author pastes, and content from Word, a PDF, or a terminal often carries raw control characters in the 0x00 to 0x1F range. XML 1.0 permits only three of them inside content: tab (0x09), line feed (0x0A), and carriage return (0x0D). The rest are illegal, and a compliant parser like the libxml behind SimpleXML and DOMDocument must reject them. That is PCDATA invalid Char value 18: a single 0x12 byte tucked into the body of some post.

Second, the ]]> sequence. WXR wraps post content in CDATA using <![CDATA[ ... ]]>, and ]]> may appear only as the terminator. But if an author ever pasted a snippet that happened to contain ]]> mid-body, that string closes the CDATA early and the rest becomes loose PCDATA. The parser sees a bare ]]> outside any CDATA and throws Sequence ']]>' not allowed in content.

Sanitize the raw bytes before parsing

The key move: clean the XML as a byte string first, before handing it to any parser. Not patching the parser, but feeding it a valid file.

function wxr_sanitize( $xml ) {
    // 1. Drop a leading UTF-8 BOM if present.
    $xml = preg_replace( '/^\xEF\xBB\xBF/', '', $xml );
 
    // 2. Strip illegal control bytes, keep tab, LF, CR.
    $xml = preg_replace( '/[\x00-\x08\x0B\x0C\x0E-\x1F]/', '', $xml );
 
    // 3. Escape stray ]]> that is not a legitimate CDATA close.
    //    In WXR, a real close is always immediately followed by </tag.
    $xml = preg_replace( '/\]\]>(?!\s*<\/)/', ']]&gt;', $xml );
 
    return $xml;
}

Three steps for three sources of trouble. A BOM at the head confuses the parser about encoding, so it goes first. The second regex removes exactly the bytes libxml complained about while keeping the legal tab, LF, and CR. The third is a heuristic that fits WXR: a valid CDATA close here always looks like ]]></tag>, so a ]]> not followed by </ is almost certainly a stray sequence in content, and I turn it into the safe entity ]]>.

After sanitizing I compared the size: out of 10MB, only 33 bytes were removed. Thirty-three invisible bytes that took down 1758 items. The parser went straight through and every post landed.

The second wall: a 524 on attachment downloads

The posts were in, but the importer was not done. There were still 465 attachments to download from the old site and register in the media library, and my first version did all of that in one request. About a hundred seconds later, up came Cloudflare's 524 A Timeout Occurred page: my origin was answering too slowly and the proxy cut the connection at 100 seconds. Raising max_execution_time did nothing, because the killer was not PHP but the proxy in front of it.

The fix was not a bigger timeout but a different shape of work. I split the importer into a phased state machine with progress saved in WP options, and the decisive part was a time-based kill switch. Instead of trusting a fixed item count, each request sets a 50-second budget and stops itself well before Cloudflare steps in.

define( 'WXR_TIME_BUDGET', 50 ); // seconds, half of Cloudflare's limit
 
function wxr_run_phase() {
    $start = microtime( true );
    $state = get_option( 'wxr_state', array( 'phase' => 'attachments', 'offset' => 0 ) );
 
    foreach ( wxr_pending_attachments( $state['offset'] ) as $item ) {
        // Bail before the proxy kills us.
        if ( microtime( true ) - $start > WXR_TIME_BUDGET ) {
            break;
        }
        wxr_sideload_attachment( $item );
        $state['offset']++;
        update_option( 'wxr_state', $state, false );
    }
 
    return $state['offset'] < wxr_total_attachments();
}

What sets this apart from a fixed-size batch: the kill switch measures real elapsed time instead of guessing how many items fit. A 40MB attachment and a 4KB one take very different amounts of time, and a time budget adapts on its own. State is saved after every item, so if anything dies mid-run, the next request resumes from the last offset.

Between batches, no JavaScript and no cron: a plain HTML meta-refresh re-calls the importer URL until the phase finishes.

if ( wxr_run_phase() ) {
    $next = admin_url( 'admin.php?page=wxr-import&resume=1' );
    echo '<meta http-equiv="refresh" content="1;url=' . esc_url( $next ) . '">';
    exit;
}

That worked out to about 22 automatic page loads, each finishing in a dozen seconds, and all 465 attachments landed without a single 524.

Checklist

• Never assume XML from a third-party export is valid. Sanitize it as a byte string first, then parse.
• PCDATA invalid Char value N means an illegal control byte. Strip 0x00-0x08, 0x0B, 0x0C, 0x0E-0x1F, keep tab, LF, CR.
• Sequence ']]>' not allowed means a stray ]]> in content. In WXR, escape any ]]> not followed by </ into ]]>.
• Compare the file size before and after sanitizing. 33 bytes out of 10MB was enough to break the whole import.
• For bulk downloads behind Cloudflare, use a time-based kill switch (say 50 seconds), not a fixed item count. A slow origin always loses to the proxy's 100-second limit.
• Save state after each item and resume via meta-refresh. No JS, no cron.