Skip to content

Add streaming (constant-memory) Xlsx writer - #4966

Open
kemo wants to merge 56 commits into
PHPOffice:masterfrom
kemo:streaming-writer
Open

kemo wants to merge 56 commits into
PHPOffice:masterfrom
kemo:streaming-writer

Conversation

@kemo

@kemo kemo commented Aug 18, 2026 •

Copy link
Copy Markdown
Contributor

This is:

  • a bugfix
  • a new feature
  • refactoring
  • additional unit tests

Checklist:

  • Changes are covered by unit tests
    • Changes are covered by existing unit tests
    • New unit tests have been added
  • Code style is respected
  • Commit message explains why the change is made (see https://github.com/erlang/otp/wiki/Writing-good-commit-messages)
  • CHANGELOG.md contains a short summary of the change and a link to the pull request if applicable
  • Documentation is updated as necessary

Why this change is needed?

PhpSpreadsheet holds every cell as an object (~1KB+ each), so writing large files exhausts memory and users switch to OpenSpout for big exports. The PSR-16 cell cache reduces resident memory but keeps O(n) storage and is slow; setUseDiskCaching() only helps the writer-side XML buffers. There was no way to write a large Xlsx file with memory independent of row count.

This PR adds an append-only streaming Xlsx writer with constant memory in row count, as an additive feature:

use PhpOffice\PhpSpreadsheet\Writer\Xlsx\Streaming\StreamingWriter;

$writer = new StreamingWriter('big.xlsx');
$bold = $writer->registerStyle(['font' => ['bold' => true]]);
$sheet = $writer->startSheet('Data');
$sheet->appendRow(['Name', 'Amount', 'Date'], $bold);
$sheet->appendRow(['Widget', 12.5, new DateTimeImmutable('2026-01-01')]);
$writer->close();

How it works

  • appendRow() serializes each row immediately to a php://temp/maxmemory stream via XMLWriter; strings are written as inline strings, so no shared-strings table accumulates.
  • All workbook-level parts (ContentTypes, rels, docProps, theme, styles, workbook) are produced by the existing Writer\Xlsx part writers running against an internal shell Spreadsheet that carries only sheet names and registered styles — no duplicated XML logic, and no changes to any existing class.
  • close() streams each finished sheet into the archive with ZipStream::addFileFromStream().

Benchmark (200,000 rows × 8 mixed columns, medians of 3 isolated processes)

ZipStream 3.1.1, Apple Silicon (Darwin 23.1.0 arm64).

PHP 8.2.27

Engine Wall time Peak memory File size
Standard Spreadsheet + Writer\Xlsx 31.43 s 1,091.06 MB 9,173,721 bytes
StreamingWriter 5.09 s 42.27 MB 8,118,639 bytes
Ratio ~6.2x faster ~25.8x less memory ~1.13x smaller

PHP 8.5.10

Engine Wall time Peak memory File size
Standard Spreadsheet + Writer\Xlsx 33.83 s 1,093.06 MB 9,173,722 bytes
StreamingWriter 5.28 s 42.27 MB 8,118,639 bytes
Ratio ~6.4x faster ~25.9x less memory ~1.13x smaller

Peak memory is flat past ~100k rows: 100k and 200k rows both peak at ~42 MB inside the writer (most of that is ZipStream's fixed 16 MB read block, not row data). The benchmark script is committed under tests/Benchmark/ (plain PHP, outside the phpunit test path) with the recorded results in its README, so the numbers are reproducible: php -d memory_limit=4G tests/Benchmark/bench.php <standard|streaming> [rows].

Scope

Supported: scalars, formulas (written without cached values, fullCalcOnLoad set), DateTimeInterface (default date format unless styled), per-row and per-cell styles via registerStyle()/StreamedCell, column widths, freeze pane, autofilter over the written range, multiple sequential sheets.

Not supported (documented): random access, reading back, merged cells, charts, drawings, comments, conditional formatting, hyperlinks, rich text. One caveat is documented explicitly: PhpSpreadsheet's own reader returns inline-string cells as RichText objects; Excel and other consumers read them as plain strings.

Testing

  • 30 unit tests / 80 assertions: round-trips through the real Reader\Xlsx for every value type, style, and sheet feature; lifecycle guard tests (append after close, stale sheets, invalid widths, unsupported forced types, failure cleanup); a corrupted-sheet guard (a failed appendRow() poisons the sheet so close() throws instead of writing malformed XML); and a memory test asserting the peak at 200k rows does not exceed the peak at 100k rows.
  • Full suite, PHPStan (level 10, no new errors), PHP-CS-Fixer, and PHPCS are clean.
  • New docs page: docs/topics/streaming-writer.md.

kemo added 30 commits August 18, 2026 11:43
close() forced fullCalcOnLoad/forceFullCalc unconditionally regardless of
whether any formula was written. Tie it to hasFormulas instead.

Also clean up on failure: open the output file in 'wb' (was 'wb+'), track
its path, and on a zero-sheet close(), a close() that throws mid-write, or
destruction without a close(), fclose the handle(s) and unlink the partial
output file instead of leaving a truncated file behind.
If appendRow() throws mid-row (unsupported value, bad style id, etc.) the
XMLWriter buffer was left holding an unclosed <row>; the next appendRow()
call would nest a new row inside it, silently corrupting the sheet.

Catch any Throwable, discard the buffered XML, and mark the sheet broken.
Once broken, assertUsable() rejects further appendRow() and finish()
calls with a Writer\Exception instead of writing the file as if nothing
happened.

Also reject an unsupported StreamedCell $dataType (only TYPE_STRING and
TYPE_STRING2 are supported) and invalid setColumnWidths() input (column
numbers below 1, non-positive widths), both of which were silently
accepted before and produced invalid or ignored output.
Covers: calcPr reflecting formula presence, a multi-sheet round trip with
distinct data per sheet, rejection of invalid setColumnWidths() input and
unsupported StreamedCell data types, the broken-sheet state left by a
failed appendRow(), and file cleanup on destruction without close() and on
a close() that fails.
- StreamedCell cannot force a date value to a string; is_scalar() rejects it.
- A lone "=" (length 1) is written as a plain string, not a formula.
- setAutoFilterToWrittenRange() may be called any time before the sheet
  finishes, not only before the first appendRow() like column widths and
  the freeze pane.
- freezePane('A1') is a no-op.
- Each finished sheet keeps up to ~2MB of buffered XML in memory until
  close(), so memory at close() scales with sheet count, not row count.
Comment thread src/PhpSpreadsheet/Writer/Xlsx/Streaming/StreamingWriter.php Outdated
Comment thread docs/topics/streaming-writer.md Outdated
```

Because the workbook contains a formula, `close()` marks the workbook for
full calculation on load, so Excel (or another spreadsheet application)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not see where close marks the workbook for full calculation. Please enlighten me.

@kemo kemo Aug 26, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment* updated 729d1b0

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't see where that attribute is set in your code. Please explain.

More to the point, 'forceFullCalc` is a very dangerous option. Please see PR #4271 for details on its unexpected side effects, and why we would not want to make it an "automatic" choice.

Comment thread src/PhpSpreadsheet/Writer/Xlsx/Streaming/StreamingWriter.php Outdated
kemo added 18 commits August 26, 2026 13:55
Check every header, row and footer write so exhausted temporary storage cannot silently drop data. Retry partial writes and reject streams that make no progress.
Pass the shell calendar explicitly so loading a 1904 workbook cannot shift dates in a later streaming export by 1462 days.
Reject rows and columns beyond the Xlsx limits without consuming a row. Validate finite column widths as a complete request so invalid options leave previous widths intact.
Valid UTF-8 can still contain characters forbidden in XML. Reject those formulas before serializing them while preserving tabs, newlines and Unicode text.
Request recalculation on load for uncached formulas while keeping the standard writer default for forceFullCalc. Formula presence should not change calculation behavior in other open workbooks.
Emit empty styled cells for null values so row fills and borders remain continuous. Keep unstyled blanks omitted and allow an explicit default style to clear the row style.
Stage local output beside its destination and rename only after the archive is closed successfully. Add explicit abort cleanup and release active sheet streams after failures as well as finished sheets.
Index registered styles by their existing Style hash, including the default style. Equivalent registrations now keep stable IDs without growing the cell XF collection.
Preserve false values and write numeric dates with matching formats in both engines. Add a small round-trip comparison and record runtime versions with each measurement.
Run each row count in a fresh benchmark process so warmed caches and retained allocator pages cannot hide growth. Include archive assembly and verify the child process exits successfully.
Apply the broken-sheet guard when a write throws outside appendRow as well. An error handler exception while finishing an empty sheet must not leave a partial header reusable.
Document shell metadata, temporary disk usage and abort semantics. Replace the previous benchmark figures with measurements of equivalent workbooks on the corrected implementation.
Non-POSIX filesystems (FAT, some SMB/NFS mounts) return false from
chmod while the file is intact.  A fatal throw here routes through
the catch block, which deletes the completed temporary file and
loses the export.  Suppress the error instead so the rename proceeds.
PHP 8.5 raises a fatal error when NAN is interpolated into a
string. Format NAN, INF and -INF explicitly before building the
exception message.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants