From Full Loads to Incremental Processing: Computing Changes Is Not Enough

A full load is often a pipeline’s first strategy because it simplifies the problem: on every run, the system reads all source data, applies the transformation, and replaces the destination. It does not need to track where the previous run stopped or interpret each update and deletion separately.

That local simplicity stops scaling when it is repeated across thousands of pipelines. The platform keeps rereading, recomputing, and rewriting information it already knows. Moving to incremental processing, however, requires more than reading fewer rows.

The decision depends on how the information changes, how the transformation responds to those changes, and what guarantees consumers expect from the result. This combination determines whether the pipeline can simply accumulate facts or must preserve information between runs, correct outputs, establish completeness, or recompute.

1. Why full loads are so common

Consider purchases made with a credit card whose credit limit is R$1,000. Each purchase has a stable identifier, an amount, and a status. For now, the pipeline sums purchases that are confirmed and have not been reversed. This total reduces the available credit, which indicates how much can still be spent, and determines the amount on the open statement, which indicates how much will be charged in the billing cycle.

On the first run, the source contains a single record:

Purchase	Current amount	Status
`C-42`	R$100	confirmed

The purchase immediately affects two results:

available credit becomes R$900;
the open statement becomes R$100.

Before the next run, the amount of C-42 is corrected to R$120, and a new R$80 purchase, C-43, appears:

Purchase	Current amount	Status
`C-42`	R$120	confirmed
`C-43`	R$80	confirmed

Changes at the source and pipeline runs occur at different times:

%%{init: {
  "theme": "base",
  "themeVariables": {
    "background": "#F8FAFC",
    "primaryColor": "#FFFFFF",
    "primaryTextColor": "#0F172A",
    "primaryBorderColor": "#E2E8F0",
    "secondaryColor": "#F1F5F9",
    "tertiaryColor": "#FFE0F0",
    "lineColor": "#5A6B7D",
    "titleColor": "#0F172A",
    "clusterBkg": "#F1F5F9",
    "clusterBorder": "#E2E8F0",
    "actorBkg": "#FFFFFF",
    "actorBorder": "#E2E8F0",
    "actorTextColor": "#0F172A",
    "actorLineColor": "#CBD5E1",
    "signalColor": "#D6006F",
    "signalTextColor": "#0F172A",
    "labelBoxBkgColor": "#F1F5F9",
    "labelBoxBorderColor": "#E2E8F0",
    "labelTextColor": "#0F172A",
    "loopTextColor": "#0F172A",
    "noteBkgColor": "#F1F5F9",
    "noteBorderColor": "#CBD5E1",
    "noteTextColor": "#0F172A",
    "activationBkgColor": "#E3F6F9",
    "activationBorderColor": "#007C91",
    "sequenceNumberColor": "#D6006F",
    "arrowheadColor": "#D6006F",
    "fontFamily": "Inter, Segoe UI, Helvetica Neue, Arial, sans-serif",
    "fontSize": "15px"
  },
  "flowchart": {"curve": "basis", "nodeSpacing": 28, "rankSpacing": 36, "padding": 8}
}}%%
sequenceDiagram
    participant O as Purchases at the source
    participant A as Analytics pipeline

    Note over O: C-42 · R$100 · confirmed
    A->>O: Read all data (Full load)
    Note over A: Available credit · R$900<br/>Open statement · R$100
    Note over O: C-42 · corrected to R$120
    Note over O: C-43 · R$80 · confirmed
    A->>O: Read all data (Full load)
    Note over A: Available credit · R$800<br/>Open statement · R$200

On both runs, the pipeline repeats the same procedure:

read every purchase in the billing cycle;
select those that are confirmed and have not been reversed;
sum their amounts;
derive the statement and available credit from that total.

On the second run, the pipeline sums the corrected amount of C-42 and the new purchase C-43, reaching R$200. The statement is set to this total, and available credit becomes R$1,000 - R$200 = R$800.

This reconstruction is the key simplification: each run derives both results from all the data available from the source at that moment. It assumes the source provides a coherent version for reading; a full load does not create that guarantee by itself.

Even so, a full load is a reasonable choice when rereading a small volume is inexpensive, runs are infrequent, or the destination must be built for the first time.

The two strategies build the new result differently:

full load: computes it again from all data read during the current run; the previous run’s result does not participate in the calculation;
incremental: starts from the previous result and applies only the changes received since then.

Incremental processing must therefore reuse previous results and preserve the information required to interpret each change correctly.

2. How incremental processing updates a result

After the first load, the pipeline starts with a statement of R$100 and R$900 in available credit instead of running another full load. It also preserves the current record for C-42 in a structure called keyed state.

This structure stores only the information required to interpret the next change; it is not the computed total and does not need to copy every column from the source.

When the amount of C-42 is corrected to R$120, the key lets the pipeline locate the previous purchase, replace its amount, and compute the effect of the correction:

%%{init: {
  "theme": "base",
  "themeVariables": {
    "background": "#F8FAFC",
    "primaryColor": "#FFFFFF",
    "primaryTextColor": "#0F172A",
    "primaryBorderColor": "#E2E8F0",
    "secondaryColor": "#F1F5F9",
    "tertiaryColor": "#FFE0F0",
    "lineColor": "#5A6B7D",
    "titleColor": "#0F172A",
    "clusterBkg": "#F1F5F9",
    "clusterBorder": "#E2E8F0",
    "actorBkg": "#FFFFFF",
    "actorBorder": "#E2E8F0",
    "actorTextColor": "#0F172A",
    "actorLineColor": "#CBD5E1",
    "signalColor": "#D6006F",
    "signalTextColor": "#0F172A",
    "labelBoxBkgColor": "#F1F5F9",
    "labelBoxBorderColor": "#E2E8F0",
    "labelTextColor": "#0F172A",
    "loopTextColor": "#0F172A",
    "noteBkgColor": "#F1F5F9",
    "noteBorderColor": "#CBD5E1",
    "noteTextColor": "#0F172A",
    "activationBkgColor": "#E3F6F9",
    "activationBorderColor": "#007C91",
    "sequenceNumberColor": "#D6006F",
    "arrowheadColor": "#D6006F",
    "fontFamily": "Inter, Segoe UI, Helvetica Neue, Arial, sans-serif",
    "fontSize": "15px"
  },
  "flowchart": {"curve": "basis", "nodeSpacing": 28, "rankSpacing": 36, "padding": 8}
}}%%
flowchart LR
    classDef active fill:#FFE0F0,stroke:#D6006F,color:#0F172A,stroke-width:2px;
    classDef muted fill:#FFFFFF,stroke:#E2E8F0,color:#5A6B7D,stroke-width:1px;
    classDef support fill:#E3F6F9,stroke:#007C91,color:#0F172A,stroke-width:1.5px,stroke-dasharray:4 2;
    classDef warning fill:#FFF4D6,stroke:#9A5B00,color:#0F172A,stroke-width:2px;

    D["<b>Change received</b><br/>Purchase: C-42<br/>Current amount: R$120"] --> Q["Locate and update the <b>keyed state</b>"]
    S["<b>Keyed state</b><br/>Purchase: C-42<br/>Current amount: R$100<br/>Status: confirmed"] --> Q
    Q --> S2["<b>Keyed state</b><br/>Purchase: C-42<br/>Current amount: R$120<br/>Status: confirmed"]
    Q --> DV["<b>Change in committed amount (ΔV)</b><br/>− R$100 + R$120 = + R$20"]
    DV --> V["<b>Updated results</b><br/>Statement: R$100 → R$120<br/>Available credit: R$900 → R$880"]
    class Q active;
    class D,DV muted;
    class S,S2 support;
    class V active;

This process has two steps.

First: compute the change in the result

To correct C-42, the previous amount must be removed and the new one added:

ΔV = - R$100 + R$120 = + R$20

ΔV represents the resulting change in the committed amount. The correction provides the new amount, while the keyed state provides the previous one. The updated state remains available for future changes; ΔV describes only the effect of the current correction.

Then: apply the change to the existing result

Only after obtaining ΔV does the pipeline update its stored results. Because the correction added R$20 to the committed amount, it increases the statement and reduces available credit by the same amount:

statement: R$100 + R$20 = R$120
available credit: R$900 - R$20 = R$880

The previous results must therefore remain available between updates. While the statement is open, each ΔV changes the two results in opposite directions:

Change received	`ΔV`	Available credit after the change	Statement after the change
`C-42` confirmed for R$100	`+ R$100`	R$900	R$100
`C-42` corrected to R$120	`+ R$20`	R$880	R$120
`C-43` confirmed for R$80	`+ R$80`	R$800	R$200
`C-42` reversed	`- R$120`	R$920	R$80

After the statement closes, it no longer accepts new purchases for that billing cycle; a payment, in turn, releases available credit without changing the issued statement.

In database literature, updating the result of a query without recomputing it from scratch is called incremental view maintenance. The survey by Gupta and Mumick shows that the strategy depends on the query, the type of change, and the information available. DBSP provides a modern formulation in which operators preserve information between updates, receive changes from the input, and produce changes in the output.

The cost of an update depends on the aggregation

Before C-42 is reversed, the keyed state is:

Purchase	Current amount	Status
`C-42`	R$120	confirmed
`C-43`	R$80	confirmed

These purchases produce SUM = R$200 and MAX = R$120, the largest purchase in the billing cycle. When C-42 is reversed, the pipeline locates its row directly and finds that R$120 no longer contributes to either aggregation.

`SUM`: update the previous result

The stored result of SUM is R$200. After locating C-42, the pipeline needs only to remove its amount:

R$200 - R$120 = R$80

With direct access to the purchase and the previous result, this update runs in expected constant time, O(1). It does not need to scan the remaining keys.

`MAX`: find a replacement result

The stored result of MAX is R$120. Because that amount belonged to C-42, the reversal removes the current maximum. The previous result does not indicate which amount should replace it.

Using only the keyed state, the pipeline scans the purchases that are confirmed and have not been reversed, finds R$80 in C-43, and replaces the result of MAX. In the worst case, this search visits every row and costs O(n).

Both computations use the same keyed state and keep only the aggregation result. The difference lies in the operation: SUM derives the new total from the previous value and the purchase that was removed; when MAX loses the current maximum, it must search for the next candidate.

DBSP assumes that the change received is much smaller than the database. When almost everything changes, or updating an aggregation requires scanning the entire keyed state, the advantage over a full load decreases. In that case, a full load may be the simpler choice.

3. When a change lacks sufficient context

An isolated change does not always contain enough information to update the result. C-45 first appears with a pending status and an unknown amount, and can affect available credit and the statement only after confirmation:

%%{init: {
  "theme": "base",
  "themeVariables": {
    "background": "#F8FAFC",
    "primaryColor": "#FFFFFF",
    "primaryTextColor": "#0F172A",
    "primaryBorderColor": "#E2E8F0",
    "secondaryColor": "#F1F5F9",
    "tertiaryColor": "#FFE0F0",
    "lineColor": "#5A6B7D",
    "titleColor": "#0F172A",
    "clusterBkg": "#F1F5F9",
    "clusterBorder": "#E2E8F0",
    "actorBkg": "#FFFFFF",
    "actorBorder": "#E2E8F0",
    "actorTextColor": "#0F172A",
    "actorLineColor": "#CBD5E1",
    "signalColor": "#D6006F",
    "signalTextColor": "#0F172A",
    "labelBoxBkgColor": "#F1F5F9",
    "labelBoxBorderColor": "#E2E8F0",
    "labelTextColor": "#0F172A",
    "loopTextColor": "#0F172A",
    "noteBkgColor": "#F1F5F9",
    "noteBorderColor": "#CBD5E1",
    "noteTextColor": "#0F172A",
    "activationBkgColor": "#E3F6F9",
    "activationBorderColor": "#007C91",
    "sequenceNumberColor": "#D6006F",
    "arrowheadColor": "#D6006F",
    "fontFamily": "Inter, Segoe UI, Helvetica Neue, Arial, sans-serif",
    "fontSize": "15px"
  },
  "flowchart": {"curve": "basis", "nodeSpacing": 28, "rankSpacing": 36, "padding": 8}
}}%%
sequenceDiagram
    title How C-45 becomes sufficient information
    participant M as Available changes
    participant E as Keyed state
    participant R as Results

    M->>E: Jul 30 · 14:02 · C-45 pending · amount unknown
    Note over E: C-45 · amount unknown · pending
    Note over E,R: A delta cannot be produced yet
    M->>E: Jul 30 · 14:05 · C-45 confirmed · R$30
    Note over E: C-45 · R$30 · confirmed
    E->>R: Produce ΔV = + R$30
    Note over R: Available credit · R$920 → R$890<br/>Statement · R$80 → R$110

Until confirmation, the pipeline retains C-45 as an incomplete entry in the keyed state. We call this the per-key reconstruction state: the minimum information retained until the current record can be reconstructed and ΔV can be produced.

4. Late-arriving records change the known past

Even when a purchase record is complete, it may become available only after the pipeline has processed the period it belongs to. This is a late-arriving record.

A late-arriving record has two relevant times:

event time: when the purchase happened and which billing cycle it belongs to;
processing time: when it became available to the pipeline.

The July billing cycle ends on July 31 at 23:59. The batch run on August 1 at 00:15 finishes with a statement of R$110 and R$890 of available credit. Ten minutes later, another purchase becomes available:

Purchase	Current amount	Status	Event time	Processing time
`C-48`	R$80	confirmed	Jul 31, 23:58	Aug 1, 00:25

C-48 belongs to the billing cycle that was already processed but became available only after the 00:15 run. This delay can also occur in incremental batches; it does not depend on real-time processing.

The pipeline forms the keyed state for C-48 and computes ΔV = + R$80. Because the statement is still open, the purchase updates both results:

Result	Previous value	Change applied	New value
Available credit	R$890	`- R$80`	R$810
Open statement	R$110	`+ R$80`	R$190

A later full load would incorporate C-48 by reconstructing the entire billing cycle; incremental processing applies the change to the results already computed. In both cases, finishing a July batch does not mean every July record is already available: the known past can still expand.

The Dataflow Model formalizes the distinction between the time when an event occurred and the time when it became available for processing.

5. When an answer must be final

A statement must be closed even without a guarantee that every July purchase has already arrived. In this example, the July statement closes on August 1 at 00:30.

All purchases below were made in July: those available by closing time can still be included in the statement; those that become available later are moved to the next one:

Purchase	Current amount	Status	Event time	Processing time
`C-42`	R$120	reversed	Jul 29, 11:00	Jul 29, 11:00
`C-43`	R$80	confirmed	Jul 28, 10:00	Jul 28, 10:00
`C-45`	R$30	confirmed	Jul 30, 14:05	Jul 30, 14:05
`C-48`	R$80	confirmed	Jul 31, 23:58	Aug 1, 00:25
`C-49`	R$40	confirmed	Jul 31, 23:59	Aug 1, 01:15

Purchase C-49 was made in July but became available only at 01:15, 45 minutes after closing. Because it arrived after the 00:30 cutoff, it does not change the statement already issued and moves to the next one.

Two frontiers determine when the result is final

To issue the July statement, the pipeline must answer two questions:

Which records belong to the billing cycle?
How do we know when all relevant inputs have advanced far enough for the statement to be closed?

Each question defines a frontier.

Completeness

Frontier	What it defines
Billing cycle	which purchases belong to the July billing cycle
Processing	purchases available by 00:30 enter the July statement

The billing-cycle frontier classifies C-48 and C-49 as July purchases. The processing frontier includes C-48 in this statement because it became available at 00:25 and assigns C-49 to the next one because it did not arrive until 01:15. When data comes from independent inputs, verifying that all of them have reached this second frontier requires coordination.

In this context, completeness means that all relevant inputs have advanced to the processing frontier. Once this condition is met, the statement can be issued.

%%{init: {
  "theme": "base",
  "themeVariables": {
    "background": "#F8FAFC",
    "primaryColor": "#FFFFFF",
    "primaryTextColor": "#0F172A",
    "primaryBorderColor": "#E2E8F0",
    "secondaryColor": "#F1F5F9",
    "tertiaryColor": "#FFE0F0",
    "lineColor": "#5A6B7D",
    "titleColor": "#0F172A",
    "clusterBkg": "#F1F5F9",
    "clusterBorder": "#E2E8F0",
    "actorBkg": "#FFFFFF",
    "actorBorder": "#E2E8F0",
    "actorTextColor": "#0F172A",
    "actorLineColor": "#CBD5E1",
    "signalColor": "#D6006F",
    "signalTextColor": "#0F172A",
    "labelBoxBkgColor": "#F1F5F9",
    "labelBoxBorderColor": "#E2E8F0",
    "labelTextColor": "#0F172A",
    "loopTextColor": "#0F172A",
    "noteBkgColor": "#F1F5F9",
    "noteBorderColor": "#CBD5E1",
    "noteTextColor": "#0F172A",
    "activationBkgColor": "#E3F6F9",
    "activationBorderColor": "#007C91",
    "sequenceNumberColor": "#D6006F",
    "arrowheadColor": "#D6006F",
    "fontFamily": "Inter, Segoe UI, Helvetica Neue, Arial, sans-serif",
    "fontSize": "15px"
  },
  "flowchart": {"curve": "basis", "nodeSpacing": 28, "rankSpacing": 36, "padding": 8}
}}%%
sequenceDiagram
    title Late-arriving records before and after closing
    participant N as Purchases in availability order
    participant C as Available credit
    participant F as Statements

    Note over C: R$890
    Note over F: July (Open) · R$110
    N-->F: Jul 31 · 23:59 · billing-cycle frontier
    Note over N: Jul 31 · 23:58<br/>C-48 · R$80
    N->>C: Reduce by R$80
    Note over C: R$810
    N->>F: Add to the July statement
    Note over F: July (Open) · R$190
    N-->F: Aug 1 · 00:30 · processing frontier
    Note over F: July (Issued) · R$190
    Note over N: Jul 31 · 23:59<br/>C-49 · R$40
    N->>C: Reduce by R$40
    Note over C: R$770
    N->>F: Add to the next statement
    Note over F: July (Issued) · R$190<br/>August (Open) · R$40

Coordination

The statement can be issued only when all inputs capable of changing it have processed all arrivals through 00:30. Suppose the pipeline receives purchases from two independent card processors:

%%{init: {
  "theme": "base",
  "themeVariables": {
    "background": "#F8FAFC",
    "primaryColor": "#FFFFFF",
    "primaryTextColor": "#0F172A",
    "primaryBorderColor": "#E2E8F0",
    "secondaryColor": "#F1F5F9",
    "tertiaryColor": "#FFE0F0",
    "lineColor": "#5A6B7D",
    "titleColor": "#0F172A",
    "clusterBkg": "#F1F5F9",
    "clusterBorder": "#E2E8F0",
    "actorBkg": "#FFFFFF",
    "actorBorder": "#E2E8F0",
    "actorTextColor": "#0F172A",
    "actorLineColor": "#CBD5E1",
    "signalColor": "#D6006F",
    "signalTextColor": "#0F172A",
    "labelBoxBkgColor": "#F1F5F9",
    "labelBoxBorderColor": "#E2E8F0",
    "labelTextColor": "#0F172A",
    "loopTextColor": "#0F172A",
    "noteBkgColor": "#F1F5F9",
    "noteBorderColor": "#CBD5E1",
    "noteTextColor": "#0F172A",
    "activationBkgColor": "#E3F6F9",
    "activationBorderColor": "#007C91",
    "sequenceNumberColor": "#D6006F",
    "arrowheadColor": "#D6006F",
    "fontFamily": "Inter, Segoe UI, Helvetica Neue, Arial, sans-serif",
    "fontSize": "15px"
  },
  "flowchart": {"curve": "basis", "nodeSpacing": 28, "rankSpacing": 36, "padding": 8}
}}%%
sequenceDiagram
    title The statement waits for both card processors
    participant A as Card processor A
    participant B as Card processor B
    participant F as July statement

    Note over F: Open
    Note over A: Jul 31 · 23:50<br/>C-50 · R$100
    A->>F: C-50 processed at 00:20
    Note over A: Processed all arrivals through 00:30
    Note over F: Do not issue<br/>Wait for card processor B
    Note over B: Jul 31 · 23:55<br/>C-51 · R$80
    B->>F: C-51 processed at 00:29
    Note over B: Processed all arrivals through 00:30
    Note over A,B: A and B processed all<br/>arrivals through 00:30
    Note over F: Issue the statement

Card processor A reaches the 00:30 processing frontier before card processor B. While B is behind, the statement remains open because the common frontier is limited by the least advanced input:

common frontier = minimum frontier across the inputs

When B also reaches 00:30, the statement can be issued. Exchanging purchases and progress notices is communication; blocking issuance until both processors have reached the common frontier is coordination.

Available credit remains revisable and incorporates each change as soon as it becomes available. The statement waits because its current value must eventually become final.

6. Theoretical grounding: how answers evolve and become final

Two questions frame how a result can behave:

How does it evolve? Does it only accumulate conclusions, or can it replace previous ones?
What stability does it offer? It can remain revisable or become final.

Monotonicity answers the first question. The CALM theorem (Consistency As Logical Monotonicity) uses this property to explain why finalizing a non-monotonic answer requires coordination.

Monotonicity: preserving or replacing conclusions

If I learn something new, will I need to retract a conclusion I have already produced?

When the answer is no, the computation is monotonic. When it is yes, the computation is non-monotonic.

The arrival of C-48 has different effects on two answers:

Answer	Before `C-48`	After `C-48`	Previous conclusion
Observed purchases	`C-42`, `C-43`, and `C-45`	`C-42`, `C-43`, `C-45`, and `C-48`	remains
Available credit	R$890	R$810	is replaced

The observed purchases form a monotonic answer because learning about C-48 does not remove the previous purchases. Available credit is non-monotonic because R$890 stops being the answer when the value becomes R$810. The open statement behaves like available credit.

We can formalize this distinction by representing both known information and the conclusions produced as sets. Let I be the information available at one point and J be that information plus new facts. Therefore, I ⊆ J.

The monotonicity condition is:

I ⊆ J ⇒ f(I) ⊆ f(J)

Here, f is the transformation that produces the answer, and ⊆ means “is contained in”:

I ⊆ J: J contains everything the pipeline knew in I and possibly new facts;
f(I) ⊆ f(J): the conclusions produced from I remain present after f processes J.

In the first row of the table, f(I) ⊆ f(J). In the second, the condition does not hold. Non-monotonicity, however, does not prevent an answer from being maintained incrementally through corrections.

CALM: when advancing independently is not enough

While card processor A has already reached 00:30 and card processor B has not, available credit and the statement are subject to the same uncertainty: B may still report a purchase capable of changing their values. The difference lies in the guarantee each answer provides:

Non-monotonic answer	Guarantee	What the pipeline can do while B is behind
available credit	revisable	publish the current value and correct it later
statement	final after issuance	keep it open and wait for B

An execution is coordination-free when each part can advance without waiting for all others to satisfy a common condition. Available credit allows this provisional progress; issuing the statement does not.

The CALM theorem formalizes the boundary of coordination-free execution. In the models in which it has been proven:

monotonic computation
⇔
admits consistent coordination-free execution

Here, consistent means that the same facts lead to the same result regardless of processing order or distribution. In this example, the consequence is direct: when a non-monotonic answer must become final, coordination is required.

The statement was maintained with deltas for the entire billing cycle. Coordination appeared only at closing and did not require a full load.

Keeping CALM: When Distributed Consistency is Easy presents this relationship, and the proof by Ameloot, Neven, and Van den Bussche demonstrates the equivalence in a specific formal model.

7. How to choose between full loads and incremental processing

The choice is about more than whether a tool can read fewer rows. It requires answering three questions.

1. Does the change contain enough information?

The pipeline needs a stable key, the information that distinguishes insertions, updates, and deletions, and the event and processing times needed to identify late-arriving records. When a change arrives incomplete, the pipeline must also be able to form the per-key reconstruction state.

If the source does not provide enough information and the pipeline cannot reconstruct the context from what it preserved, it cannot derive correct deltas. In that case, a full load may be the only reliable way to rebuild the result.

2. Does maintenance cost less than reconstruction?

The fact that a transformation is incrementalizable does not automatically make incremental maintenance worthwhile. The benefit appears when the amount of work tracks the size of the change rather than the size of the database. This assessment must consider the preserved keyed state and the cost of updating each aggregation. In the example, SUM applies the delta directly; MAX may need to scan every valid key to find a replacement.

When almost the entire database changes, or preserving and querying this information costs as much as recomputation, a full load remains a reasonable choice.

3. What guarantee must the result provide?

A result can continue to evolve without a closing step when it only accumulates conclusions or when the consumer accepts revisions. A final result must declare which data belong to the computation, the cutoff for accepting them, and what happens to late-arriving records. If the computation is non-monotonic and depends on distributed inputs, closing also requires completeness and coordination.

Incremental processing makes sense when the pipeline can interpret changes, maintain the result more cheaply than reconstructing it, and deliver the expected guarantees.

When the context of changes cannot be reconstructed or maintenance costs approach the cost of rereading the database, a full load remains reasonable. It also remains useful for initialization, reconciliation, and recovery.

The choice, therefore, is not between an old technique and a modern one. It is between reconstructing the result and explicitly managing the information, corrections, and guarantees required to maintain it over time.