AB-400: Synchronizing Dataverse with Change Tracking and Upsert

Introduction

Build a restartable Dataverse synchronization loop by assigning field ownership, fixing a stable identity, completing an initial extraction, then applying each change-tracking interval before committing its next token. Use an alternate key and UpsertRequest when the destination is Dataverse. Make replay safe for both updates and deletions, and reconcile after a lost or expired checkpoint rather than assuming that another initial extraction removes stale destination rows.

The central implementation rule is that a committed checkpoint means every preceding change has a durable, accounted-for outcome. Receiving a token is insufficient. This guide develops that rule into a procedure for synchronizing Dataverse with change tracking and Upsert, including the gap between a successful destination write and a failed checkpoint write. The procedure is a recommended application design; its checkpoint store and recovery behavior are components you implement.

Architecture Overview: One Owner and One Projection

Consider an illustrative asset register in a source Dataverse environment and a service application in a separate destination Dataverse environment. The service application needs asset names and maintenance categories, while its own users maintain local dispatch notes. The source owns the replicated name and category; the destination owns dispatch notes. Synchronization must never send dispatch notes back as authoritative asset data or overwrite them during a replay.

Use one stream for this source table, destination and mapping version. A stream consists of a change reader, a deterministic mapper, a destination writer and a durable checkpoint record. Keep a separate row-level error journal for diagnosis. Run one active writer per stream, enforced by your application's coordination mechanism, so that a delayed worker cannot overwrite a newer worker's checkpoint.

This is state replication, so success means that the destination converges to the source-owned values. If the business needs every intermediate transition for an audit or workflow, define that requirement separately; a state synchronization acceptance test cannot prove delivery of every business event.

Before implementation, establish an isolated development pair of environments, authenticated clients, approved permissions and an explicit deletion policy. Verify the source table's eligibility and enable change tracking deliberately: enabling it cannot be reversed. The documented .NET retrieval requires organization-level read access. Treat that access and the destination copy as a data-governance decision, including which columns may leave the source environment.

Core Components and Their Contracts

Stable identity and alternate-key readiness

For the example destination table, define an alternate key over two illustrative text columns: lpx_sourcesystem and lpx_sourceid. The first contains a configured source-environment tag; the second contains the source row's globally unique identifier (GUID) in one agreed representation. This pair is the external identity from the destination's perspective. Keep both values stable across every run and preserve their derivation when handling deletion records.

An asset's display name is a poor identity: a rename should update one row, not create another. The environment tag also prevents two independent sources from sharing an accidental identity namespace. Record a change to that tag as a migration, because treating it as ordinary configuration would create an entirely different set of destination identities.

Microsoft documents alternate-key uniqueness through an index and exposes its creation status. Require Active before allowing this stream to write; stop for Pending, In Progress or Failed. Validate existing data and diagnose failed index creation before starting. For a Web API implementation, check the documented restrictions on alternate-key characters. The example uses deliberately simple values rather than trying to repair unsafe identifiers by ad hoc URL encoding.

Change reader and checkpoint state

The .NET RetrieveEntityChangesRequest provides an initial population and a version for subsequent changes. A request without a token returns current rows, without historical deletions. Continue pages using MoreRecords, PagingCookie and page numbering; retain the final DataToken for the next interval. The service retains changes only for its configured retention period, so an excessively old checkpoint can fail.

Keep these application-owned state fields distinct:

State	Meaning in this design
Stream definition	Source, table, selected columns, destination and mapping version
Committed token	Last fully applied change interval
Pending interval	Immutable input token, run identifier and any durable staged changes
Page position	Progress inside that interval, never evidence of overall completion
Generation	Identifier for a rebuild being reconciled before activation

Persist tokens with their stream definition and protect their contents in storage and logs. Do not reuse a token after changing the table or mapping contract merely because it remains syntactically valid. That would make the checkpoint say that a different projection had already been applied.

An alternative Web API reader uses Prefer: odata.track-changes and the returned opaque @odata.deltaLink. Preserve that link rather than constructing tokens yourself. Its change-tracking requests do not support $filter, $orderby, $expand or $top. The procedure below uses the .NET reader; a Web API implementation needs equivalent complete traversal and commit rules rather than a mixture of SDK paging cookies and delta links.

How Components Work Together: The Synchronization Procedure

1. Freeze ownership, mapping and identity

Write a small mapping contract before moving data: source table, selected columns, target columns, conversion rules, null handling, identity derivation and delete behavior. In this example, map a source name and category into the target's replicated fields and never include dispatch notes in the write payload. Define whether an explicit source null clears the target field; otherwise a mapper can preserve an old value indefinitely without reporting an error.

Choose physical deletion only when the destination row is an expendable projection. If local service history must survive, the example's safer policy is to mark the replicated asset unavailable while retaining its local references. That flag and the code that sets it are application design, not a change-tracking feature. Decide how a later authorized restoration should reverse that flag.

2. Populate the initial generation without claiming completion early

Start with no committed token and create a new generation record. Read the source using the selected columns, map each returned row, and apply it to the destination. Track identities seen in this generation. When another page is indicated, continue the same extraction using the returned paging state. Keep the stream visibly initializing until the complete traversal and all destination outcomes have been accounted for.

For a small implementation, use the conservative recovery policy: if the process crashes during traversal, discard the partial page position and restart from the preceding committed token, or restart initialization when none exists. Repeated writes are expected. If you instead require page-level restart, durably stage the pending interval, its input token and paging state together; validate that recovery implementation independently. Never promote a partial page's progress to a completed interval.

3. Apply each row through its stable key

For a standard destination table, Upsert chooses an update or creation and returns UpsertResponse.RecordCreated and Target. Put the alternate-key identity in Entity.KeyAttributes and the owned data in Entity.Attributes; do not repeat key fields among the data being saved. The following illustrative fragment assumes an authenticated IOrganizationService, an Active alternate key and the custom columns described above. It has not been executed against a tenant.

var target = new Entity("lpx_assetprojection");
target.KeyAttributes["lpx_sourcesystem"] = sourceTag;
target.KeyAttributes["lpx_sourceid"] = sourceRowId.ToString("D");
target["lpx_name"] = sourceName;
target["lpx_category"] = mappedCategory;

var result = (UpsertResponse)service.Execute(
new UpsertRequest { Target = target });

Record a successful destination outcome only after the request succeeds. A lost response creates uncertainty, so retry the same identity and desired values rather than assigning a fresh key. The repeated call may be an update, and a different RecordCreated result does not itself indicate a duplicate row.

Stable identity prevents duplicate logical rows in this design, but it does not make all business side effects idempotent. Audit destination automation for notifications, counters or external calls that a repeated write could trigger. Those effects need their own deduplication or an explicit decision to tolerate repetition. This procedure targets standard tables; elastic-table Upsert has different overwrite and event behavior and requires a separate mapping and business-logic review.

4. Process deletions without needing yesterday's business fields

A change-tracking deletion can identify a row that the consumer never previously saw. In the example, derive its destination key from the deletion's source GUID and the configured source tag. If no matching destination row exists, record the intended absent state as satisfied. Do not create a blank asset just to acknowledge a deletion.

If the destination row exists, apply the agreed delete or unavailable-state policy. Never infer deletion from a missing name, missing optional column or a page that happens not to contain the asset. A deletion record, sometimes called a tombstone, is a different input from an ordinary row payload. Keep the identity mapping independent of mutable fields so that a rename followed by deletion still targets the original projection.

5. Commit the completed interval, then poll from it

After initialization succeeds, durably commit its final token and generation state. For each delta interval, keep the current committed token unchanged while reading and applying all pages. Only after every row and tombstone has a durable outcome should the worker replace it with the interval's final token. A failed required write blocks advancement; placing it in an error log does not mean synchronization has completed.

There is normally a failure window between a destination write and checkpoint persistence when the stores do not share an application transaction. This design resolves that window through replay: after a crash, resume from the old committed token, reapply the interval and commit only after completion. Commit the token and its stream-definition version atomically within the checkpoint store, and reject a stale worker's attempt to replace a newer checkpoint.

Design Considerations: Recovery and Reconciliation

Recover from token loss without hiding stale rows

For a rejected or lost token, start a controlled new generation and rebuild the desired projection. Preserve the previous generation until reconciliation is complete. Compare source identities and owned values with destination rows in this stream's namespace, then investigate identities present only at the destination. A clean initial extraction cannot by itself identify rows deleted before that extraction began.

Apply cleanup only to rows whose ownership and absence have been established. A partial extraction, failed page, changed permissions or incompatible mapping must stop the cleanup phase. For the illustrative asset register, schedule a controlled reconciliation window: pause source writes, complete the baseline and difference checks, approve any unavailable-state changes, commit the new generation, then resume writers and delta processing. This avoids treating a moving comparison as an authoritative deletion list. Where pausing writers is unacceptable, design and test a concurrent reconciliation protocol before enabling destructive cleanup.

Make acceptance about durable state

The following are proposed tests for the implementation, not observed tenant results. Seed a few assets and local dispatch notes, complete initialization, then perform one rename and one deletion. Inspect destination identities, owned values, retained local fields and the committed token. Add a multi-page fixture so that a successful first page cannot accidentally pass the test.

Injected condition	Required observation and reason
Crash after a target write but before checkpoint commit	Restart replays from the old token; one logical target row remains because its identity is unchanged.
Reject a required write in a later page	Token does not advance, and the failing identity is diagnosable; otherwise that interval could be skipped permanently.
Deliver the same deletion again, including an identity never initialized	Final absent/unavailable state remains correct without creating a placeholder row.
Rename an asset while retaining its GUID	The existing projection changes; local dispatch notes survive because they are outside the mapping.
Restart with a lost or invalid token and a stale target-only asset	Reconciliation detects the stale identity; cleanup waits for a complete, controlled comparison.
Start a second worker from an older checkpoint	Application coordination prevents its writes or checkpoint commit from superseding the active run.

Track checkpoint age, pending intervals, failed identities and reconciliation differences. A recent successful request is weak evidence if the committed interval remains old. Alert on the ability to finish the backlog within the configured retention window, leaving room to investigate failures before a rebuild becomes necessary.

Common Mistakes

Saving a token as soon as it arrives can skip unapplied rows after a crash. Treating a paging cookie as the overall checkpoint confuses traversal progress with a completed change interval. Using a mutable business label as the key creates duplicate projections when the label changes. Recording only successful Upsert calls misses deletion handling and stale rows after a rebuild.

Another failure is letting a permanent mapping error disappear into a retry loop. Preserve the rejected source identity and enough diagnostic context to repair the mapping, while limiting sensitive payload retention. Resume only after the failed interval can be completed or after a deliberate, documented scope change; silently abandoning a row invalidates the checkpoint contract.

For development cleanup, stop the worker before removing its disposable projection rows and checkpoint records. Select those rows through the test stream's namespace, verify the selected identities and preserve unrelated destination data. Do not expect removing test data to reverse source-table change-tracking enablement.

Certification Relevance

The selected AB-400 study-guide baseline, verified October 15, 2026 for the October 16 effective scope, includes implementing Dataverse synchronization through change tracking, alternate keys and UpsertRequest. The useful skill is explaining how those mechanisms combine into recovery behavior: the token bounds progress, the key preserves identity and the application decides when a change interval is safe to acknowledge. This guide makes no claim about live exam availability or assessment questions.

Conclusion

A restartable synchronization procedure needs a durable meaning for completion. Define ownership and identity, finish every initial or delta traversal, handle tombstones explicitly and commit the next token only after the destination outcomes are durable. Use replay for uncertain writes and reconciliation for gaps that an initial extraction cannot repair. Prove those behaviors with failures at the checkpoint boundary before allowing the stream to change production data.