DP-600: Discovering, Connecting, and Ingesting Data in Microsoft Fabric

An analytics engineer should make the acquisition decision in this order: discover a governed source, confirm its owner and intended analytical use, choose the identity that will run the connection, and then select the least complex method that satisfies movement, freshness, and shaping requirements. Use copy activity inside a Data Factory pipeline for repeatable movement that needs orchestration, Dataflow Gen2 when low-code Power Query shaping is part of acquisition, streaming into an Eventhouse when continuously arriving events require low-latency analysis, and in-place access when the existing data can be used without creating another managed copy.

Keep four boundaries explicit. The destination store must already fit the workload; the connection determines how Fabric authenticates to the source and destination; the acquisition method determines whether and when data moves; and transformation determines how values and structure change. OneLake Catalog and Real-Time hub help find suitable data, but discovery does not grant access or decide the ingestion architecture. This framework prevents a convenient connector or interface from silently deciding storage, security, freshness, and transformation policy.

Architecture Overview

Microsoft Fabric data acquisition spans a control path rather than a single tool. Discovery identifies a candidate asset or stream. Governance evidence establishes whether it is appropriate to use. A connection records the endpoint and authentication context needed by a Fabric workload. An acquisition method then copies, orchestrates, transforms, streams, or references the data. Finally, the selected lakehouse, Warehouse, or Eventhouse owns the resulting analytical responsibility.

The sequence matters because each stage answers a different question:

Discovery: What data exists, where is it, and who owns it?
Authorization: Is the engineer allowed to use it for this analytical purpose?
Connection identity: Which credential or service identity will execute development, scheduled, and production runs?
Acquisition: Must the data move, be shaped while it moves, arrive continuously, or remain in place?
Destination: Which already-selected analytical item receives or exposes it?
Operations: How will the team validate freshness, failures, schema changes, cost, and access?

Microsoft's current DP-600 study guide places creating connections, discovering data through OneLake Catalog and Real-Time hub, and selecting or ingesting data within the Get data scope. Product documentation supplies the implementation behavior; the exam scope alone is not evidence that every source or connector supports the same options.

Discover Governed Data Before Building a Connection

Use OneLake Catalog for Fabric analytical assets

OneLake Catalog is the central discovery and governance experience for Fabric data and analytical assets. Start there when the likely source is already represented in Fabric. Search and filtering help narrow the candidates, while item context and governance information help distinguish an authoritative asset from a similarly named development copy.

Discovery should produce a source record, not an immediate ingestion job. Record the item, workspace, owner, business purpose, expected schema, freshness, sensitivity, and downstream usage constraints. Open the asset through its appropriate experience to confirm that its tables or files contain the required grain and history. A discoverable item is not automatically approved, queryable, or suitable for redistribution; access still depends on the relevant workspace, item, and data permissions.

OneLake is the tenant-wide logical data lake built into Fabric, but shared storage does not erase item boundaries. A lakehouse, Warehouse, and Eventhouse expose different engines and operational behaviors. The Microsoft Fabric overview describes the common platform and OneLake foundation, while the target decision belongs to DP-600: Choosing Microsoft Fabric Analytics Stores: Lakehouse, Warehouse, Eventhouse, and Semantic Models.

Use Real-Time hub for event sources and streams

Use Real-Time hub when the requirement begins with continuously arriving data, an event source, or a stream that needs real-time discovery and routing. It provides the discovery entry point for real-time data in Fabric and connects that discovery to Real-Time Intelligence experiences. Confirm the source's event shape, timestamp semantics, expected arrival rate, retention need, and acceptable latency before selecting an ingestion path.

Do not equate the presence of timestamps with a streaming requirement. Periodically exported transactions may still suit scheduled copy into a lakehouse or Warehouse. Streaming is justified when continuous arrival and low-latency analysis materially affect the solution. The Real-Time Intelligence overview positions Eventhouse and Kusto Query Language (KQL) for event-driven analytics, but the business requirement must still justify that architecture.

Verify fitness and governance at the source

Before connecting, answer five questions:

Authority: Is this the governed system or item that owns the required data?
Meaning: Do the fields, grain, history, and timestamps answer the intended analytical question?
Access: Can the planned runtime identity read the source under least privilege?
Freshness: Is the source updated at a cadence compatible with the service-level objective?
Movement constraints: Do policy, residency, licensing, volume, or source-load concerns limit copying or downstream use?

If one of these answers is unknown, resolve it with the source owner before implementation. Connecting first and investigating later can produce a technically successful pipeline that acquires the wrong, stale, or unauthorized data.

Design the Connection Around Its Runtime Identity

A Fabric connection is an operational security boundary. It combines source location with an authentication method and the credential or identity used by the workload. Design it for the identity that must execute in the deployed environment, not merely for the developer who can make an interactive test succeed.

Separate three identities when evaluating access:

The discovering user needs enough permission to find and inspect candidate data.
The authoring user needs permission to create or use the Fabric item and connection during development.
The runtime identity needs the source read access and destination write access required when a pipeline, dataflow, or streaming process runs unattended.

These may be the same during a short-lived experiment, but production should not accidentally depend on one person's account. Prefer an organizationally managed identity or credential pattern when the connector and operating model support it. Apply least privilege, document ownership, and establish a controlled process for credential rotation. A successful connection test proves only that the tested identity reached the endpoint at that moment; it does not prove that a scheduled run has target permissions or that the credential will remain valid.

Network reachability is part of the same design. If a source is not directly reachable from the Fabric service, the connection may require an approved gateway or other supported connectivity arrangement. Treat that dependency as production infrastructure: assign an owner, verify capacity and availability, and test the actual scheduled runtime path. Never place secrets in article examples, pipeline parameters intended for ordinary configuration, or source-controlled definitions.

Connection reuse can reduce duplicated credentials and simplify rotation, but only when the security boundary is truly shared. Do not reuse a broad connection across teams simply for convenience. Confirm that its endpoint, authentication context, environment, data classification, and allowed consumers match the workload.

Choose the Acquisition Method

The primary choice is not “which Fabric tool is best?” It is “what must happen between the governed source and the selected destination?” The following matrix makes that decision explicit.

Requirement	Preferred starting method	Why it fits	Boundary to preserve
Move data on demand or on a schedule with minimal shaping	Copy activity in a Data Factory pipeline	Copies between supported sources and destinations and can participate in a managed run	The pipeline orchestrates movement; it does not determine the target store's analytical role
Coordinate several activities, dependencies, schedules, or failure paths	Data Factory pipeline	Provides orchestration around copy and other activities	Keep substantive cleansing and modeling logic in the appropriate transformation layer
Acquire and shape data through a low-code Power Query experience	Dataflow Gen2	Combines connector-based ingestion with reusable low-code preparation and explicit destinations	Use it because shaping is required, not merely as another way to copy unchanged data
Continuously ingest event data for low-latency analysis	Streaming path into Eventhouse	Matches continuous arrival and KQL-based event analysis	Do not use streaming for ordinary timestamped batch data without a latency requirement
Use existing data without maintaining another copy	In-place access, such as an appropriate OneLake shortcut pattern	Avoids unnecessary movement when format, identity, and engine compatibility allow it	Shortcut and OneLake integration design belongs in the dedicated integration layer

Choose copy when movement is the requirement

Use a copy activity when the source data must be materialized in the selected Fabric destination and the required shaping is minimal. A Data Factory pipeline can schedule and coordinate the copy, pass parameters, sequence dependencies, and expose run status. The pipeline overview describes pipelines as the orchestration capability for data-movement and processing activities.

Copy is a strong default for repeatable batch movement, but it creates another managed instance of the data. Define whether each run is a full reload or an incremental acquisition, how duplicates are prevented, what happens when the source schema changes, and which system is authoritative. Validate row counts or other reconciliation measures at the source-to-destination boundary. Copying without ownership and reconciliation converts source issues into durable downstream issues.

Choose a pipeline when the workflow needs orchestration

A pipeline becomes the organizing layer when acquisition includes multiple sources, ordered activities, schedules, parameters, or failure handling. It can coordinate copy and invoke other processing, but orchestration should not become an unstructured collection of business rules. Keep each activity's input, output, identity, retry behavior, and idempotency expectations explicit.

Use retries only for failures that are safe to repeat. A rerun must not silently duplicate data or advance a dependent step after a partial load. Design checkpoints and validation around the destination's ingestion pattern, and separate operational status from evidence that the resulting data is complete and correct.

Choose Dataflow Gen2 when low-code shaping is part of acquisition

Dataflow Gen2 is the natural choice when analysts or data engineers need a Power Query-based, low-code path to connect, filter, combine, type, or otherwise prepare data before writing it to a supported destination. The Dataflow Gen2 overview describes its ingestion, transformation, destination, and pipeline-integration roles.

Choose it when the team benefits from the Power Query authoring experience and the transformation is understandable and maintainable there. If the requirement is only high-throughput movement, a copy activity can provide a clearer responsibility. If complex transformation engineering dominates, make that a separate design decision rather than hiding it inside acquisition; DP-600: Transforming and Cleaning Data in Microsoft Fabric owns the detailed choice among Dataflow Gen2, pipelines, notebooks, and Transact-SQL (T-SQL).

Choose streaming when continuous events drive the design

Use a streaming path when events arrive continuously and their analytical value depends on becoming queryable with low delay. Eventhouse is the destination when high-volume, time-oriented data and KQL analysis define the workload. Establish event-time and ingestion-time semantics, expected late or duplicate events, retention, and monitoring before treating the stream as production-ready.

Streaming adds operational responsibilities that scheduled batch movement may not need. Monitor source connectivity, ingestion delay, malformed events, volume changes, and destination availability. Define what downstream users should expect during an interruption and whether replay or backfill is available. These controls protect analytical meaning; a stream that is connected but silently delayed is not meeting a real-time requirement.

Choose in-place access when another copy adds no value

Prefer in-place access when data already resides in a usable location, the consuming engine supports its format and access path, the runtime identity can be authorized, and freshness is better served by referencing rather than copying it. This can reduce storage duplication and the operational burden of synchronizing two versions.

In-place access is not automatically simpler. Performance, availability, source lifecycle, security propagation, and schema changes remain dependencies. Record who owns the source and what happens if it moves or changes. When a OneLake shortcut or related integration pattern is the solution, use DP-600: OneLake Shortcuts and Integration Patterns for Fabric Analytics for the implementation architecture. This article decides whether in-place access fits; it does not define shortcut behavior.

Preserve Store, Shortcut, and Transformation Boundaries

The acquisition decision becomes clearer when three neighboring responsibilities are kept out of it.

First, select the analytical store before designing movement. A lakehouse, Warehouse, and Eventhouse have different primary engines, write patterns, and consumers. A familiar connector or an easy copy wizard is not a reason to change that architectural fit.

Second, treat a shortcut as an access pattern, not a generic synonym for ingestion. If the requirement is to expose existing data without copying it, validate compatibility, identity, and ownership, then design the shortcut in the OneLake integration layer. If the requirement is to retain an independent snapshot, isolate the source from analytical demand, or transform the data into a governed target, copying may still be appropriate.

Third, distinguish acquisition from transformation. Filtering a large source early or applying straightforward Power Query shaping may legitimately be part of Dataflow Gen2 acquisition. Dimensional modeling, extensive cleansing, enrichment, and reusable business logic deserve an explicit transformation design. This distinction makes lineage, failure diagnosis, testing, and ownership easier to understand.

Operational Validation and Common Mistakes

Validate the complete path with the intended runtime identity and a representative data slice before scheduling production loads. Confirm source access, destination write access, schema mapping, freshness, reconciliation, failure visibility, and rerun behavior. For streaming, also validate event timestamps, ingestion delay, malformed-record handling, and replay expectations. For in-place access, test the dependency on source availability and permissions rather than assuming that “no copy” means “no operations.”

Common mistakes include:

Treating discovery as authorization: Finding an asset in OneLake Catalog or Real-Time hub does not grant permission to use, move, or redistribute it.
Using a developer's identity as the production design: Interactive success can conceal an unattended-run failure or create a personal credential dependency.
Letting the connector choose the architecture: Connector availability does not establish the right destination, latency model, or ownership boundary.
Using a pipeline as a transformation dumping ground: Pipelines should orchestrate clear activities; transformation logic needs an intentional tool and test boundary.
Copying by default: Extra copies introduce storage, freshness, reconciliation, security, and lifecycle responsibilities.
Choosing streaming because data has timestamps: Continuous arrival and low-latency use, not a date column, justify a streaming design.
Assuming in-place access removes governance: The referenced source still has owners, permissions, schema evolution, availability, and performance constraints.
Skipping reconciliation: A completed run indicates execution status, not necessarily complete, unique, timely, or correct data.

Certification Relevance

For Exam DP-600: Implementing Analytics Solutions Using Microsoft Fabric, the important skill is to turn a requirement into a defensible sequence of decisions. Be able to explain how OneLake Catalog and Real-Time hub support different discovery contexts, why connection identity must match scheduled operation, and when copy, pipeline orchestration, Dataflow Gen2, streaming, or in-place access best fits the movement and freshness requirement.

The strongest preparation uses small implementation scenarios without reconstructing exam questions. For each scenario, state the authoritative source, runtime identity, selected destination, movement requirement, freshness target, transformation boundary, and operational evidence. Then explain why the rejected methods add unnecessary movement, latency, complexity, or dependency risk.

Conclusion

Discover first, authorize deliberately, connect with the production runtime identity, and choose the acquisition method from movement, freshness, shaping, and operational requirements. Copy activity handles repeatable materialization, pipelines coordinate workflows, Dataflow Gen2 supports low-code acquisition and shaping, streaming serves continuous event analysis, and in-place access avoids copies when compatibility and ownership make that safe.

The decision remains sound only when its boundaries remain visible. The analytical store owns workload fit, the connection owns authentication context, the acquisition method owns movement and cadence, shortcuts own a specific in-place integration pattern, and transformation owns substantive changes to data. That separation gives the team a design it can secure, operate, and explain.