Enrich Assets

You can enrich your assets in ADOC without changing the underlying data by:

• Creating virtual assets that combine or transform data from existing assets.
• Adding query lineage to show how data flows between assets.

Virtual assets

In ADOC, virtual assets are assets that do not exist as physical tables but behave like standard data assets. They are defined using SQL queries or visual graphs, allowing you to model complex data logic without duplicating physical data.

Key Benefits of Virtual Assets:

• Combine data from multiple sources without creating new tables
• Apply data quality policies, profiling, and monitoring as if it were a physical asset
• Simplify reporting and analytics workflows by creating reusable datasets
• Reduce storage and maintenance overhead since the data is computed dynamically

Virtual assets come in two main types: SQL Views and Visual Views.

1. SQL View

An SQL View is a virtual asset defined by a custom SQL query. Instead of referencing a physical table, the asset dynamically computes its content using the query provided.

Use Cases:

• Joining multiple tables for analysis
• Aggregating metrics like total sales or customer counts
• Applying filters to focus on specific segments

Create an SQL View

1. On the Discover Assets page, click the ⌘ icon (Actions in list view), and then select Add SQL View.
2. In Name, enter a name for the SQL view.
3. In Select Datasource, select the data source. Then select the database or catalog, depending on the data source.
4. In Description, enter the purpose of the view.
5. In Query, enter the SQL query that defines the virtual asset.
6. Click Preview to check sample results.
7. Click Save.

Example:

SELECT customer_id, email, SUM(order_total) AS total_spent FROM orders WHERE order_date >= '2025-01-01' GROUP BY customer_id, email;

This SQL View aggregates total spending per customer without creating a new physical table.

2. Visual View

A Visual View is a graphical representation of a virtual asset. It allows you to visually define joins, transformations, and aggregations across multiple datasets.

Use Cases:

• Quickly combining tables from different sources
• Visualizing relationships between datasets
• Creating reusable virtual datasets for monitoring and analytics

Create a Visual View

Supported asset types

Visual View supports tables, views, files, and streaming topics as source assets. Google Cloud Pub/Sub topics can be added as left or right assets in a join, or as a standalone asset with filter conditions applied. A Pub/Sub data source must be connected and crawled in ADOC before its topics appear in the asset selector.

1. On the Discover Assets page, click the ⌘ icon (Actions in list view), and then select Add Visual View.
2. In Visual View Name, enter a name. Optionally, enter a Description.
3. In Select Data Source, select the data source.
4. Click Next to open an empty canvas.
5. Click Add Asset to bring in tables or datasets.
6. Define joins, transformations, and relationships using the visual editor.
7. Click Validate to check syntax and preview result data.
8. Click Save to store the virtual asset.

Incremental strategies for Pub/Sub assets

When a Visual View includes a Pub/Sub topic, incremental profiling uses Pub/Sub-specific time-bound strategies: timestamp-based (processes messages published after the last run) or lookback-based (processes messages within a defined time window). Standard SQL-based incremental strategies do not apply to Pub/Sub assets.

Edit a Visual View:

1. Open the Visual View asset.
2. Click the Edit icon to modify the canvas.
3. Update joins, filters, or transformations as needed.
4. Click Validate and then Save Changes to apply updates.

Query lineage

Query Lineage helps you visualize how data flows into and out of an asset. It automatically identifies upstream and downstream dependencies based on your SQL queries—capturing how data is transformed, joined, or aggregated before it reaches its final form.

You can add lineage by uploading a query file for ADOC to analyze. Once created, the lineage is visible directly within the asset’s Lineage tab.

Add a query lineage

1. In the left navigation, select Data Observability Cloud > Discover > Assets.
2. Click the ⌘ icon (Actions in list view), and then select Query Lineage.
3. On the Create Query Lineage page, provide the following details:
   1. Select Data Source: The data source where the queries run.
   2. Select Database: The database that contains the referenced objects.
   3. Select Schema: The schema that contains the referenced objects.
   4. Queries file: Drop or select a text file that contains your SQL queries. Put each query on a separate line.
4. Click Submit to analyze the queries and register the lineage.

Ensure all SQL statements are valid and reference existing tables or views. Syntax errors or inaccessible objects may prevent lineage extraction.

View lineage

After a lineage has been created, you can view its details directly from the asset’s page.

1. In the left navigation, select Data Observability Cloud > Discover > Assets.
2. Locate and click the asset name to open its Asset Details page.
3. In the left pane, select Lineage.
4. The Lineage tab displays a graphical view showing:
   1. Upstream dependencies (data sources feeding into the asset)
   2. Downstream dependencies (assets or dashboards using this data)
   3. Transformation paths inferred from SQL queries
   4. Related systems (such as Snowflake, BigQuery, and Tableau)

Use zoom, pan, or filter controls to focus on specific parts of the lineage graph. For more information, see Lineage.

Best practices and use cases

• Trace data origins and dependencies: Understand which upstream tables feed into a given asset.
• Impact analysis: Identify downstream dependencies affected by schema or data changes.
• Governance and auditing: Verify that data transformations align with business rules or compliance requirements.
• Complex pipelines: Upload multi-step SQL workflows (including CTEs, joins, and unions) to infer lineage across systems.