Wiring Data Science and Marketing into One Automated Pipeline with Knowledge Catalog and Gemini

In many companies there is a deep seam between data scientists and marketers. The former know where the data lives and how to build models; the latter know what to say to whom. But the two sides are joined by manual data pulls, email handoffs, and ad-hoc scripts. Running the full loop from data exploration to campaign review often takes months.

The real problem is not that the models are weak; it is that the context is sliced into several pieces. Closing that seam does not require yet another tool. It requires making a large language model (LLM) and a centrally managed data catalog native to each other, so they become the organization's single source of truth.

Google Cloud's Knowledge Catalog is exactly that foundation. It manages not only data resources built on Google Cloud but also heterogeneous datasets, metadata, table schemas, and business glossaries scattered across multicloud and on-premises environments. It also leaves room for data engineers and data scientists to register, define, and visualize new datasets and business logic.

More importantly, its role has changed. The old data catalog was a static dictionary for people to browse; Knowledge Catalog is a dynamic execution foundation where people and AI agents share the same context and act on it. Only when humans and LLMs share the same data definitions and business logic can the whole data lifecycle speed up while governance and security hold.

Below, we follow this pipeline through a retail bank's loan cross-sell scenario.

1. Start exploring data with one natural-language sentence

To prevent churn and steer customers toward more profitable products, the first requirement is to locate and analyze three kinds of data immediately: customer accounts, credit card spending records, and historical loan offer records.

The data scientist does not need to write SQL or leaf through a data dictionary first. They just tell Gemini:

Tell me what data exists on customer accounts, card spending, and historical loan offers, and visualize my data landscape.

Gemini automatically calls its built-in "Data Cloud skill," finds the relevant datasets from the metadata registered in Knowledge Catalog, and draws the data landscape right away. The scenario can be concrete: pitching a low-interest personal loan to customers carrying high-interest credit card balances as a debt switch. At that point the data scientist only has to say, "find me the customers who would save money by moving their high-interest card balance to a low-interest personal loan," and they move smoothly into query design.

2. Compute without moving data: auto-generated PySpark and zero-copy joins on Lightning Engine

In a petabyte-scale lakehouse, physically extracting data and transferring it for analysis creates huge network cost and latency. What a modern data architecture wants is to push compute next to the data, not to move the data to the compute.

Using the Data Cloud skill, Gemini automatically generates optimized PySpark code that joins and aggregates customer data with card spending data in the lakehouse, then pushes that work down to Lightning Engine, a high-performance query optimization engine that sits close to the lakehouse storage.

This removes the "extract from storage (ETL), then serialize/deserialize over the network" overhead of the old architecture and enables zero-copy joins: no data movement, very fast analysis, and data locality and security fully preserved.

The next-generation Python Notebook environment adds two handy capabilities:

• Line-by-line live editing: the PySpark code Gemini generates can be edited and customized line by line, in real time, right in the Notebook cell.
• Interactive recomputation: change a parameter in a Notebook dropdown and every chart and visualization below it recalculates automatically.

[Data scientist's natural-language instruction]
       │
       ▼ (schema analysis by the Data Cloud skill)
[Auto-generated PySpark code]
       │
       ▼ (query pushdown execution)
[Lightning Engine / lakehouse] ─── zero-copy join (no data movement / ultra-low latency)
       │
       ▼ (instant visualization)
[Initial analysis result]
  ⇒ 50,000+ target customers identified (average savings of $3,400)

That first pass reaches a conclusion quickly: there are already more than 50,000 target customers who could save an average of $3,400 by moving from high-interest cards to a low-interest personal loan.

With that list in hand, the next step is to build a model that predicts, precisely, who is most likely to accept the loan-switch offer.

3. Build the model with Gemini and Managed Spark, and explain why it decides

Picking the customers who would actually say yes to an offer requires choosing an algorithm and training it.

Through Knowledge Catalog, Gemini reads the "past three years of campaign history data" stored in BigQuery. Weighing gradient boosting's strong performance and algorithmic fit for structured tabular data, it automatically selects XGBoost as the best model. And to avoid the overhead of building and managing distributed infrastructure, it automatically chooses fully managed Managed Spark as the runtime.

Because Gemini accurately grasps the context of the columns needed for feature engineering, it can produce high-quality Python code directly, freeing the data scientist from writing complex pipelines from scratch.

Beyond accuracy, the model has to explain itself

In enterprise decisions, the question "why did the model decide this?" matters as much as predictive accuracy. That is explainability (Explainability / XAI). From a financial-regulation and compliance standpoint, a black-box model is hard to put into production.

Evaluated on historical test data, the model delivered solid results:

• Model accuracy (Accuracy): 90%
• Precision (Precision): 85%

Gemini also visualizes and explains the main feature signals (Propensity drivers) behind the predictions:

• High interest credit card rates
• Steady monthly payroll deposits

With these explainable signals in hand, the marketing team no longer has to rely on gut feel and rules of thumb. It can build a more persuasive communication strategy backed by objective data.

One-click publish to the catalog, cutting handoff cost to zero

Once training and evaluation are done, the ML model can be published from the Notebook environment straight to Knowledge Catalog with one click.

[Build and evaluate the ML model in the Notebook] (90% accuracy, 85% precision)
       │
       ▼ (one-click publish)
[Knowledge Catalog (governance foundation)]
       │
       ├─► [Marketing manager]: on-demand calls via the GUI / Gemini interface
       └─► [Other AI agents]: called as a model inference API inside automated workflows

The real value of registering the model in the catalog is that it removes the bottleneck that used to sit on the data science team — operationalizing the model, wrapping an API, and writing batch scripts. It lets business users run the predictive model themselves through Gemini. The published model becomes available across the organization immediately, and the flow moves seamlessly into the downstream marketing execution stage.

4. Downstream: the marketing manager runs an AI-led campaign

Because the ML model the data scientist built is rooted in Knowledge Catalog, the marketing front line can put an advanced predictive model to work without troubling the development team at all.

The marketing manager also uses one natural-language sentence with Gemini:

Among customers in Illinois, identify the group most likely to accept this loan offer, and create an email campaign that includes digital marketing assets.

The Gemini agent immediately searches Knowledge Catalog, automatically discovers and calls the ML model that was just published, and runs inference. From the Illinois customer data, it extracts the target segment with the highest acceptance probability.

Running on the same integrated platform, a single Gemini agent foundation shares the same tools, skills, and memory (context), so it can handle the multi-step work end to end, from data analysis to content generation to campaign staging:

Once the campaign is staged automatically, the next stage is monitoring, where results are tracked and optimized in real time.

5. Real-time dashboards and Dynamic Query Templates: solving cost, speed, and accuracy together

After a campaign goes live (say a week later), verifying its revenue impact accurately requires a real-time, interactive measurement environment.

When the marketing manager asks Gemini to "show the personal loan campaign's performance by the top 5 counties," a BI dashboard connected to live data is generated within seconds. Adding a new KPI to dig deeper into the data story is just as easy: instruct it to "add the median time from campaign to loan conversion (median time)," and a new KPI tile is generated and placed on the dashboard immediately.

[Dashboard generation prompt]
       │
       ▼ (connected to live data within seconds)
[Top-5-counties performance dashboard rendered]
       │
       ▼ (add KPI: median conversion time)
[New KPI tile added instantly]
       │
       ▼ (user switches the "county" filter)
[Dynamic Query Templates fire] ──► sub-second redraw with zero LLM tokens!

The catch: once you integrate an LLM into an enterprise BI system, calling LLM inference every time a user changes a dashboard filter or parameter not only drives up token cost but also introduces inference latency and the risk of output drift (hallucination).

The fix is to embed Dynamic Query Templates (context templates) registered in Knowledge Catalog into the dashboard backend. That delivers four layers of technical and economic value:

• Zero LLM token cost (Zero LLM Tokens): when a user changes a parameter such as "counties," the request does not go through LLM inference; an optimized template query runs directly on the database, so LLM token consumption is zero.
• Sub-second response: by skipping the LLM generation phase, response time for a graphical parameter change drops below one second, making interactive analysis smooth.
• Hallucination risk eliminated: because the LLM no longer regenerates SQL, the risk of wrong queries and misstated numbers is ruled out by design, guaranteeing BI results that are 100% accurate and truly grounded in enterprise data.
• Instant team sharing: an updated live analysis can be shared across the organization in real time with a single link.

With this query optimization and governance mechanism, cost, speed, and accuracy can all hold at once, completing a sustainable data-use cycle.

6. Conclusion: compressing a months-long process into minutes

In the past, the full enterprise process — data exploration, writing PySpark, building and evaluating an ML model, handing it off to marketing, configuring a campaign, and measuring results with a dashboard — took months of manual work and cross-department coordination.

By rooting enterprise data firmly in Knowledge Catalog and layering Gemini agents into an integrated flow, those steps can be compressed into "just a few minutes."

Grounding enterprise data correctly in advanced AI agents is precisely how today's companies, with ever more complex data environments, gain an edge in a fast-moving market.