From Raw PostHog Data Events to AI-Powered Executive Reports with Databricks

Every company collects data.

Website clicks, purchases, quote requests, page views, marketing campaigns, customer interactions—the problem has never been collecting data.

The challenge has always been turning that data into something people can actually use.

For years, the process looked something like this:

A stakeholder asks a question.

An analyst writes SQL.

The results are cleaned, validated, and eventually turned into a dashboard or report.

Then the next question comes along, and the process starts over.

After spending the last couple of weeks exploring Databricks, I realized that this workflow is beginning to change.

Building the Pipeline

To better understand the platform, I built an end-to-end analytics pipeline using PostHog event data.

The goal wasn't simply to move data from one location to another—it was to build a trusted data foundation that could answer business questions.

The architecture followed Databricks' Medallion Architecture:

Bronze Layer

• Raw event data ingested from PostHog
• Minimal transformations
• Historical source of truth

Silver Layer

• Cleaned and validated events
• Standardized schemas
• Removed duplicates
• Business-ready event data

Gold Layer

• Aggregated business metrics
• Marketing performance
• User behavior
• Conversion funnels
• Executive reporting tables

Once the transformations were complete, I scheduled the pipeline to run automatically so the Gold tables remained continuously up to date.

At that point, the engineering work was essentially finished.

That's when things became interesting.

From SQL to Natural Language

Rather than opening a notebook and writing SQL, I used Databricks Genie Spaces to ask questions in plain English.

Questions like:

• Which marketing campaign performed the best?
• Why did conversions increase in June?
• Where are users abandoning the buying process?
• Which devices convert the best?
• Which landing pages generate the highest quality traffic?

Within seconds, Genie generated a comprehensive executive report using the Gold tables as its source of truth.

No dashboards.

No SQL.

No manually assembled reports.

Just business questions answered using curated data.

The Insights

The report surfaced several findings almost immediately.

Google Ads significantly outperformed Instagram and Facebook campaigns, producing the strongest conversion performance despite running for a much shorter period.

Desktop visitors converted nearly five times better than mobile users, even though mobile represented the majority of website traffic. That immediately highlighted mobile experience as a priority for future optimization.

The conversion funnel also revealed a substantial drop-off between product configuration and quote initiation, indicating friction in the customer journey that likely wasn't obvious from surface-level metrics alone.

Perhaps the most surprising finding was that June generated the highest number of quote submissions despite having the lowest overall traffic during the reporting period. Rather than simply attracting more visitors, traffic quality had improved, suggesting recent marketing efforts were attracting users with stronger purchase intent.

These weren't simply charts or KPIs.

They were business recommendations generated directly from trusted analytical models.

The Real Lesson

Going into this project, I expected the ETL pipeline to be the most valuable part.

It wasn't.

The real value came from the Gold layer.

AI can only provide useful answers when it has reliable data to reason over.

If the underlying data is inconsistent, duplicated, or poorly modeled, natural language queries simply produce unreliable conclusions.

But when the data has been carefully validated, transformed, and modeled around the business, something different happens.

Anyone—not just analysts—can begin asking meaningful questions.

The role of the data engineer shifts from writing reports to creating trusted datasets that power intelligent decision-making across an organization.

Why This Matters

As someone who works across both full-stack development and data analytics, this project reinforced something I've believed for a long time.

Applications and analytics are becoming increasingly interconnected.

Modern applications don't just generate data.

They continuously feed analytical systems that can monitor customer behavior, measure business performance, detect anomalies, and increasingly allow AI to explain what is happening in plain language.

That changes the role of software developers as much as it changes the role of data engineers.

Building the application is only half the story.

Designing the data architecture behind it is becoming just as important.

Final Thoughts

This project gave me a much deeper appreciation for the Databricks ecosystem—not because it made ETL easier, but because it demonstrated what becomes possible once your data foundation is in place.

The future of analytics isn't replacing analysts with AI.

It's giving organizations the confidence to let anyone ask questions of trusted data.

Natural language interfaces are only as powerful as the datasets behind them.

The better the data engineering, the better the answers.

And that's where I believe the industry is headed.