Finding the Insights Hiding in Your Data: Building an AI Research-Discovery Agent on Oracle AI Data Platform
How BRAIN (Business Research and Analytical Insight Nexus) scans raw enterprise data, discovers and validates hidden patterns, and hands them to a chat agent that helps you decide what to do next — all without a single byte of your data leaving your own tenancy.
The problem
Every organization sits on operational data that quietly contains valuable, non-obvious insights — churn precursors, hidden risk segments, patterns that only show up when you join tables nobody thought to join. The data isn't the bottleneck. Nobody has the time, or the awareness, to go looking.
This is POC-01 of a small proof-of-concept series built on Oracle AI Data Platform (AIDP): a system that scans a business's own data, surfaces hidden patterns in plain English, and puts a conversational agent in front of them — entirely inside the organization's own tenancy, using on-prem/OCI-hosted models. No raw data ever has to leave.
I have been exploring Oracle AI Data Platform for some time to better understand what is possible with its integrated capabilities. This POC is a practical showcase of that exploration: I wanted to see how easily an end-to-end application could be built in AIDP - from data ingestion and insight discovery to workflow automation, an AI agent, and an executive dashboard. The experience has been both straightforward and genuinely impressive.
The data
The demo domain is retail/e-commerce: three related tables — customers, orders, and support tickets — about 100 rows each. I deliberately baked in real (but non-obvious) statistical relationships across the three tables before ever "discovering" anything: acquisition channel and discount depth predicting returns, one region running both higher order value and higher escalation rate, and — the one I was proudest to see the pipeline actually catch — an escalated support ticket predicting an above-average chance a customer's next order gets returned. None of that is stated anywhere in a column name. It only shows up on cross-table analysis, which is exactly what the system had to find on its own.
Discovery (Bronze)
Bronze's job is boring on purpose: read the raw source tables, clean up the types, stamp audit columns, write.
Insight (Silver) — the actual discovery step
This is the core of the POC, and it's not a trained model. It's a three-stage pattern:
- AI hypothesizes. The model sees a small random sample of rows plus full column headers/stats from each table — never the full data — and proposes hypotheses, but constrained to four executable templates (compare a rate/mean across categories, split on a threshold, or compare an outcome before/after a flagged event across a join) with real column names, not free-form prose.
- Code validates. Every proposed hypothesis actually runs against the data to get a real computed effect size. Nothing gets called a "finding" on the AI's word alone.
- AI narrates — but only what's validated. Only hypotheses clearing the effect-size threshold get a plain-English write-up, grounded in the real numbers.
On the run I'm writing this from: 8 hypotheses proposed, 6 validated, top effect size 0.71 — a pattern showing delayed-delivery support tickets take nearly double the resolution time of product-defect tickets, which directly contradicted my own assumption going in that defects would be the slower category. That's the kind of thing this whole project exists to surface.
Institutional memory (Gold)
Gold merges validated insights into a permanent history table (never overwritten, only appended to), deduplicates across every run into a clean "here's what we currently know" summary, and writes a structured findings file for the agent's knowledge base — each entry carrying the real table/column names behind it, not just a narrative sentence, plus an AI-suggested modeling approach for anyone who wants to formalize it further.
Workflow
Bronze, Silver, Gold — currently runs as automatically-triggered once in a month Workflow.
Every run logs its parameters and results to MLflow, including the top and average effect size found. Since MLflow keeps every run's history by default, that's a free trend line — "are recent runs finding more, or stronger, insights than older ones" — without building any separate tracking infrastructure.
The Agent
The agent is built entirely in AIDP's GUI Visual Flow Builder — genuinely no code for the agent itself. The architecture ended up mattering more than I expected: one Supervisor routing to two Executors, split specifically along a trust boundary. One Executor has the RAG and SQL lookup tools and can never call anything external — it physically has no tool that reaches outside AIDP. The other Executor has exactly one tool: a hand-off action that drafts a research brief and a link, which a human has to open and send themselves. No data is ever transmitted externally without an explicit human decision in the loop. That's not just a policy stated in a prompt — it's true by construction, since the internal executor has nothing capable of violating it even if it wanted to.
A few real exchanges from testing:
"What's our return rate for platinum-tier customers?"
(Trap question — loyalty tiers are Bronze/Silver/Gold, not Platinum.)
The agent correctly identified that no such finding — or column — exists, rather than fabricating a number, and offered to run a fresh analysis.
"Build me a research plan for something you haven't found yet."
The agent inventoried the real schema and proposed five genuinely well-reasoned unexplored angles (marketing-channel interaction with returns, payment-method/delivery-time moderation, ticket-resolution as a predictor) rather than just asking for clarification.
An executive view (OAC)
The same Gold data, made scannable in one screen: KPI cards for the headline numbers, a breakdown of validated insights by pattern type, and a ranked view of effect sizes — reusing the exact color palette from the notebook charts, so a reader who saw the notebook screenshot in this post recognizes the same visual language in the dashboard.
Acknowledgements
My sincere thanks to @InsightSeeker-Oracle ma'am for her encouragement, motivation, and guidance throughout this exploration. I am also grateful to Bhanu Pratap Singh sir and Sudhakar Gali sir for providing the resources and support that made this POC possible.
This is POC-01 of an ongoing mini-POC series built on Oracle AI Data Platform, each one telling a complete story from raw data to chat agent to dashboard. Feedback and questions welcome.
Note: Views expressed are personal and not Oracle’s
