Project Overview
As the Senior UX Researcher for a stealth-mode digital advertising startup, I led a comprehensive mixed-methods research initiative to understand how advertisers and publishers interact with programmatic ad platforms. This project was unique in that I pioneered the use of Claude AI to accelerate research synthesis while maintaining methodological rigor—reducing analysis time by 40% without compromising quality.
Problem Statement
The startup was building a next-generation advertising platform but lacked deep understanding of user workflows, pain points, and decision-making criteria. With a compressed timeline to inform the product roadmap before Series A funding, we needed to conduct robust research quickly. How could we leverage AI tools to accelerate research without sacrificing quality or introducing bias?
Context & Challenge
Business Context
The startup was developing a platform to streamline digital advertising workflows for mid-market advertisers—a segment underserved by existing solutions designed for either small businesses or enterprise clients. Key business pressures:
- 16-week timeline to deliver insights before Series A pitch
- Need to validate product-market fit assumptions
- Pressure to move fast without compromising research quality
- Limited existing data due to stealth mode (no current users)
Research Challenges
- Volume of data: Planning 30+ user interviews would generate hundreds of pages of transcripts
- Time constraints: Traditional thematic analysis would take 6-8 weeks—we had 4 weeks for synthesis
- Complexity: Needed to understand both advertiser and publisher perspectives
- Validation pressure: Stakeholders needed high confidence in insights to make strategic bets
The AI Research Hypothesis
"What if we could use Claude AI to accelerate the mechanical parts of analysis—transcript coding, pattern identification, initial theme generation—while researchers focus on critical thinking, validation, and insight synthesis?" — Research Plan Hypothesis
Research Approach
Mixed-Methods Research Design
I designed a sequential mixed-methods study combining qualitative depth with quantitative validation:
Phase 1: Qualitative Discovery (Weeks 1-6)
User Interviews (n=32)
- 16 advertisers (marketing managers, media buyers, CMOs)
- 16 publishers (ad ops managers, revenue directors)
- 90-minute semi-structured interviews via Zoom
- Topics: current workflows, pain points, decision criteria, tool evaluation
Phase 2: Quantitative Validation (Weeks 7-10)
Survey (n=156)
- Validated interview insights with larger sample
- Prioritized feature requests and pain points
- Quantified market segments and use cases
Phase 3: Behavioral Analysis (Weeks 8-12)
Pendo Analytics (Beta Users, n=24)
- Tracked feature usage and navigation patterns
- Identified friction points and drop-off areas
- Triangulated behavioral data with interview insights
Why This Approach?
The mixed-methods design allowed us to:
- Discover rich, unexpected insights through qualitative interviews
- Validate those insights quantitatively across broader audience
- Observe actual behavior through Pendo to see what users do vs. say
- Triangulate findings across methods to increase confidence
AI-Assisted Research Methods
How I Used Claude AI for Research Synthesis
I developed a rigorous framework for using Claude AI to accelerate analysis while maintaining research integrity:
1. AI-Powered Transcript Coding
Rather than manually coding 32 interview transcripts (estimated 80-100 hours), I used Claude to apply both deductive and inductive coding:
My Process:
- Deductive coding: Provided Claude with my research questions and asked it to identify relevant quotes
- Inductive coding: Asked Claude to identify emerging themes I hadn't anticipated
- Human validation: Reviewed all AI-generated codes against raw transcripts
- Iterative refinement: Refined prompts based on accuracy of initial coding
Example Prompt
You are analyzing a user research interview transcript. Please:
1. Identify quotes related to these research questions:
- What are the main pain points in current ad platforms?
- How do users evaluate new advertising tools?
- What workflow steps consume the most time?
2. Also identify any unexpected themes or patterns that emerge
3. For each quote, provide:
- The quote itself
- The theme/code it represents
- Why it's significant
Format your response as a structured list.
2. Automated Pattern Recognition Across Transcripts
Used Claude to identify patterns and themes across multiple interview transcripts—a task that typically requires reading all transcripts multiple times:
- Fed Claude 4-6 transcripts at a time
- Asked it to identify recurring themes and contradictions
- Generated initial affinity maps automatically
- Human researcher validated patterns and added nuance
"Claude identified a pattern around 'attribution frustration' that I hadn't explicitly included in my research questions but appeared in 18 of 32 interviews. This became one of our most important findings." — Research Journal Entry
3. AI-Assisted Thematic Synthesis
After completing coding and pattern recognition, I used Claude to help synthesize themes into higher-level insights:
Synthesis Process
- Provided Claude with all coded themes and patterns
- Asked it to group related themes into higher-level categories
- Requested identification of relationships between themes
- Human researcher validated, refined, and added strategic context
AI-assisted synthesis workflow showing human-AI collaboration touchpoints
4. Quantitative Data Integration
Used Claude to help integrate Pendo analytics with qualitative findings:
- Exported Pendo data showing feature usage, clicks, and navigation flows
- Asked Claude to identify correlations between behavioral data and interview themes
- Generated hypotheses about why certain features had high/low adoption
- Validated hypotheses through follow-up conversations with users
Example Integration
Qualitative finding: Users said campaign creation was "too complex"
Pendo data: 67% of users abandoned campaign creation at step 3
Claude insight: Identified that step 3 required data not available until after step 5, creating logical inconsistency
Validation: Follow-up interviews confirmed users were confused by out-of-order workflow
Maintaining Research Rigor with AI
Critical safeguards I implemented to ensure AI didn't introduce bias or reduce quality:
Validation Checkpoints
- Manually reviewed 100% of AI-generated codes
- Spot-checked quotes against original transcripts
- Validated AI-identified patterns with peer researcher
- Conducted member checking with 8 participants
Transparency Practices
- Documented which insights came from AI vs. human analysis
- Shared AI prompts with stakeholders
- Noted limitations and assumptions in reports
- Maintained audit trail of coding decisions
Key Research Findings
1. The "Three Platform Problem"
Advertisers were managing campaigns across an average of 3.4 different advertising platforms, creating significant overhead and data fragmentation.
"I spend my Mondays just logging into different platforms and exporting data to Excel so I can see my total spend. It's ridiculous that there isn't one place to manage everything." — Marketing Manager, E-commerce Company
Pendo Validation: Beta users who could manage multiple channels in our platform had 3.2x higher retention than single-channel users.
2. Attribution Anxiety
The AI-identified theme of "attribution frustration" revealed deep anxiety about proving ROI. Users didn't trust attribution models but lacked alternatives.
- 72% of advertisers didn't trust their current attribution data
- Yet 94% used it to make budget decisions
- Main concern: multi-touch attribution across channels
"I know the attribution is wrong, but what else am I supposed to use? At least it's consistently wrong, so I can track trends." — Media Buyer, B2B SaaS Company
Mixed-Methods Insight: Survey data showed 78% would pay premium for "more accurate attribution," but interviews revealed they actually wanted "attribution they could explain to executives."
3. Automation Paradox
Users wanted automation but feared losing control and understanding of their campaigns.
What Users Said
- "I want AI to optimize my campaigns"
- "Automation would save me so much time"
- "I need help managing hundreds of campaigns"
What Pendo Showed
- Only 34% enabled automation features
- Users disabled automation after 2-3 days
- Clicked "Manual Override" frequently
Claude Synthesis: AI analysis identified the core tension: users wanted "supervised automation" where they maintained oversight and understanding, not "black box automation" that made decisions they couldn't explain.
4. Publisher-Advertiser Empathy Gap
Advertisers and publishers had fundamentally different mental models of how advertising platforms should work, creating design challenges.
Divergent mental models revealed through comparative interview analysis
- Advertisers thought in terms of campaigns, audiences, and conversions
- Publishers thought in terms of inventory, fill rates, and eCPM
- Shared terminology meant different things to each group
5. Mobile-First Reality, Desktop-First Tools
Pendo data revealed surprising mobile usage patterns contradicting user interviews:
- 47% of platform access happened on mobile devices
- But users claimed they "only use desktop" for ad management
- Mobile usage was for quick checks, alerts, and approvals
- Desktop was for campaign creation and analysis
Design Implication: Need different UX for mobile (monitoring/responding) vs. desktop (creating/analyzing).
Recommendations & Impact
Strategic Recommendations
1. Multi-Channel Campaign Management
Build unified dashboard for managing campaigns across multiple ad platforms. Solve the "three platform problem" as core value proposition.
Supporting Data: 89% use 3+ platforms; beta users with multi-channel features had 3.2x retention
2. Transparent "Explainable" Attribution
Focus on attribution models users can explain to stakeholders, not just mathematical accuracy. Provide visualizations showing how attribution is calculated.
Supporting Data: 72% don't trust current attribution; want "explainability" over precision
3. Supervised Automation with Guardrails
Implement automation with clear oversight, guardrails, and the ability to understand/override AI decisions. Show "what would happen" before applying changes.
Supporting Data: High stated interest (78%) but low adoption (34%); users disabled automation features
4. Role-Specific Workflows
Design separate workflows for advertisers and publishers rather than unified interface. Different mental models require different UX.
Supporting Data: Comparative interview analysis revealed fundamentally different user needs
5. Mobile Monitoring Experience
Build lightweight mobile experience optimized for monitoring, alerts, and quick approvals—not campaign creation.
Supporting Data: 47% of sessions on mobile; usage patterns show checking behavior, not creation
Business Impact
Product Decisions Influenced
- Pivoted from single-channel to multi-channel platform based on "three platform problem" finding
- Deprioritized fully automated campaign optimization in favor of supervised automation
- Designed separate advertiser and publisher workflows rather than unified experience
- Added "explainable attribution" as key differentiator in product positioning
AI-Assisted Research Learnings
What Worked Well with AI
- Speed without sacrificing quality: Reduced synthesis time by 40% while maintaining research rigor
- Pattern detection: AI identified themes across transcripts I might have missed (like "attribution anxiety")
- Consistency: AI applied coding frameworks more consistently than manual coding
- Cross-method integration: AI helped connect qualitative and quantitative data effectively
Where Human Judgment Was Critical
- Nuance and context: AI sometimes missed subtle meanings or cultural context
- Strategic insights: Translating findings into business recommendations required human judgment
- Contradictions: Human researcher needed to identify when AI patterns contradicted behavioral data
- Ethical considerations: Decisions about what to share, emphasize, or investigate further
Best Practices I Developed
1. Always Validate AI Output
Never trust AI-generated codes or themes without checking against source material. I validated 100% of AI analysis.
2. Use AI for Pattern Recognition, Not Pattern Creation
AI is excellent at finding patterns you tell it to look for, and surfacing patterns you didn't expect. But the researcher must still interpret significance.
3. Maintain Clear Audit Trail
Document which parts of analysis used AI assistance, what prompts were used, and how outputs were validated.
4. Combine AI Speed with Human Depth
Use AI to accelerate mechanical tasks (coding, pattern matching) so human researchers can spend more time on strategic thinking and stakeholder collaboration.
When NOT to Use AI in Research
- Sensitive topics where empathy and emotional intelligence are critical
- Small sample sizes where manual analysis is feasible and preferable
- Exploratory research where you don't yet know what patterns to look for
- Situations where stakeholders need to trust process over outcomes
Reflection & Learnings
What Went Well
- AI acceleration: Successfully reduced synthesis time without compromising quality or introducing bias
- Mixed methods: Combining qualitative, quantitative, and behavioral data provided comprehensive understanding
- Validation framework: Multiple validation checkpoints ensured AI-assisted findings were trustworthy
- Business impact: Research directly influenced Series A funding and product roadmap
What I'd Do Differently
- Earlier Pendo implementation: Wish we'd started behavioral tracking sooner to have more baseline data
- AI prompt library: Should have documented effective prompts earlier for reuse across transcripts
- Stakeholder education: Needed to better explain AI role to stakeholders who were skeptical of "AI research"
- Competitive analysis: Could have integrated competitive product analysis more systematically
Skills Developed
- Pioneered AI-assisted research methodology with rigorous validation framework
- Deepened expertise in mixed-methods research and triangulation
- Learned to integrate Pendo analytics with qualitative insights
- Improved ability to conduct research under tight timeline constraints
Key Takeaways
On AI in research: AI is a powerful tool for accelerating research synthesis, but it's a tool, not a replacement for researcher expertise. The value comes from combining AI's speed and pattern recognition with human critical thinking and contextual understanding.
On mixed methods: The most valuable insights came from triangulating across methods—when interviews, surveys, and Pendo data all pointed to the same conclusion, we had high confidence. When they diverged, we knew to investigate further.
On transparency: Being transparent about AI use in research actually increased stakeholder trust rather than diminishing it. Showing the validation checkpoints demonstrated rigor.
On speed vs. quality: With the right framework, you don't have to choose between speed and quality. AI helped us move faster while validation checkpoints maintained quality.