Other · Advertisement Kpi Automation Platform
Decrypt
AI-powered Advertisement Intelligence & Neuromarketing Analytics Platform.
The brief
Build a proprietary advertisement intelligence platform for Decrypt Marketing Services that automates the measurement of 355 neuromarketing and consumer insights KPIs across visual, audio, and textual dimensions of ad content—replacing manual research workflows with an AI-powered analysis engine, while delivering a multi-role SaaS dashboard for study creation, analysis management, and client reporting.
What we built
A full-stack advertisement effectiveness measurement platform built for a Mumbai-based market research firm (Decrypt Marketing Services). The system has two layers:
- A React-based multi-role SaaS dashboard enabling researchers and clients to create studies, upload ad stimuli, trigger analysis, monitor processing, and download static/dynamic output reports with researcher-adjustable threshold controls and speedometer/graph visualizations.
- A Python/FastAPI multimodal ML pipeline that automatically scores 355 advertisement KPIs across visual (SAM3, YOLOv8, MiVOLO, RetinaFace, InsightFace), audio (Librosa, Demucs, wav2vec2, Pyannote), text (GCV OCR, RoBERTa, Gemini Flash 2.5), and neuromarketing signal (EEG + eye-tracking) dimensions—with results auto-populated into client Excel reports. The 355 KPIs were clustered into 42 validated groups across 4 execution phases covering scene environment, demographics, brand visibility, color psychology, audio psychoacoustics, and emotional response.
Key deliverables
Deep consumer insights & market understanding across FMCG and non-FMCG sectors. End-to-end decision-making support using System 1 and System 2 research tools. Custom research solutions for buying behaviour, shopper journeys, and brand perception. Actionable strategy and optimisation based on quantitative and qualitative data. Brand growth support through ML-enhanced forecasting and deep audience profiling.
Production deployment confirmed: backend API live with CORS configuration, study processing pipeline operational (with sound/video rendering fixes applied), UI deployed and smoke-tested. Phase 1 pipeline (visual segmentation, face analysis, color metrics, audio extraction + transcription, text OCR) validated on real advertisement content including the Dhurandar trailer. EEG+ET output pipeline stable and async. Client-facing dashboard with study creation, analysis output, static/dynamic report downloads functional.
Delivery timeline
How it was built, phase by phase.
8 workstreams across 11 weeks of operated delivery.
- buildWeek 1–6 (Mar 2 – Apr 7)
Dashboard & Navigation Architecture
Multi-role responsive dashboard with KPI cards, sidebar navigation, header, study/analysis routing, global theming, and role-based access control built for a research analytics platform.
Fully themed, multi-role dashboard with study creation flow, analysis landing pages, error handling, and role-adaptive UX delivered across 6 weeks.
ReactFramer MotionCSS Design TokensPython - buildWeek 1–11 (Mar 2 – May 12)
Computer Vision & Segmentation Pipeline (SAM3 / YOLO)
Multi-model visual analysis pipeline using SAM3 for segmentation and YOLOv8/v12 for object detection across image and video ad content.
Phase 1 visual pipeline covering segmentation, object detection, logo/brand visibility, scene classification, human/clothing detection.
SAM3YOLOv8YOLOv12PythonOpenCVTriton - designWeek 1–6 (Mar 25 – Apr 17)
KPI Taxonomy, Corpus Design & Phased Roadmap
Systematic reduction of 355 client-provided KPIs into clustered, labeled, feasibility-assessed groups—mapped to processing type (visual/audio/text), assigned corpus IDs.
Structured KPI taxonomy enabling phased automation of advertisement effectiveness measurement; became the master spec for all pipeline development.
ExcelPythonColor-coding logicCorpus ID framework - deployWeek 1–8 (Mar 2 – Apr 24)
Backend API, Job Queue & Pipeline Infrastructure
Python/FastAPI backend covering study/results APIs, CORS configuration, job queue reliability (retries, idempotency, timeouts), async task handling, blob size optimization, serialization.
Production-deployed backend with robust study processing pipeline, optimized storage, async execution.
PythonFastAPIJob QueueREST APIUbuntu Server - integrateWeek 2–6 (Mar 9 – Apr 8)
Studies & Analysis Workflow Engine
End-to-end study lifecycle management: create study (multi-step stepper), trigger processing, monitor status, surface results, and complete/redirect—with state-aware UI visibility logic.
Functional study lifecycle from creation through analysis output, with conditional navigation and state-preservation across modules.
ReactREST APIPythonFastAPI - buildWeek 3–11 (Mar 20 – May 11)
Audio Analysis & Multimodal Pipeline
Audio extraction from video, transcription (Whisper, Demucs, wav2vec2), speaker diarization (Pyannote), audio KPI computation (3D sound, beat-cut sync, silence, A/V mismatch), and integration into master KPI output.
Functional audio pipeline from extraction through KPI output: transcription, speaker segmentation, music isolation.
LibrosaWhisperDemucswav2vec2PyannoteAST - buildWeek 5–8 (Apr 6 – Apr 21)
Face Analysis & Demographic Intelligence
Multi-model face detection (RetinaFace, InsightFace, DeepFace), emotion recognition, age/gender estimation (MiVOLO fine-tuned), ethnicity classification.
Deployed face analysis module covering detection, emotion, age, gender.
RetinaFaceInsightFaceDeepFaceMiVOLOYOLOv8Python - buildWeek 7–9 (Apr 17 – Apr 29)
Text / OCR Pipeline & Semantic Analysis
Textual KPI extraction from advertisement frames using Google Cloud Vision OCR, followed by RoBERTa for semantic analysis and multimodal LLMs (Qwen, Gemma, Gemini Flash 2.5 Pro) for OCR and contextual understanding.
Text extraction and semantic analysis module integrated into master Phase 1 pipeline with automated Excel KPI population.
Google Cloud Vision OCRRoBERTaQwenGemmaGemini Flash 2.5 ProLlama 3.2
More case studies
Related work
09 · Run a function
Stop renting hours. Start running functions.
Pick the function you want off your plate. We'll map the brain and name the outcome we'd commit to — before you do.