Data & AI solutions
An independent data & AI practice — we design and build the data engineering, cloud platforms, and applied AI that turn raw data into decisions, on Databricks, Azure, AWS, and Kafka.
The stack we build on, in production
What we do
From strategy through to a running platform — engaged for a single audit or as a long-term build-and-operate partner.
Before a pipeline gets built, we map the migration path, the eval bar, and the platform trade-offs so the build doesn’t get re-done in a year.
Python/SQL/Spark pipelines — ETL, CDC, and streaming — that hold up under real production load, not just the happy-path demo.
Platform work on Databricks, Snowflake, Azure, AWS, and GCP — built for the throughput you actually run, not a benchmark slide.
Applied GenAI that ships as a working tool — agentic workflows, RAG, and the evaluation layer that keeps it honest.
Proof, not promises
Real engagements across the industries we’ve worked in — filter by domain.
Structured and semi-structured shipment data needed heavy manual processing, and the client had no proven pattern yet for running LLM agents reliably at enterprise scale.
Ran a POC for "Agents at Scale" on FastAPI + LangGraph — stateful workflows with tool calling, checkpointing, and long-term memory in PostgreSQL — alongside a Kedro + XGBoost pipeline for shipment-success prediction.
Analysts had to hand-write SQL against a warehouse whose schema and table relationships lived in people’s heads, while pricing ML features were rebuilt separately by every team that needed them.
Built an LLM Text-to-SQL app (Streamlit, OpenAI embeddings, ChromaDB) using RAG, knowledge graphs, and data lineage for accurate SQL generation, with LangFuse tracing and an automated eval framework. Led a Databricks Feature Store POC to centralize pricing features, and migrated legacy Postgres pipelines from Avro/Parquet to Delta Lake.
Live-match traffic spikes needed resilient APIs and real-time event processing, plus real-time visibility into security logs at a scale most stream processors aren’t tuned for.
Built RESTful APIs on Spring Boot + Kafka, KSQL transformation pipelines with custom UDFs, and streamed AWS WAF logs into Kafka + Elasticsearch with indexing tuned for 100M RPM — backed by distributed Spark jobs and automated producer/consumer test suites.
Bringing a third-party courier network online meant reliably handling courier creation, updates, and cancellations without blocking the rest of the logistics stack.
Integrated the courier platform via Spring Boot + RabbitMQ, with asynchronous event-driven workflows for the full courier lifecycle.
Deep dive
A composite look at the customer / unified data platform patterns we’ve built and operated — from raw event to activation, tuned for hyperscale traffic.
MongoDB collections · Postgres OLTP · scraped OTA pricing · WAF & clickstream logs
Debezium · Kafka Connect · custom Airflow operators for automated Spark submission
Kafka Streams schema validators · KSQL + custom UDFs · Spark structured streaming — the layer tuned to sustain 100M+ RPM
Delta Lake, migrated off Avro/Parquet · Hive warehouse · governed schema evolution
Databricks Feature Store · knowledge graph + data lineage · Text-to-SQL (RAG, ChromaDB, embeddings)
Dynamic pricing engines · LangGraph agents (tool calling, memory) · Metabase KPI dashboards
Composite of platform patterns built across engagements (OYO, Dream11, McKinsey) — not a reproduction of any single client’s proprietary system.
Schema drift silently corrupts downstream tables
Kafka Streams validation services enforcing schema contracts between MongoDB collections and every downstream consumer.
Elasticsearch indexing chokes under log volume
Tuned indexing pipelines for AWS WAF logs streamed through Kafka, holding up to 100M requests/minute.
Batch warehouses can’t keep pace with streaming volume
Migrated legacy Postgres pipelines off Avro/Parquet onto Delta Lake for reliability and query performance at scale.
Manual Spark submissions don’t scale with pipeline count
Custom Airflow operators automating Spark job submission end-to-end, plus automated test suites for Kafka producers & consumers.
How we engage
We look at your current pipelines, warehouse, and eval gaps to find where a data or AI investment actually pays off — not just where it’s trendy to invest.
Streaming vs. batch, lakehouse vs. warehouse, agent vs. script — the trade-offs get made explicit and documented before anything gets built.
Pipelines, platforms, and AI products on Databricks, Kafka, Spark, Azure, or AWS — built and owned end-to-end, not handed off mid-sprint.
We stay on to monitor, trace (LangFuse), and tune — so pipeline reliability and eval scores keep improving after launch, not just at demo day.
Why ArrowDash
We named this practice after two ideas: an arrow, which only matters once it’s pointed somewhere, and a dash, the burst of speed once that direction is set. Too much data work stalls at infrastructure for its own sake — pipelines with no destination, dashboards nobody acts on, models that never ship.
ArrowDash is built on 6 years of hands-on delivery across McKinsey & Company, OYO Rooms, Dream11, and Sayurbox — shipping GenAI agents, streaming pipelines, lakehouses, and the ML systems that sit on top of them, grounded in a B.E. from Netaji Subhas Institute of Technology, Delhi. We work backward from the decision or product your data and AI investment is meant to enable, then build only what that outcome requires — on the cloud and platform stack you already run.
Every engagement starts with the business decision the data needs to serve.
Deep on Databricks, Azure, and AWS — no lock-in to a stack you don’t run.
We design for the team that maintains it, not just the demo day.
Let’s talk
Whether it’s an architecture review, a Databricks migration, or a custom AI agent — we start with a conversation, not a proposal template.
solutions@arrowdash.orgPick a time on our calendar, right here on the site.