AI Data Analysis

Problem

After Propellyr pivoted into generative AI, the bet was that analysis should not require SQL or a data team. Users had CSVs and documents and questions, and nothing in between.

Solution

Built an AI data analysis application where users upload a CSV and ask questions in natural language. The app generates SQL against DuckDB, then feeds the results back to the model to draw conclusions and generate Python code for visualisations. Also built a RAG pipeline that extracts requested data points from unstructured files based on a user’s query.

Architecture

A Python/FastAPI backend and Next.js frontend on AWS. Uploaded CSVs are processed into DuckDB tables; natural-language queries are turned into SQL by the model, executed, and the results are summarised with generated Python for charts. For unstructured data, a processing service extracts text, chunks it, and stores embeddings in a vector database; the RAG extractor expands an input metric into embedded queries, searches the store, and answers from the extracted data.

Engineering decisions

• DuckDB for embedded, file-based analytics instead of standing up a data warehouse per upload
• Generated Python for charts so visualisations were code, not hand-configured dashboards
• A metric-to-queries expansion step in the RAG extractor, since one embedded query rarely covers a full data point

Technology

Python · FastAPI · Next.js · DuckDB · AWS

Results

• Contributed to successful fundraising after the pivot
• Established the company’s technical foundation in generative AI

What Daniel learned

Letting the model write both the SQL and the chart code worked better than expected. The guardrails that mattered were the schema context going in and the result summary coming out.