DataPilot -- Private AI Data Analyst

Overview

DataPilot lets anyone analyse a dataset by describing what they want in plain language. Instead of guessing the answer, it writes the actual Python code (pandas, numpy, matplotlib) to preprocess, analyse and visualise your data, runs that code against your file, and returns a report: charts, printed results, and the code itself so everything is auditable. You then refine the report by asking follow-up questions.

It is built for organisations that cannot send their data to a cloud AI. The analytics is done by code running on your server, and the AI that writes the code can be a local model, so raw data never leaves your network.

How it works

Each request goes through a short pipeline. The AI is only ever given the structure of your data (column names, types, and a 5-row sample) to write the code. The full dataset is only ever touched by the code that runs locally.

your request + dataset
        |
        v
1. AI writes Python   (sees only: column names, types, 5-row sample)
        |
        v
2. Sandbox runs it    (against the FULL data, locally:
                       no network, 30s timeout, memory cap)
        |
        v
3. Report             (charts + printed results + the code)
        |
        v
4. You refine         (follow-up prompts build on prior steps)

Charts and numbers are produced by the executed code, not invented by the language model. That is what keeps the results trustworthy: the AI reasons about how to analyse the data, but the figures come from real computation.

User workflow

Using DataPilot takes four simple steps:

Privacy & security

Privacy is the core design goal. Two things keep your data safe:

• The AI never sees your rows. To write the code, the model receives only the column names, their inferred types, and a 5-row sample. Your full dataset is never sent to the AI.
• The data is only processed locally. The generated Python runs on your own server, against the full file. Results (charts, numbers) are returned to your browser; the raw data never leaves the machine.
• The code runs in a sandbox. Generated code executes in an isolated process with no plt.show, a 30-second timeout, a memory cap, and a denylist for dangerous operations (network, subprocess, file deletion). Production deployments add container/nsjail isolation with no network.
• The AI itself can be local. The code-generation model is a swappable layer. Point it at a local model (Qwen2.5-Coder, Llama, Mistral via Ollama or vLLM) and the whole system is on-premise and air-gappable.

Self-correction loop

AI-written code is not always right the first time. If the generated code throws an error, DataPilot feeds the traceback back to the model with the failing code and asks it to fix the problem, then re-runs it. This repeats up to three times. When a fix succeeds, the report is shown with an "auto-fixed an error" note so you know it happened.

What you can ask

Anything you would ask a data analyst to do with a table, for example:

It handles preprocessing (type conversion, missing values, date parsing), aggregation, statistics, correlation, simple modelling (scikit-learn), and any matplotlib visualisation.

Running fully local

For a private deployment, two pieces run on the customer's infrastructure:

1. The code model -- served with Ollama or vLLM. A 7-8B coder (Qwen2.5-Coder 7B, DeepSeek-Coder) runs on a modest GPU or on CPU; a 14-32B coder is sharper and wants a GPU. Nothing is sent to any external API.

2. The execution sandbox -- a locked-down container (Docker / nsjail) with no network access and strict CPU, memory and time limits, where the generated code runs against the data.

The result is an analyst that an enterprise can put behind its own firewall with no data egress.

Limitations & notes

• - A smaller local model writes slightly less polished code than a top cloud model; the View code button lets you review what ran.
• - Very large datasets should be sampled or pushed to a database step; the demo targets files that fit comfortably in memory.
• - Generated analysis is a strong starting point, not a substitute for domain review of important decisions.
• - This hosted demo runs code-gen on a cloud model for speed and uses a representative sample dataset.

FAQ