Francisco Carneiro

Senior Data Engineer · Lisbon

Francisco
Carneiro

I start with the business problem, not the technology stack. I build the cloud data platforms companies run on, and I'm now extending that same foundation into the agentic systems running on top of it, across the wider company ecosystem.

Get in touch LinkedIn GitHub GCP · BigQuery · Python AI infrastructure · Automation
01Summary

Five-plus years turning business problems into reliable, production-grade data systems — across industry, utilities, and fintech.

I own data platforms end to end: architecture, ingestion, orchestration, CI/CD, and cost. My work sits where engineering rigour meets business impact — I partner closely with product, analysts, and stakeholders to ship data products people actually depend on.

Grounded in aerospace-engineering rigour, with shipped machine-learning foundations. The active focus: the data and infrastructure layer beneath LLM and agentic systems — retrieval pipelines, orchestration, and the observability that carries them from prototype to production.

02Impact at a glance
/ FINOPS
30%
Platform cost reduced through warehouse & workload optimization
/ SCALE
20+
Cloud data platforms deployed across business units
/ LEADERSHIP
4
Data engineers led as technical lead
/ TENURE
5+
Years across data-driven industries
03Experience
Agicap
Senior Data Engineer
Paris / Lisbon2023 — Present
  • Own business-critical data systems end to end — from scoping and architecture to implementation and production support — across batch and near-real-time workflows.
  • Led platform initiatives spanning CDC, orchestration, ingestion, CI/CD, and infrastructure security, lifting scalability, reliability, and maintainability.
  • Cut platform costs by 30% through data-warehouse optimization, asset cleanup, workload tuning, and cost observability.
  • Partner with analysts, product, and business to ship robust data products, marts, alerts, and integrations for operational and decision-making use cases.
Veolia
Lead Data Engineer
Paris2021 — 2023
  • Spearheaded a Data-Platform-as-a-Service initiative, deploying 20+ cloud data platforms across Veolia's business units.
  • Designed and maintained platform infrastructure and CI/CD pipelines for scalable, repeatable deployments.
  • Built transversal capabilities — alerting, connectors, usage dashboards — that raised reliability and adoption.
  • Managed a team of 4 engineers and advised stakeholders on architecture, workflow design, and data modeling.
AKKA Technologies
Data Architect
Toulouse2020 — 2021
  • Designed an Industry 4.0 big-data platform for industrial IoT, with batch and streaming in a lambda architecture (load-tested at ~1 MB/s).
  • Implemented real-time anomaly detection using supervised and unsupervised machine-learning methods.
  • Delivered a horizontally scalable system serving real operational use cases — alerting, real-time and batch analytics.
04Projects
Independent  ·  Shopify → Pipedrive

Awake — sales automation integration

Designed and deployed automation connecting Awake's Shopify storefront to their Pipedrive CRM — syncing orders and line items straight into the sales pipeline. Killed the manual data entry and kept commercial data in lockstep, so the sales team works from a single source of truth.

Shopify APIPipedriveWebhooksOAuthAutomation
@ Agicap  ·  Fintech

Change-data-capture ingestion

Deployed CDC streams from PostgreSQL to Google Cloud Storage using Airbyte, unlocking near-real-time data flows for business-critical systems.

AirbytePostgreSQLCDCGCSStreaming
@ Agicap  ·  Fintech

Platform FinOps & cost optimization

Led a sustained cost-reduction effort across the data platform — optimizing dbt models, tuning orchestration workflows, and fine-tuning cluster scaling — cutting platform spend by 30%. Separately halved a streaming reporting instance through targeted optimization, cutting its running cost by roughly half.

dbtBigQueryAirflowCluster tuningCost
@ Agicap  ·  Fintech

Churn Me & Downsell Me

Managed Admin APIs that automate company- and client-management processes, and exposed them to HubSpot — so business teams trigger and run these operations directly from their CRM.

Admin APIHubSpotPythonAutomation
@ Agicap  ·  Fintech

Airbyte migration to Kubernetes

Migrated Airbyte from a single-VM deployment to a Kubernetes cluster — moving batch data collection onto scalable, container-orchestrated infrastructure.

AirbyteKubernetesGKEDockerInfrastructure
@ Agicap  ·  Ongoing

AI enablement for analytics

Building an AI context layer over the analytics platform — Looker MCP for natural-language exploration, with structured context across data-engineering repos — so LLM tooling works reliably against real company data.

Looker MCPLLM contextdbtAnalytics
@ Veolia  ·  Enterprise

Data Platform-as-a-Service

Productised a reusable cloud data platform and deployed 20+ instances across Veolia's business units — with shared alerting, connectors, and usage dashboards baked in. One model, many teams, repeatable delivery. Led a team of four to ship it.

TerraformGKECI/CDAirflowLooker
@ AKKA  ·  Industrial IoT

Real-time anomaly detection at scale

Architected an Industry 4.0 big-data platform for industrial IoT — batch and streaming in a lambda design, load-tested at ~1 MB/s — with supervised machine-learning models detecting anomalies live against real operational use cases.

KafkaSparkscikit-learnSpark MLCassandra
Read the paper · IST / JNOS
05Recommendations
  • Francisco quickly became a pillar of the team thanks to his adaptability and excellent communication. Beyond his technical skills, he stood out for his energy, initiative, and proactivity.
    Arnaud RoletTech Lead Data, VWIS · managed Francisco at Veolia · translated from French
  • Francisco is a talented engineer who is passionate about his field, with strong technical skills in problem-solving, software design, architecture, and development. His ability to solve complex technical problems has been an asset to our team.
    Sadeq Zougari, PhD, PMPEU Project & Data Manager, AKKODIS · managed Francisco
  • He took the lead on a new Google Cloud Platform product and quickly built the expertise needed to deliver value to the customer. He's an excellent professional and a great person, always enthusiastic and ready for any challenge.
    Quentin BaretAlliance Manager, SFEIR · worked with Francisco
07Core technologies
Languages & query
PythonSQL
Cloud & warehouse
GCPCloud RunPub/SubBigQueryKubernetesTerraform
Pipelines & orchestration
dbtAirflowComposerAirbyteKafkaSparkGitLab CI/CD
Stores, ML & BI
PostgreSQLCassandrascikit-learnSpark MLLooker
Automation & integration
Dockern8nZapierREST APIs
Languages spoken
Portuguese — nativeEnglish — fluentFrench — fluentSpanish — basic
08Education
  • 2018 — 2020
    MSc Engineering — Double Degree
    ISAE-SUPAERO / Instituto Superior Técnico
    Specialization in computer science and autonomous systems.
  • 2019 — 2020
    Certificate — Data Science, ML & Big Data
    University of Toulouse
    100-hour program covering machine learning and big-data infrastructure.
  • 2015 — 2018
    BSc Aerospace Engineering
    Instituto Superior Técnico
    Recipient of academic achievement awards, 2016 & 2018.