Experience#

A selection of my work since 2019, grouped by domain. Most engagements were hands-on builds; the ones marked Advisory were reviews and workshops. Each entry gives the context, what I delivered, and what the client said. Publications are on the Research page.

Consulting projects#

AI, LLMs and NLP#

LLM-Powered Data Analyst for Rice University#

Context: An LLM-powered data analyst enhances decision-making by quickly analyzing complex datasets, revealing patterns, and delivering actionable insights. This boosts efficiency, minimizes manual effort, and democratizes access to advanced analytics, resulting in more informed and timely decisions. The HR department at Rice University has shown interest in implementing such a system.

Contributions: As a Data Scientist, I created an agent-based system to translate natural language questions into SQL queries by integrating large language models (LLMs) with our database schema and data distribution. However, due to the limitations in LLM accuracy, we shifted to a simplified schema incorporating common table expressions (CTEs) to establish better control. This change significantly improved the neural typeahead and SQL generation for data analysts, resulting in more practical and high-quality outcomes. The system shipped as a Snowflake Streamlit app that automates simple queries and reduces analysts' workload.

Agentic Cybersecurity Assessments#

Context: Security teams face a constant stream of events from many sources. Turning them into a clear picture of risk, and into actions worth taking first, is slow manual work.

Contributions: I built an agentic orchestration system that acquires and analyzes security events, identifies risks, and prioritizes remediation actions with explanations that security teams can verify.

Voice-to-Text Agent Benchmarking#

Context: Voice agents depend on accurate speech-to-text across languages. Comparing transcription quality across locales is hard: spoken numbers, dates and names are written differently, and some languages need romanization before outputs can be compared.

Contributions: I built a multilingual evaluation framework for a conversational AI company, spanning 20+ locales and handling inverse text normalization, romanization, and cross-locale comparability.

AI in Smart Contracts for Decentralized Finance#

Context: Smart contracts can utilize the outputs of AI models to orchestrate and direct on-chain transactions. Extending the trust modeling of blockchain, how can you verify that the outputs from a machine learning inference API are trustworthy and produced by the model you're paying for, rather than being generated by a less expensive model or fabricated?

Contributions: As a Data Scientist, I contributed to the development of a robust verification framework for authenticating machine learning predictions, addressing the issue by generating cryptographic fingerprints at key inference stages. Through a concrete demonstration of the prover and verifier protocol, I showcased its applicability to various ML inference processes, including ensembles, agentic workflows, streaming, and batching of inferences.

Text Classification for Regulatory Compliance#

Context: Local Planning Authorities (LPAs) in the UK rely on written representations from the community to inform their Local Plans which outline development needs for their area. With an average of 2000 representations per consultation and 4 rounds of consultation per Local Plan, the volume of information can be overwhelming for both LPAs and the Planning Inspectorate tasked with examining the legality and soundness of plans.

Contributions: Streamlining the text analysis for both Local Planning Authorities and Inspectors, using fine-tuned LLMs to pre-fill text categories & topics. Simulations demonstrated that the percentage of correct predictions increases from 61% to 89% as the ratio of verified labels is expanded from 10% to 50%, significantly increasing the planning effectiveness. Simulations indicate savings of 13–26 FTE days.

Recommendation Systems for Researchers#

Context: Researchers must figure out what are the most relevant journals to publish their work. Publishers are in the unique position to push recommendations.

Contributions: As a Data Engineer and Data Scientist, I researched and implemented a family of recommender systems (SQL-based probabilistic models) being used in 400 community pages and serving 5m researchers with a CTR of 5.6%. The latest version combines submission history, fine-tuned LLMs, vector embeddings, and statistical models. Recently, implemented improvements to the algorithm logic, with a CTR increase of 36%.

Forecasting and risk assessment#

Monitoring and Assurance for HK Development Bureau#

Context: The Government of Hong Kong oversees over 500 active capital projects, spanning various infrastructure and development initiatives. To manage the large volume of data and complexities, there is a critical need for advanced automation tools to detect anomalies and outliers. These tools will enable timely issue identification and quicker intervention.

Contributions: As a Data Scientist, I developed and implemented a new set of forecasting models that leverage historical data on projects with varying durations and budgets for training. Our simulations showed that these new models reduced errors in cost and schedule overruns by over 20%.

Railway Construction Cost Forecasting#

Context: High Speed 2 (HS2) is a high-speed railway which is under construction in England. The project has become controversial due to delays and cost increases, with cost projections in the £35-45bn range. Planners need up to date estimates for monitoring and assurance.

Contributions: As a Data Scientist, I have trained forecasting models by leveraging empirical data from both current and past projects, reporting the forecasted project progress during mid-delivery and comparing the results with existing Reference Class Forecasting (RCF) estimates. The models combine conformal prediction, transfer learning and machine learning. These forecasts are being used by the senior leadership to direct future strategic moves.

Performance Forecasting for Megaprojects#

Context: A megaproject is a large-scale, complex venture that takes many years to develop and build, involves multiple stakeholders, and impacts millions of people. Examples include building a new train station. Effective management requires monitoring progress, forecasting delays and costs, and making budget adjustments.

Contributions: Development of an AI-based early-warning system for spotting high-risk projects, including outlier prediction and forecasting future portfolio spending. The system uses data from a total of 2,700 years of combined construction activity, with an aggregate cash flow of USD 60bn.

Tree-Fall Risk Scoring for Urban Forests#

Context: Cities manage thousands of trees with limited inspection capacity. Falling trees put people and property at risk, so inspections should go where the risk is highest.

Contributions: I developed an explainable tree-fall risk scoring and prioritization system for two Italian cities, combining tree inventories, maintenance logs, and ERA5-Land reanalysis weather data to prioritize inspections.

Contact Center Forecasting for Furniture Online Shop#

Context: In contact centers, forecasting involves predicting future demand for communication channels like phone, chat, and email, as well as determining the number of agents required to handle that volume. The process relies heavily on analyzing past data on volume, which can reveal patterns such as seasonal fluctuations and long-term growth or decline.

Contributions: Research and deployment of a family of AI models to forecast the phone contact volume, reducing the mean absolute percentage error (MAPE) by 30% with an estimated business impact of USD 5m-10m in savings per year.

Demand Forecasting for Textile Industry#

Context: Forecasting demand is crucial in manufacturing, particularly in the textile industry, to optimize production. The textile production process is intricate and requires careful planning and execution, taking into account factors such as staffing, logistics, and various stages of fabric production, including pre-treatments, dyeing, printing, and finishing. This process can take several months. Manufacturers aim to minimize delivery times and unsold stock to remain competitive, which means producing the appropriate quantity at the appropriate time.

Contributions: As a Data Scientist, I conducted a thorough examination of historical data points through exploratory data analysis. I identified data quality concerns and identified potential opportunities to integrate additional data sources to enhance the accuracy of the demand forecasting model. I developed and evaluated the model, and finally, presented the findings in a detailed report. The technologies used for this task were AWS Forecast, Jupyter Lab, Pandas, and Matplotlib.

Mobility and location analytics#

Optimization in Public Transportation#

Context: In urban public transportation, bus routes and schedules are frequently modified to align with service demand, fleet and driver availability. These changes are typically carried out through manual processes and rely on the expertise of individuals, which can lead to suboptimal decisions being made.

Contributions: As a Data Scientist and Transport Engineer, I designed a comprehensive solution that estimates demand, simulates fleet operations to analyze various scenarios, and determines optimal adjustments to the service. The solution aims to improve key business performance indicators such as network service costs and profits, driver mileage, passenger waiting times, and CO2 emissions. Technologies used in this project include SUMO, Spark, Origin-Destination estimation, and simulation-aided optimization.

Adaptive Traffic Signal Analysis & Optimization#

Context: Urban traffic modeling and analysis is a critical aspect of traffic management and control. Its goal is to anticipate congestion states and suggest improvements to the traffic network. One such improvement is traffic signal control, which aims to reduce the travel time of vehicles by coordinating their movements at intersections.

Contributions: As a Data Scientist and Research Engineer, I constructed an urban traffic simulation model utilizing noisy sensor measurements and used reinforcement learning techniques to determine the best traffic light plan. The technologies used in this project include Flink, Kafka, SUMO microscopic agent-based traffic simulator, Java, Python, Jupyter Lab, NetworkX, probabilistic modeling methods, and simulator-in-the-loop optimization strategies.

City-Scale People's Movement Analytics#

Context: Mobile devices establish regular connections to the cellular network to receive messages, make calls, and transfer data. The logs generated by these connections offer a detailed view of the population's movement in urban areas and can be used for a variety of purposes, such as optimizing out-of-home advertising.

Contributions: As a Data Engineer and Data Scientist, I designed and developed a GDPR-compliant analytics service that offers population demographics information for urban areas by analyzing people's location and movements using cellular network data. The service ingests billions of records per day. The technologies used in this project include AWS EMR clusters, AWS S3, AWS Lambda, AWS CloudWatch, HDFS, Parquet, Hadoop, Spark, Scala, Python, Zeppelin notebooks and Docker.

Location Intelligence for Retail Analytics#

Context: WiFi radio signals emitted from devices such as smartphones can be used to determine their location within indoor environments. This data can be used in retail analytics to identify high-traffic areas, evaluate the success of promotions, and inform the placement of products for improved sales.

Source: Telefónica Germany press release, May 2017

Contributions: As a Senior Data Engineer and later Lead Data Scientist, I restructured the WiFi analytics pipeline using Spark jobs, resulting in more efficient data and ML pipelines and an improvement of double-digit percentage accuracy. Technologies used included PostgreSQL, Airflow, Celery, Cassandra, AWS S3, Python Docker, JupyterLab, Scikit-learn, Pandas, Fabric, and Flask.

Public Health and Contact Tracing for COVID-19#

Advisory

Context: Contact tracing, when implemented consistently, can interrupt the spread of infectious diseases and is a crucial public health strategy for managing disease outbreaks. By integrating mobile phone usage data with other information, public health organizations can assess the situation on a country-wide scale, detect potential risks such as crowded areas and track the contacts of potentially infected individuals.

Contributions: As a Data Scientist and Research Engineer, I evaluated the methodology and algorithms of an existing contact tracing system. Based on my findings, I provided a series of recommendations that have been implemented, resulting in improved performance and more accurate outcomes. The technologies used in this project include ElasticSearch, Python, Pandas, data cleansing, and trajectory mining algorithms.

Machine Learning Workshop for Mobility as a Service Provider#

Advisory

Context: Mobility as a Service (MaaS) encompasses a variety of digital tools aimed at streamlining transportation for both passengers and transport operators. These tools include trip planning, booking, ticketing, payment, and updates for passengers, as well as fleet management, demand forecasting, predictive maintenance, and optimized fixed and demand-responsive transit for operators. In order to facilitate the Company's transition to a MaaS provider for transport operators, I proposed and led a workshop on this topic.

Contributions: I led a three-day remote workshop for 20 participants, comprising of sales and management personnel, introducing them to the concepts of Big Data, including its motivations, architectures, and strategies for integrating with legacy systems. The workshop also covered the topic of Machine Learning, including challenges in implementing ML services and opportunities in the field of mobility. We delved into specific use cases of relevance to the Company. The workshop concluded with a discussion on best practices for managing data products.

Business analytics#

Discovering and Mitigating Inaccuracies in Tax Filings#

Context: In certain financial services, such as tax filing, refunds are granted to customers based on their responses to questionnaires. However, these responses may not be entirely accurate or consistent, which could impact the refund calculations. To address this issue, mitigation strategies include reducing uncertainty through bounds, providing estimates with quality guarantees, automatically correcting inconsistencies, and manually reviewing and correcting any discrepancies.

Contributions: As a Data Scientist and Machine Learning Engineer, I assisted in defining the business problem, acquiring and integrating the training data, constructing predictive models, conducting experiments with domain experts for validation, and implementing the prediction APIs in production systems. The technologies used for this project were Google AI platform, Google App Engine, Flask, Kubernetes, Snowflake, Jupyter Lab, Scikit-learn, and various ML algorithms including regression, binary classification, multi-class classification, and quantile prediction.

BI and NLP for Tax Filings#

Context: Tax refunds often go unclaimed because the relevant details are buried in millions of heterogeneous documents from the tax office.

Contributions: I delivered BI and NLP tools that enable rapid analysis of millions of tax documents from the German Federal Tax Office, improving the identification of overpaid income taxes, such as missed childcare deductions.

Multi-Touch Marketing Attribution Modeling#

Context: An attribution model is a system of rules that assigns credit to various touchpoints in a customer's journey to conversion. These touchpoints may include clicking ads, visiting blog posts, using referral codes from influencers or partners, and organic search. Attribution models vary in the way they weigh the significance of different touchpoints, such as first-click, last-click, and multi-touch. Understanding a customer's behavior leading up to conversion is essential for marketing departments to effectively measure and optimize their operations and allocate their budget.

Contributions: As a Data Scientist, I developed a multi-touch attribution model that included data cleaning and integration, a tracking model for extracting information on converted visitors, an attribution model that assigns credit to visits and users, and marketing analytics to provide insights. The technologies used for this project were Piwik/Matomo data model, MySQL, PostgreSQL, SQLite, custom-built visit-user matching with data provenance, and handling of missing attribution data.

Visual Analysis of Pharma Business Processes#

Context: Pharmaceutical companies are subject to stringent regulations regarding the handling of confidential information, product development, and management of medical trials. These processes can be complex, interconnected, and hard to grasp. Analyzing the recorded logs of business processes is a valuable tool that can reveal these complexities and provide insights that can be used to drive automation and optimization through analytical services.

Contributions: As a Data Scientist, I developed an application component that allows for the visualization of business processes and emphasizes different areas of interest. This component has been integrated into a comprehensive Business Intelligence dashboard. The technologies used in this project include Node.js, Vue, TypeScript, and yFiles for diagramming.

Entrepreneurship#

Cloud Robotics & Drone Charging Stations#

Context: To function properly, fully autonomous robots need remote management services such as fleet management, predictive maintenance, and teleoperation, as well as the ability to charge autonomously for extended and unattended operations.

Contributions: As the founder and CTO, I oversaw the creation, development, and launch of charging stations and protective hangars for commercial drones, along with related management and connectivity services. My efforts resulted in contracts with over 30 international clients, including the CIA, NASA, NIH, Google X, Parrot, and Stanford University. Technologies used included ROS, Python, C++, Flask, Docker, Fabric, OpenCV, and Jupyter Lab.

Speaking#

I am a frequent speaker at international events, including PyData Berlin, MLOps Munich, ML Week EU, and EuroPython.

• MLtraq: tracking at hyperspeed (slides, April 2024)

Technology stack#