The Migration Agent
An AI agent that moves Alteryx workflows to Databricks from a chat prompt. In production at Nike.
The problem
Nike’s supply chain analytics teams ran years of daily work in Alteryx. The platform was moving to Databricks. Every workflow had to be rebuilt in PySpark, by hand, by the analyst who owned it. Weeks per workflow, subtle logic lost on the way, and a per-seat licence ticking for every workflow that had not moved yet.
Migration vendors quote this as a project. I thought it looked like a compiler problem with a conversation on top.
What I built
Through the MSC AI Accelerator, I built an agent that runs inside Databricks. An analyst opens a chat, points it at an Alteryx workflow, and gets back a runnable Databricks notebook, the documentation for it, and a record of what was decided along the way.
The part that does the heavy lifting is not the LLM. It is a deterministic compiler I wrote and open-sourced as alteryx2dbx: it parses the Alteryx XML, resolves the execution order across 34 tool types, and emits PySpark with every mapping explicit. 608 tests hold it to that. Anything it cannot convert gets flagged, never silently dropped.
The LLM sits around that core and does the three things a compiler cannot:
- Hold the conversation. “Migrate this one, same conventions as last week’s” means something, and the agent remembers across sessions what those conventions were.
- Resolve the ambiguous bits with the analyst instead of guessing. Null handling, case sensitivity, a formula that only makes sense with the business context. It asks.
- Write the docs. Every migrated workflow gets a page describing what it does, what changed, and where to look when it breaks.
That split matters. Pure-LLM migration tools invent PySpark functions that do not exist and quietly change how nulls behave. You end up debugging generated code anyway. Putting the compiler underneath means the output is the same every time and the LLM only spends its judgment where judgment is needed.
What changed
Migration went from a weeks-long rewrite to a conversation and a review. The agent runs in production for the EMEA supply chain analytics team, on the platform the analysts already use, with no per-seat licence left behind once a workflow has moved.
The build is not the story I tell about it, though. Shipping it was the easy half. Getting analysts to move their daily work onto a new tool is the half I am still doing, and it changed what I count as done. Done is not deployed. Done is someone using it every week without me in the room. I took that gap to the head of the AI-for-Operations lab as the problem worth owning, and now I sit with the analysts before I write a line.
What I learned
Launch Ready taught me that most problems do not need AI. This one did, but only in a small, specific place. The right shape was a boring, tested core with a model at the edges, not a model in the middle. And the harder engineering problem, the one nobody staffs, is the last mile between a working tool and a changed habit.