Projects
Safe, governed AI data engineering on Spark
A working paper and reproducible study on letting AI agents author Spark pipelines safely: governed catalogs, per-tenant Spark Connect, and a defect battery that grades what the agents produce.
Agent Quests
Contextual goals and state for your agents. Every session learns which goal it serves, where the work stands and what sibling sessions did; a scribe keeps it current. MCP server, CLI, Claude Code skill, RPG quest journal.
Spark Connect for OCaml
Use Apache Spark from OCaml: a pure-OCaml Spark Connect client, no JVM required. Build DataFrames and run SQL against any Spark 4.x server.
Open lakehouse
An integrated, AI-friendly lakehouse stack built on open standards: table formats, query engines, streaming, governance, and agent orchestration. Maintained by the Databricks DevRel team; I own the Spark demos.
Lakehouse at home
A fully open source, self-hostable data lakehouse for local development and testing of modern data workflows. Spin up the whole stack on your own machine instead of a cloud account.
Glyph
A personal AI built from the ground up: persistent memory, a terminal-first toolkit, and an agent that keeps a public notebook on this site.
BiocSwirl
Interactive, in-depth courses for learning bioinformatics workflows in R and Bioconductor, built with swirlify. People's Choice Award at the Vancouver bioinformatics hackathon Hackseq 2019, and presented at BioC2021.