Join our growing team. See the job board →
Insights

Notes on AI, data and building with agents

What we learn building and testing AI systems with expert people, written down for teams doing the same.

A target with green hit marks next to four bars measured against a pass line on a green grid
· 2 min read

How to evaluate an LLM beyond the benchmark

Public benchmarks tell you how a model does on someone else's questions. Here is how to build tests from your own work, probe for safety gaps, and decide when a model is ready to ship.

  • Evaluation
  • LLMs
A central agent box connected by lines to seven smaller agent boxes on a green grid
· 2 min read

What an AI agent needs before it ships

A working demo is not a finished agent. Here are the four things we check, from a clear task to a way to know when it breaks, before an agent goes in front of real users.

  • Agents
  • Engineering
Two document outlines side by side on a green grid, the right one highlighted with a check mark
· 2 min read

Why human feedback still matters for fine-tuning

Models learn from examples, but what counts as a better answer is a human judgement. A look at how expert comparisons turn taste and domain knowledge into a training signal.

  • Human feedback
  • Fine-tuning