In-house build on model APIs
Your own agents and scripts on top of LLM APIs
Engineers on your team write the agents, data pipelines and guardrails themselves, calling model APIs directly.
Verdict: Right only if you will staff it; otherwise you are building what an agent platform already runs.
Who it suits
Teams with GTM engineers who have time to build and, more importantly, to maintain.
Who should skip it
You do not have an engineer whose job includes this next quarter and the one after.
Score per criterion
| Criterion | Score | Why, and how to check |
|---|---|---|
| Data freshness | 3 | As fresh as the data pipeline you build and keep running. |
| Repeat-run reliability | 2 | Model output varies run to run unless you build tests, retries and resumable runs. |
| Human review controls | 3 | Any review flow you want, once you build it. |
| Send-layer safety | 2 | Guardrails exist only where you wrote them. |
| Stack fit | 5 | Built for your stack. |
| Custom signal coverage | 5 | Any signal you can get data for. |
Strengths
- Complete control over signals, data and logic.
- Writes wherever you want.
- Model costs can be low.
Limits
- Repeat-run reliability, review and guardrails are all yours to build.
- Hidden cost is maintenance, not tokens.
- Hard to benchmark without a test harness.
Why it ranks #5
An in-house build scores 5 on stack fit and signals because it can do anything. It scores low on reliability and send safety because those are exactly the parts that take longest to get right.
PipelineBench's finding that 1 in 10 model-and-task pairs passed one run and failed the next is the reason to build a repeated-run test before anything else.
Best for
- Unusual signals no tool covers
- Internal research tools
- Prototypes
Cost
Engineering time plus model API and data costs. For scale: PipelineBench reports model API costs of $9 to $411 for a simulated year of GTM work, depending on the model.
Check before buying
- Write the repeated-run test before the agent.
- Put suppression and Do Not Contact checks in the code that sends, not in the prompt.
- Budget maintenance time, not just build time.