Built an app with AI and wondering who checks each change before it reaches your users? At Awakast part of that work is done by two instances of Hermes Agent, the open-source AI agent from Nous Research. One, Simon, writes code. The other, Eryk, reviews it. An experienced engineer then tests and accepts every stage. Below is how that works on one real pull request: sent back four times, merged in just over two hours.
Like any developer here, Simon and Eryk have company email addresses, accounts in our tools and their own permissions. They argue about code in review comments, and when they cannot agree, they call Kasper, our founder.
They did not replace our engineers: they joined a team where it is clear who does a task, who checks it and who makes the final call. For a founder, that is the difference between "the AI added something" and a change that someone can accept or reject. For a CTO, it means every change can be traced back to how it was made.
Why the agents got accounts instead of another chat window
We did not invent a separate process because some of the work is done by AI. The agents got what a new developer gets: an account, a defined set of permissions, a task board and a person to go to when something is stuck.
Tasks live in YouTrack, a task board similar to Jira. Code and code discussions live in Gitea, our self-hosted equivalent of GitHub. The tools matter less than the principle: the task, the work and the review each have their place, instead of disappearing into a chat history.
That matters to whoever is paying for the software. "Done" can mean the agent finished writing, the second agent approved the change, or an engineer tested it. For us these are three separate moments, and accepting a stage is never left to the agents alone.
Setup was the easy part. Nobody configured Hermes by hand: we told another AI that Hermes was installed, and Hermes configured itself.
Who does what: Hermes Agent in two roles
Simon is the developer. Eryk is the reviewer. They are separate roles with different permissions.
Simon can write code and submit it for review. Eryk cannot write code. He can review it, request changes and, once he approves, merge the change into the stage branch: a separate copy of the code set aside for that stage of work.
Kasper settles disputes the agents cannot close.
One task, from the board to a reviewed stage
An engineer starts the work. They choose a stage for the agents and create a branch for it, a working copy of the application where changes are made before they are accepted.
- Simon picks up the next available task. When he is idle, he takes the next task from YouTrack that can be started now, skipping anything still waiting on another task.
- Simon writes the code on his own branch and opens a pull request: a proposed change the other side can read and comment on.
- Eryk starts the review automatically as soon as the pull request appears. He leaves comments, Simon answers and fixes. The discussion sits next to the code it is about, where anyone can find it later.
- An unresolved dispute goes to Kasper. If there is no agreement after five rounds, the agents call him on Slack.
- Eryk merges the change into the stage branch, and Simon updates the task.
- An engineer signs off on the stage. They run their own review and tests, then decide whether the stage goes into dev, the shared development branch. Eryk's approval does not replace that decision, and for a client this is the step that matters most.
What it looked like on a real change
The example comes from our own recruitment product. The task: a new search results and candidate detail view, where a recruiter can see why a candidate made the list, add a note and correct contact details.
Simon committed the first version on October 26 at 10:57 p.m. Warsaw time. Eryk sent it back for changes four times; the pull request collected 11 comments, and the fifth round was approved. Between the first version and the merge, Simon pushed five fixes:
- Less explaining. That evening a new rule went into our design system: the interface has to make sense without explanatory text. Simon applied it to his screen and removed an explanatory subtitle and a help tooltip.
- Two edits saved at once. When a recruiter saved a note and corrected candidate details at the same time, whichever save finished last could overwrite the other. After the fix both are kept, and a test covers both orders.
- Filters that tell the truth. The search summary listed keywords as applied even when the keyword filter was off. Now they appear only when they are actually in use.
- A draft that does not vanish. A contact address typed but not yet saved was lost when the recruiter saved an unrelated note. Now it survives the note save, and the recruiter can then save it, retry after a failed save, or cancel back to the saved address.
- Text in four languages. After the changes were combined, one set of interface texts overwrote another. Now both work in all four language versions, and a test checks it.
The change was merged into the stage branch on October 27 at 1:06 a.m., just over two hours after the first version, at night, without waiting for anyone. The escalation rule never triggered: four rounds is within the limit of five.
Three of these are bugs a demo does not show: they appear only when a real recruiter saves two things at once or switches a filter off. All five were fixed during the agents' review, before the stage reached an engineer.
What separate accounts and permissions give you
Separate accounts mean Gitea shows who submitted each change, who reviewed it and who accepted it, so when you ask why something is in the code, there is a name and a comment to point to.
Eryk cannot write code, and only a person can move a stage into dev. It is the rule you would set for any team: everyone does their own part of the work and nothing after it.
Where it took some breaking in
Agents can overcomplicate a simple job: ask for a small fix and you may get a proposal to restructure the architecture. Our engineers keep the fix small and decide whether the bigger change is needed at all.
The second problem is length. A review comment can be longer than the change it is about. Someone has to separate the remark that actually changes the code from the reasoning that does not help close the task.
We had to refine the instructions and learn to work with a team like this. Some of that was about the agents, and some was about how we prepare work for them and judge the result. The main lesson: keep "make this change" separate from "do we need this change". Simon picks his own tasks from the queue, but he does not set the direction of the product.
What this means if you bring us an app built with AI
A working MVP is where the conversation starts. What it needs next is a decision on what to keep, what to fix and who owns development from here.
At Awakast that work is done by experienced engineers supported by agents. People choose the stage, sign off on the result and make the decisions that matter for the product and the business.
If you only have an idea, the first conversation is about what the product should change in your business. If you already have an app, we start with what works and where it breaks: this is how we check whether an AI-built app is ready for production. Either way, you will not need to read a pull request.
Is there any reason not to find out how we'd do it? Show us what you've built or tell us what you're building. A few sentences, and you'll know how we'd tackle it.