I Made Codex Desktop QA My Pull Requests Every Hour

AI has made us much faster at writing code. It has not given me more hours to verify what that code actually does.

As development sped up, reviewing pull requests became harder. Reading the diff is only part of the job. A UI change still needs to be opened, clicked, and checked in the real app. I wanted that verification to happen as regularly as the code was arriving.

So I set up Codex Desktop to do it every hour. It picks one pull request, builds it in a dedicated QA checkout, works out what to test from the code changes, drives the development app with Computer Use, captures screenshots, and leaves a real GitHub review.

There is no separate QA framework or orchestration service behind the loop. Codex Desktop schedules and runs the task. I wrote one small skill to bridge a missing piece in GitHub’s API, but the reviewer itself is a prompt.

One prompt, running every hour

Scheduled tasks can run against a local project in the desktop app. My task runs in a checkout reserved for QA, so it is allowed to throw away local changes and replace them with the pull request it needs to test.

The prompt starts by narrowing the queue:

Pick one pull request. It must:

- have no merge conflicts
- not already be approved by kdy1

It must also match this condition:

- It was opened by labor0-bot[bot] and is assigned to kdy1.

That gives the task one concrete PR and one decision to make. Before testing, it checks out the latest head and prepares the repository:

Check out the pull request and confirm that the checkout is current.

- You may discard every local change.
- Run pnpm i.
- Stop existing development sessions with pnpm stop:dev or pnpm tap stop.
- Before the first build, run cargo clean so the build starts without cache.
- Start the app with pnpm tap dev --profile local when possible.

The checkout is disposable, but the app environment is not. The production version of The AI Platform may already be running on the same computer, and the task must never touch it. The prompt identifies the development build explicitly:

Use @Computer to perform the manual QA.

- Never interact with the production version of The AI Platform.
- The development app is The AI Platform Dev.
- Its bundle identifier is io.the-ai-platform.app.dev.

The diff writes the test plan

I do not give the task one fixed click path. I tell it to inspect the PR’s code changes and work out the manual verification from those changes.

A pull request that changes a settings form should be tested through that form. A pull request that changes navigation should be tested by navigating. The point is to verify the behavior the PR claims to change, not to run the same generic tour of the app every hour.

Codex then invokes Computer Use to operate the development app. OpenAI’s Computer Use QA guide describes the same basic pattern: name the environment, exercise the important flow, and return evidence that another person can review.

Screenshots are part of that evidence. They show what Codex saw when it made the decision, rather than asking the reviewer to trust a summary alone.

GitHub reviews need somewhere to put the screenshots

GitHub does not provide an API path for uploading an image binary into a pull-request review. A person can drag an image into the web interface, but an API-driven reviewer cannot do the equivalent.

That is why I made the $upload-image skill. The workflow inspects each selected screenshot for sensitive information, then the skill uploads it to a public Cloudflare R2 evidence bucket and returns a URL. Codex embeds the image inline in the GitHub review and adds a caption below it.

The skill is deliberately small. It does not decide what passed, what failed, or which flow to test. It only gives Codex a way to attach the evidence that GitHub’s review API cannot upload.

The review is the output

The final part of the prompt defines three outcomes:

If QA passes:

- Approve the pull request with the manual QA evidence.

If QA fails:

- Request Changes.
- Describe the symptoms and cause in enough detail that the developer can
  fix the problem without running the PR first.

If the pull request changes after QA finishes:

- Do not test it again in the same run.
- Leave the result as a regular Comment, not Approve or Request Changes.
- Finish the task successfully.

Upload screenshots with $upload-image. Embed each image inline and put its
caption underneath.

The head-change rule matters because the review must describe the commit Codex actually tested. If a new commit lands after the test, the evidence is still useful, but it cannot approve or block code it never ran. The next hourly task can verify the updated head.

Request Changes is also not always the end of the loop. When the PR came from Labor0, Labor0 uses Codex’s symptoms, cause, and screenshots to fix the same pull request. A later QA run then gets a new commit to verify.

The awkward cases are in the prompt too

The current setup has several cases that need explicit handling.

Production and development copies of The AI Platform can be running at the same time, so the prompt names the development app and bundle identifier. A PR can change while QA is running, so the prompt falls back to Comment. GitHub cannot receive screenshot uploads through the review API, so the workflow needs $upload-image.

Concurrency is the harder case. Only one QA session may run at a time. If another session is already running, the later task must fail and stop. That gets more complicated when the same scheduled task can run from multiple computers: the task must recognize a session running somewhere else, not only another process on its own machine.

Labor0’s Chatty QA already has related handling for these coordination and evidence problems. Chatty QA is still in private alpha, so it is not a publicly available Labor0 feature today. This is part of the evidence-backed QA direction in Labor0’s public roadmap. We plan to refine these mechanisms and bring them into the broader Labor0 offering later.

For now, the Codex Desktop version is the useful part: one scheduled prompt turns a code diff into a running app, a manual QA pass, evidence, and a review every hour.