Two different jobs
Creating digital products and software has become far more accessible with the help of generative AI, especially coding agents like Codex or Claude Code. A founder can write a prompt describing a product or a feature, ask a model to implement it, and receive working code. That speed is fantastic, but it can also create false confidence. Developers, and anyone who writes code, should still pay close attention to the code they ship.
The main risk is often not code that is broken outright. It compiles, looks reasonable, and works during a basic test, but it might not fit the wider system. This does not mean AI-generated code is always poor quality — state of the art models are often very good at generating code, especially for popular languages and frameworks with robust documentation, such as TypeScript and React. What it means is that generating code and verifying code are two completely different jobs.
Here are some common problems we encounter when using AI coding tools.
The model uses outdated frameworks and libraries
A model may generate code based on an older version of a framework, library or API. This can happen because the model was trained on code and documentation from previous versions, and newer ones were released after its training cutoff date. Despite the fact that new models and coding tools can search the web or read documentation, they still may choose an outdated option unless they are given clear specifications.
This can show up as a deprecated method, an old configuration format, a package that has been replaced, parameters that are no longer supported, or an API that does not exist in the installed version.
These problems are usually easy to notice when the code fails to compile. The situation becomes harder to spot when the old API still works but behaves slightly differently from its newer equivalent.
Models might not choose the correct version of the framework despite writing "use the latest version" in the prompt. A better approach is to give the model the actual dependency information from files such as package.json, package-lock.json or pyproject.toml. You can also provide a link or an extract from the relevant official documentation.
Compare a vague instruction, such as "create an authentication endpoint using Next.js", with a precise one: "create an authentication endpoint for a Next.js 16 project using the App Router; use the dependencies listed in the attached package.json; do not use APIs that are missing from these versions."
Asking a different model to review the code or answer might help, but it should not be the only form of verification. The generated code should still be checked by running the compiler, the linter and automated tests.
Code that works only on the happy path
AI-generated code often handles the expected sequence of events but overlooks partial failures or minor bugs. Take an order-processing function as an example, where the payment is collected first and the order is then updated in the database.
async function processOrder(orderId: string, amount: number) {
await paymentProvider.charge(amount);
await database.orders.update(orderId, {
status: "paid",
});
}This works when both operations succeed, but the issue appears when the payment succeeds and the database update fails. The customer has been charged, but the application has no record of the completed payment. If the process repeats, the customer may be charged again.
One safeguard is to use an idempotency key:
async function processOrder(orderId: string, amount: number) {
const payment = await paymentProvider.charge(amount, {
idempotencyKey: `order:${orderId}:payment`,
});
await database.orders.update(orderId, {
status: "paid",
paymentId: payment.id,
});
}When the same payment request is retried with the same key, a payment provider that supports idempotency should return the original outcome instead of charging the customer again.
Although this reduces the risk of duplicate payments, it does not solve the entire problem. Because the database update can still fail, a production system may also need payment webhooks, retry logic, or a reconciliation process that compares payment records with orders stored in the application.
The important point is that code should be reviewed from different angles and checked for what happens between the happy path and a complete failure.
Useful questions include what happens if only some operations succeed and others fail; whether functions can be safely retried without creating unexpected duplicates; whether the same request could create duplicate data; and how the system will detect an inconsistent state.
It is worth remembering that a model rarely considers questions like these unless they are part of the requirements.
Tests that check implementation instead of behaviour
LLMs can generate tests that look very professional while providing little protection. This is especially risky when you are not experienced in writing tests and trust what the model generated without checking it.
Take this test for a discount:
it("applies a discount", async () => {
const discountSpy = vi.spyOn(priceService, "calculateDiscount");
await checkoutService.createOrder({
price: 100,
discountCode: "SUMMER20",
});
expect(discountSpy).toHaveBeenCalledWith(100, "SUMMER20");
});The test confirms that calculateDiscount was called with the expected arguments, but it does not confirm what happened to the return value. With this kind of test, we would not be able to spot that the system could call the discount function, ignore its return value, and still charge the customer the full amount.
A better test checks the behaviour visible to the rest of the system. In this version, the final price should be reduced by 20%, and the test checks the final value:
it("reduces the final price by 20%", async () => {
const order = await checkoutService.createOrder({
price: 100,
discountCode: "SUMMER20",
});
expect(order.finalPrice).toBe(80);
});Tests that focus only on internal methods create two problems: they may pass while the feature is broken, and they may fail after an internal implementation has changed even though the feature still works.
When working with an AI-generated test, developers should ask one question: if the feature stopped working for the user, could this test still pass? If the answer is yes, the test probably needs to check more than the implementation.
Important instructions get lost in a long conversation
Coding agents collect a large amount of context, such as architectural decisions, database conventions, security requirements and business rules. All of these instructions can eventually get buried under less relevant messages. Long conversations might also get shortened during context compaction, or given less attention than a recent request. This causes problems for developers and should be actively monitored — it is well established that conversations with models should stay as short as possible to keep the context window free, since the quality of responses declines as more of the context window fills up.
Take the following example. A rule was established at the beginning of a conversation:
// Orders above 10,000 must be reviewed manually.
// The customer must not be charged automatically.Later, the model is asked to implement the order-processing function:
async function processOrder(order: Order) {
await paymentProvider.charge(order.customerId, order.total);
await database.orders.update(order.id, {
status: "paid",
});
}The code is valid TypeScript and works correctly for most orders. However, it breaks a business rule, because every order is charged automatically without checking whether the value should be reviewed manually.
Here is an implementation that checks the order value before starting the payment:
async function processOrder(order: Order) {
if (order.total > 10_000) {
await database.orders.update(order.id, {
status: "pending_manual_review",
});
return;
}
await paymentProvider.charge(order.customerId, order.total);
await database.orders.update(order.id, {
status: "paid",
});
}This shows a structural problem with keeping requirements only in the conversation history. Conversation history should not be treated as reliable storage. All rules that are important for a project should be stored as instruction files, a technical specification, an architecture document, or acceptance criteria attached to the task.
Business rules should also be covered by tests:
it("does not charge orders requiring manual review", async () => {
const order = {
id: "order-123",
customerId: "customer-456",
total: 15_000,
};
await processOrder(order);
expect(paymentProvider.charge).not.toHaveBeenCalled();
expect(database.orders.update).toHaveBeenCalledWith(order.id, {
status: "pending_manual_review",
});
});Generating code is only the first step
AI coding tools can reduce the time needed to create a first implementation, but they can also produce code that looks finished while causing issues down the road. This matters most as a project grows, new features get implemented, and small mistakes compound into real consequences.
A sensible workflow should include the same checks used for human-written code: review the assumptions behind the implementation; run type checks, builds and automated tests; check compatibility with the project's dependencies; test failure scenarios and edge cases; compare the code with documented business rules; and review security-sensitive changes manually.
Code that compiles and works is a useful starting point. It is not evidence that it is safe, complete, or suitable for shipping or releasing to users.
How do you make sure your code matches the expectations? We are always interested in how people build their products and use AI — let us know whether you want to build a product, update an existing one, or need help implementing AI in your business.