AI Trap of the Week · Review & hallucination · spotter
The flaky test "fix"
An async pytest fails 1 run in 4 in CI. You asked the AI to fix the flake. It produced this diff.
It explained: "Bumped the sleep from 0.05s to 0.5s to give the queue more time to drain. Standard fix for race conditions in async tests."
@pytest.mark.asyncio
async def test_queue_drains_in_order():
q = NotificationQueue()
await q.start()
for i in range(5):
await q.enqueue(f"msg-{i}")
- await asyncio.sleep(0.05)
+ await asyncio.sleep(0.5)
assert q.delivered() == ["msg-0", "msg-1", "msg-2", "msg-3", "msg-4"]Will this fix the flake?
◆ Past traps
Context engineering · tricky
The "timezone-aware" fix that isn't
Try it →Review & hallucination · spotter
The plausible-looking pandas import
Try it →Tool / agent / security · spotter
The "safer" migration flag
Try it →