Get this in your inbox — free, every weekday.
SMRTR Programming Newsletter
Header Image
Today’s edition brings a mix of testing insights and clever coding tips.
  • Unlock the secrets of testing conversational AI effectively.
  • Navigate the tricky waters of migrating from Heroku to AWS.
  • Discover how to swap AI providers in Nuxt with ease.
  • Micro-Tutorials on Python and TypeScript for practical coding enhancements.
  • Explore music theory's surprising connections to programming and more.
Featured
Story Image

How to Test Conversational AI: A Practical Guide for QA Engineers

Testing conversational AI is a puzzle that stumps even seasoned quality engineers. Ask a chatbot "How do I reset my password?" twice, and you may get two completely different answers. Both could be correct. So how do you test something that doesn't behave the same way twice?

The answer, it turns out, is to stop chasing exact outputs and start defining what a good response must accomplish. Rather than matching words precisely, testers evaluate responses across dimensions like accuracy, relevance, clarity, and helpfulness.

But the challenge runs deeper. Multi-turn conversations, where users switch topics or correct themselves mid-chat, require testing entire exchanges, not just individual replies. And when an AI confidently invents a policy for a product that doesn't exist, that's a hallucination, and catching it requires deliberately asking about things the system should not know.

The toolkit sounds new, but the instincts are familiar. Edge cases, integration checks, risk-based prioritization, regression suites. What changes is the definition of "expected result." It's no longer a fixed answer. It's a set of criteria a trustworthy response must meet.
 
More Stories

AI Code Review Tools: Benchmarks & Comparison

The AI code review market is flooded with tools making identical claims, but real differences emerge in five areas: context depth, standards enforcement, review architecture, SDLC coverage, and enterprise readiness. Benchmarks matter too — most vendor data measures isolated models, not real production behavior. A 2025 study showed a 50-point performance drop between synthetic and real-codebase testing, making benchmark methodology as important as the scores themselves.
 

Migrating off Heroku to AWS without killing your deploy speed

Moving from Heroku to AWS is deceptively hard — containers migrate easily, but the platform layer (git-push deploys, preview environments, managed databases, rollbacks) doesn't. This guide breaks down three migration paths, compares AWS App Runner vs. ECS Fargate vs. EKS vs. PaaS alternatives, and lays out a 5-phase migration sequence. It also explains how AWS Partner funding can offset migration costs.
GitHub Logo GitHub Repo: session-migrate
GitHub Repo Image
Session-migrate is a specialized tool designed for developers who work with multiple coding agents like Claude Code and Codex. It enables the migration of session data, preserving user and assistant messages, tool calls, and other relevant information, while ensuring that the source session remains untouched.
This tool is particularly useful in scenarios where developers need to switch between platforms for different tasks, allowing them to continue their work without interruption.
By providing a structured migration process, session-migrate alleviates the frustration of losing context when changing coding environments.
Get Out of Today's Standup
Can't make standup today because my canary tests are leaking memory into GitHub's cache, so every build fails before the YAML parser even wakes up.
Got a better excuse? Reply to this email!
And a few more...

Music theory for programmers

Music theory feels arbitrary until you trace it back to physics and math. Twelve notes exist because of how prime numbers 2 and 3 interact; scales use uneven gaps so your ear can navigate them; chords sound good because their harmonics overlap. Starting from a single sine wave, everything in music can be derived through code.
 

Agents on Rails: lemans goes open source

The team behind Agents on Rails has open-sourced lemans, their Ruby-based LLM benchmarking harness. Built with convention-over-configuration and CLI-first design, it sandboxes agents, hides verification tests, blocks internet access, and scrubs credentials from logs. Four new models were also benchmarked, including a locally-runnable Qwen and a mysterious unnamed model on OpenRouter scoring competitively against paid options.
Bite-Size Stories

How to Exit Vim: A Complete Guide for Beginners and Pros

Vim starts in Normal mode (not Insert mode), which confuses beginners. To exit, press Esc, then type :wq (save and quit), :q! (quit without saving), or ZZ (quick save-quit shortcut).
 

A Friendly Introduction to Racket

Racket descends from Lisp (1958) and leverages homoiconicity—where code is data—letting developers write macros, generate code, and build entirely new programming languages.
 

You Don't Always Need a Workflow Engine to Roll Back a Failed Checkout

Compensator is a Laravel package that orchestrates multi-step distributed transactions using the Saga pattern, handling ordered rollbacks, retries, and idempotency—synchronously, with no queue, migrations, or database state required.
 

My agent.md to improve LLM-assisted code quality

Using an `agent.md` file injected at session start encodes style preferences once, eliminating repeated prompting and improving LLM code quality. Combat context dilution by keeping sessions short per feature and reloading `agent.md` when output degrades.
 

OpenAI Brings GPT-5.6 Model Family to AWS’s Kiro

AWS added OpenAI's GPT-5.6 model family to Kiro, its spec-driven AI IDE. Joint testing on Terminal-Bench 2.1 showed an 82% reduction in task completion costs, boosting efficiency for spec-first developers.
Micro-Tutorial: python

Add Strict Prompt Rules and Human Approval to VIN Verification

A raw extraction prompt can return a confident-looking but wrong VIN. Add a strict rule that forces the model to reply UNKNOWN, cross-check the proposed VIN against a SQLite table, and make a human approve the value. The AI guesses, but it doesn't get the final word. For example, a support agent can paste a VIN from a customer photo without trusting the model's first guess.

The script runs that prompt against gpt-4o-mini, checks the VIN in SQLite, and asks for approval.

python
import sqlite3
from openai import OpenAI

client = OpenAI()
PROMPT_RULE = 'Extract a VIN. If no VIN is present or you are not confident, reply UNKNOWN. Reply with only the VIN or UNKNOWN.'
VALID_VINS = {'1HGCM82633A004352', '5YJ3E1EA7HF000337'}

db = sqlite3.connect(':memory:')
db.execute('CREATE TABLE vins (vin TEXT PRIMARY KEY)')
db.executemany('INSERT INTO vins VALUES (?)', ((vin,) for vin in VALID_VINS))

def propose_vin(text):
    response = client.chat.completions.create(model='gpt-4o-mini', messages=[{'role': 'system', 'content': PROMPT_RULE}, {'role': 'user', 'content': text}], temperature=0)
    return response.choices[0].message.content.strip().upper()

def cross_check(vin):
    if vin == 'UNKNOWN':
        return False
    row = db.execute('SELECT 1 FROM vins WHERE vin = ?', (vin,)).fetchone()
    return row is not None

text = input('Paste text with a VIN: ')
vin = propose_vin(text)
print('AI proposes:', vin)
if not cross_check(vin):
    print('Rejected: missing or unknown VIN.')
elif input('Approve this VIN? [y/n]: ').strip().lower() == 'y':
    print('Verified and approved.')
else:
    print('Rejected by human.')

These steps mirror the code above.

  1. Install the openai library with pip install openai and set OPENAI_API_KEY in your environment.
  2. Save the code above as verify_vin.py; the static VALID_VINS set is a stand-in for your real database.
  3. Run python verify_vin.py, paste text that contains a VIN, and approve or reject the proposal.
  4. Replace VALID_VINS with your real VIN database or API lookup before using it in production.

When to use it: Use it whenever free-form text can produce a VIN that must not be trusted until verified.

Micro-Tutorial: typescript

Swap AI providers in Nuxt by changing only the provider line

An LLM call in Nuxt often starts with a provider object such as gateway('gpt-4o-mini') inside a server route. That works until you need a cheaper worker-backed model. The Vercel AI SDK normalizes every provider behind the same generateText interface, so swapping from a hosted gateway to Cloudflare Workers AI touches only the model line. For a translation feature, you can keep the same prompt and streaming.

This Nuxt handler builds the Workers AI provider from the request's Cloudflare binding; the model assignment is the only provider-specific line.

typescript
import { createOpenAI } from '@ai-sdk/openai'
import { createWorkersAI } from '@ai-sdk/cloudflare-workers-ai'
import { generateText } from 'ai'
import { defineEventHandler, readBody } from 'h3'

const gateway = createOpenAI({
  apiKey: process.env.GATEWAY_API_KEY,
  baseURL: process.env.GATEWAY_URL,
})

export default defineEventHandler(async (event) => {
  const workersAI = createWorkersAI({
    binding: event.context.cloudflare?.env.AI,
  })

  const model = workersAI('@cf/meta/llama-3.1-8b-instruct')
  // Switch back: const model = gateway('gpt-4o-mini')

  const body = await readBody(event)
  const { text } = await generateText({
    model,
    prompt: body.prompt,
  })

  return { text }
})

Walk through the endpoint and the switch.

  1. Install ai, @ai-sdk/openai, and @ai-sdk/cloudflare-workers-ai with npm install ai @ai-sdk/openai @ai-sdk/cloudflare-workers-ai.
  2. Create server/api/chat.post.ts and paste the handler above.
  3. Add GATEWAY_API_KEY and GATEWAY_URL in .env.
  4. Deploy to Cloudflare Workers with an AI binding, then POST { "prompt": "hello" } to /api/chat.
  5. Flip to the gateway by editing just const model = gateway('gpt-4o-mini').

When to use it: Use it when a cost-conscious production feature, like a support bot, must move from a hosted gateway to Cloudflare Workers AI without rewriting the route.

Like what you read?

Get SMRTR delivered to your inbox every weekday — free, concise, and hand-curated.

Daily delivery⏱ 5-minute read✓ 100% free