![]() |
- Unlock the secrets of testing conversational AI effectively.
- Navigate the tricky waters of migrating from Heroku to AWS.
- Discover how to swap AI providers in Nuxt with ease.
- Micro-Tutorials on Python and TypeScript for practical coding enhancements.
- Explore music theory's surprising connections to programming and more.
How to Test Conversational AI: A Practical Guide for QA Engineers
Testing conversational AI is a puzzle that stumps even seasoned quality engineers. Ask a chatbot "How do I reset my password?" twice, and you may get two completely different answers. Both could be correct. So how do you test something that doesn't behave the same way twice?The answer, it turns out, is to stop chasing exact outputs and start defining what a good response must accomplish. Rather than matching words precisely, testers evaluate responses across dimensions like accuracy, relevance, clarity, and helpfulness.
But the challenge runs deeper. Multi-turn conversations, where users switch topics or correct themselves mid-chat, require testing entire exchanges, not just individual replies. And when an AI confidently invents a policy for a product that doesn't exist, that's a hallucination, and catching it requires deliberately asking about things the system should not know.
The toolkit sounds new, but the instincts are familiar. Edge cases, integration checks, risk-based prioritization, regression suites. What changes is the definition of "expected result." It's no longer a fixed answer. It's a set of criteria a trustworthy response must meet.
AI Code Review Tools: Benchmarks & Comparison
The AI code review market is flooded with tools making identical claims, but real differences emerge in five areas: context depth, standards enforcement, review architecture, SDLC coverage, and enterprise readiness. Benchmarks matter too — most vendor data measures isolated models, not real production behavior. A 2025 study showed a 50-point performance drop between synthetic and real-codebase testing, making benchmark methodology as important as the scores themselves.Migrating off Heroku to AWS without killing your deploy speed
Moving from Heroku to AWS is deceptively hard — containers migrate easily, but the platform layer (git-push deploys, preview environments, managed databases, rollbacks) doesn't. This guide breaks down three migration paths, compares AWS App Runner vs. ECS Fargate vs. EKS vs. PaaS alternatives, and lays out a 5-phase migration sequence. It also explains how AWS Partner funding can offset migration costs.
GitHub Repo: session-migrate
Music theory for programmers
Music theory feels arbitrary until you trace it back to physics and math. Twelve notes exist because of how prime numbers 2 and 3 interact; scales use uneven gaps so your ear can navigate them; chords sound good because their harmonics overlap. Starting from a single sine wave, everything in music can be derived through code.Agents on Rails: lemans goes open source
The team behind Agents on Rails has open-sourced lemans, their Ruby-based LLM benchmarking harness. Built with convention-over-configuration and CLI-first design, it sandboxes agents, hides verification tests, blocks internet access, and scrubs credentials from logs. Four new models were also benchmarked, including a locally-runnable Qwen and a mysterious unnamed model on OpenRouter scoring competitively against paid options.How to Exit Vim: A Complete Guide for Beginners and Pros
Vim starts in Normal mode (not Insert mode), which confuses beginners. To exit, press Esc, then type :wq (save and quit), :q! (quit without saving), or ZZ (quick save-quit shortcut).A Friendly Introduction to Racket
Racket descends from Lisp (1958) and leverages homoiconicity—where code is data—letting developers write macros, generate code, and build entirely new programming languages.You Don't Always Need a Workflow Engine to Roll Back a Failed Checkout
Compensator is a Laravel package that orchestrates multi-step distributed transactions using the Saga pattern, handling ordered rollbacks, retries, and idempotency—synchronously, with no queue, migrations, or database state required.My agent.md to improve LLM-assisted code quality
Using an `agent.md` file injected at session start encodes style preferences once, eliminating repeated prompting and improving LLM code quality. Combat context dilution by keeping sessions short per feature and reloading `agent.md` when output degrades.OpenAI Brings GPT-5.6 Model Family to AWS’s Kiro
AWS added OpenAI's GPT-5.6 model family to Kiro, its spec-driven AI IDE. Joint testing on Terminal-Bench 2.1 showed an 82% reduction in task completion costs, boosting efficiency for spec-first developers.Add Strict Prompt Rules and Human Approval to VIN Verification
A raw extraction prompt can return a confident-looking but wrong VIN. Add a strict rule that forces the model to reply UNKNOWN, cross-check the proposed VIN against a SQLite table, and make a human approve the value. The AI guesses, but it doesn't get the final word. For example, a support agent can paste a VIN from a customer photo without trusting the model's first guess.
The script runs that prompt against gpt-4o-mini, checks the VIN in SQLite, and asks for approval.
import sqlite3
from openai import OpenAI
client = OpenAI()
PROMPT_RULE = 'Extract a VIN. If no VIN is present or you are not confident, reply UNKNOWN. Reply with only the VIN or UNKNOWN.'
VALID_VINS = {'1HGCM82633A004352', '5YJ3E1EA7HF000337'}
db = sqlite3.connect(':memory:')
db.execute('CREATE TABLE vins (vin TEXT PRIMARY KEY)')
db.executemany('INSERT INTO vins VALUES (?)', ((vin,) for vin in VALID_VINS))
def propose_vin(text):
response = client.chat.completions.create(model='gpt-4o-mini', messages=[{'role': 'system', 'content': PROMPT_RULE}, {'role': 'user', 'content': text}], temperature=0)
return response.choices[0].message.content.strip().upper()
def cross_check(vin):
if vin == 'UNKNOWN':
return False
row = db.execute('SELECT 1 FROM vins WHERE vin = ?', (vin,)).fetchone()
return row is not None
text = input('Paste text with a VIN: ')
vin = propose_vin(text)
print('AI proposes:', vin)
if not cross_check(vin):
print('Rejected: missing or unknown VIN.')
elif input('Approve this VIN? [y/n]: ').strip().lower() == 'y':
print('Verified and approved.')
else:
print('Rejected by human.')
These steps mirror the code above.
- Install the
openailibrary withpip install openaiand setOPENAI_API_KEYin your environment. - Save the code above as
verify_vin.py; the staticVALID_VINSset is a stand-in for your real database. - Run
python verify_vin.py, paste text that contains a VIN, and approve or reject the proposal. - Replace
VALID_VINSwith your real VIN database or API lookup before using it in production.
When to use it: Use it whenever free-form text can produce a VIN that must not be trusted until verified.
Swap AI providers in Nuxt by changing only the provider line
An LLM call in Nuxt often starts with a provider object such as gateway('gpt-4o-mini') inside a server route. That works until you need a cheaper worker-backed model. The Vercel AI SDK normalizes every provider behind the same generateText interface, so swapping from a hosted gateway to Cloudflare Workers AI touches only the model line. For a translation feature, you can keep the same prompt and streaming.
This Nuxt handler builds the Workers AI provider from the request's Cloudflare binding; the model assignment is the only provider-specific line.
import { createOpenAI } from '@ai-sdk/openai'
import { createWorkersAI } from '@ai-sdk/cloudflare-workers-ai'
import { generateText } from 'ai'
import { defineEventHandler, readBody } from 'h3'
const gateway = createOpenAI({
apiKey: process.env.GATEWAY_API_KEY,
baseURL: process.env.GATEWAY_URL,
})
export default defineEventHandler(async (event) => {
const workersAI = createWorkersAI({
binding: event.context.cloudflare?.env.AI,
})
const model = workersAI('@cf/meta/llama-3.1-8b-instruct')
// Switch back: const model = gateway('gpt-4o-mini')
const body = await readBody(event)
const { text } = await generateText({
model,
prompt: body.prompt,
})
return { text }
})
Walk through the endpoint and the switch.
- Install
ai,@ai-sdk/openai, and@ai-sdk/cloudflare-workers-aiwithnpm install ai @ai-sdk/openai @ai-sdk/cloudflare-workers-ai. - Create
server/api/chat.post.tsand paste the handler above. - Add
GATEWAY_API_KEYandGATEWAY_URLin.env. - Deploy to Cloudflare Workers with an
AIbinding, then POST{ "prompt": "hello" }to/api/chat. - Flip to the gateway by editing just
const model = gateway('gpt-4o-mini').
When to use it: Use it when a cost-conscious production feature, like a support bot, must move from a hosted gateway to Cloudflare Workers AI without rewriting the route.
Like what you read?
Get SMRTR delivered to your inbox every weekday — free, concise, and hand-curated.
