01 / 47
workshop // BSW 2026
Your Vibe Coded App Rules.
Now what?
Whether it's Lovable, Replit, v0 or Bolt — it's living inside the system you prompted to build it. And you have no idea what would happen if it broke at 2am.
02 / 47
who's talking
Quick about me
- I run Boulder New Tech — 17-year-old community, took it over last year
- Builders' Room at BSW — 3 years and counting
- Threshold Labs — geometric approaches to neural nets
- Recent builds: FlowDJ (generative music from typing rhythm), Dropin (venue/musician matching for rural CO)
- ~12 years in Boulder, watching founders ship and break things
// tomorrow 9am: "My Covariance Matrix is Better Than Your Shit-Tier LLM" — the technical version of this conversation.
03 / 47
audience check
Who's here?
Are you a developer?
Are you supporting a site that has 100 daily users?
1,000?
5,000?
04 / 47
the gap nobody warns you about
The gap nobody warns you about
Week 1
It feels like magic.
- You ship faster than engineers you used to hire.
- The app does what you wanted.
- Investors are impressed.
Month 6
You're a stranger to it.
- Auth is broken in a way you can't debug.
- The OpenAI bill is $4,300 and you don't know why.
- Asking the AI to fix it makes things worse.
- You consider rewriting from scratch.
05 / 47
empathy check
Any devs here?
This isn't that different from looking at code you wrote six months ago.
06 / 47
the thesis
The fix isn't to rewrite it.
The fix is to own it.
Six things to own your app.
07 / 47
section 01
Own your code
If you can't see your code, you don't have control.
08 / 47
reality check
Where does your code live right now?
"In Replit. I think."
What if Replit goes down? What if you forget the password?
"Lovable has it."
Their server. Their database. Their decision when to delete it.
"I deployed it straight to a Cloudflare Worker."
Cool. Where's the source? Can you redeploy it? From what?
GitHub. Yours.
On your account. Versioned. Backed up. Portable.
Goal: get here. →
09 / 47
step 1
Push it to GitHub. Today.
- A GitHub repo on your account — not your contractor's, not your platform's.
- Every change tracked. You can see what the AI did, when, and why.
- You can hand it to any developer in the world. They'll know what to do.
- Cursor, Claude Code, every AI tool talks to GitHub natively.
Lovable, Bolt, v0 all have a "Connect to GitHub" button. Click it.
10 / 47
before you push
Kill the secrets
Secrets are anything that's a password in disguise:
- OpenAI / Anthropic / Stripe API keys
- Database URLs with passwords in them
- AWS access keys, SendGrid tokens, Twilio SIDs
The rule: they go in .env, and .env goes in .gitignore. Never the repo.
Ask Cursor: "Audit my repo for hardcoded API keys." It's good at this.
11 / 47
why this matters — the data
Why this matters
39M
secrets leaked
across GitHub in 2024 alone.
Source: GitHub Security Blog
6.4%
of Copilot repos
leak secrets — vs 4.5% baseline.
Source: GitGuardian, 2026
94
days to remediate
the median credential leak.
Source: Verizon DBIR 2025
AI tools ship code faster. They leak keys faster too.
12 / 47
The $4,300 OpenAI bill
war story #1
Someone I was working with wanted to ship a ChatGPT clone. He found one on Twitter and we deployed it.
One Saturday morning, OpenAI emailed: "Your usage spiked 300x."
The API key was leaked by the open source repo. It went to who-knows-where and cost a ton of money.
Lesson: Vet your open source repos. Not everyone is a good samaritan.
13 / 47
section 02
Iterate without breaking
Branches and PRs are how you stop being scared to change things.
14 / 47
the most expensive habit
The most expensive habit you have
✗ What you do now
"Hey AI, fix the login bug."
- It edits 14 files you didn't know existed.
- The login works. Checkout breaks.
- You can't roll back. Production is broken.
✓ What we're going to do
Branch → change → review → merge.
- Every change lives in its own branch first.
- Production keeps running, untouched.
- Bad change? Delete the branch. Done.
15 / 47
the safety net
The branch is your safety net
Think of branches like document drafts in Google Docs.
- main = the version your customers are using right now. Sacred.
- fix-checkout = a draft. The AI can do whatever it wants here.
- You test it on the branch. Looks good? Merge it into main.
- Looks bad? Throw the branch away. Main never moved.
Tell Cursor / Claude Code: "Make a branch called fix-checkout, then make the change there."
16 / 47
the checkpoint
Pull requests = the "wait, what did it just change?" checkpoint
A PR is a single screen that shows you, in red and green:
- Every line the AI added.
- Every line it deleted.
- Every file it touched. Including the ones you didn't ask it to.
You don't need to read every line. You need to skim and ask: "does this match what I asked for?"
Confused by what you see? Paste the diff into Claude and ask "explain this in plain English."
17 / 47
second pair of eyes
Get a second pair of (AI) eyes
You don't have a senior engineer. Hire a robot one for $20–$40/mo.
- CodeRabbit — reviews every PR automatically. Catches bugs, security issues, style problems.
- Greptile — deeper context, knows your whole codebase. Better at "this contradicts what we did last week."
- Claude Code review — if you're already a Claude user, free with your subscription. Just point it at a PR.
Pick one. Install it. Forget about it.
18 / 47
git best practice
Just glance at the diff.
You don't need to read every line. Ask one question:
- Is it adding code or removing code?
- If it's adding a lot — it might be spot-fixing edge cases instead of making a general solution. That's how codebases turn into spaghetti.
- If it's removing code — that's usually a good sign. Simpler is better.
- If it's replacing a block — does the new version actually make sense, or did it just rewrite something it didn't understand?
You don't need to be a developer to spot a red flag in a diff.
19 / 47
The "tiny tweak" that nuked the database
war story #2
Founder asks Cursor to "rename the 'email' field to 'contact_email' for clarity."
Cursor writes a database migration. Confidently. Drops the old column first.
Founder hits "run" on production. 3,200 customer emails — gone.
Lesson: a PR with CodeRabbit catches this in 30 seconds.
20 / 47
section 03
Deploy without the panic.
21 / 47
reality check
Right now your app is on… what, exactly?
🤷 "It's on Replit / Lovable"
Fine for demos. Their server, their rules, their bill if you blow up.
🚀 Static deploy
Vercel, Netlify, Cloudflare. Free for most apps. Push to GitHub → live URL.
⚙️ Real backend
Railway, Render, Fly.io. ~$5/mo. Needed when you have a database or server jobs.
22 / 47
pick one and go
Pick one and go. Seriously.
| Vercel | Netlify | Railway |
|---|
| Best for | Next.js apps | Static sites + edge funcs | Full backend + DB |
| Free tier | Hobby (generous) | Generous | $5 trial credit |
| Setup time | ~2 min | ~2 min | ~5 min |
All three: connect GitHub, click deploy, get URL. You can always switch later — this is not forever.
23 / 47
the killer feature
The rollback button
Every push creates a new "deployment."
- Broke production at 11pm? One click.
- Investor demo in 5 min and the app is white-screening? One click.
- You can roll back to any version, instantly. This alone is worth the migration.
24 / 47
the trap
Stop letting the AI push to prod.
- Once you give the LLM deploy access, every fix becomes a hotfix. No staging, no review, just vibes.
- Force deployment through GitHub Actions or similar — a merge to
main triggers the deploy, not you typing "ship it."
- No direct database access without backups and atomic rollback points.
- If you bundle services together with no staging environment, one "small change" can take down everything.
live example
I've been live-pushing changes to revelco.org/bsw — the site this entire conference runs on — all week. It feels great until you realize there's no rollback, no staging, and the database has no backup. Don't be me.
25 / 47
"My demo URL stopped working an hour before the pitch."
war story #3
Founder had been demoing off the Lovable preview link. Free tier sleeps after inactivity. Investor clicks the link, sees a loading spinner for 40 seconds, closes the tab.
Lesson: before any meeting that matters, deploy it to a real URL you control. Vercel free tier doesn't sleep.
26 / 47
section 04 / keynote
The LLM is lying to you.
27 / 47
Not maliciously. Not always. But constantly.
And it always sounds confident when it does.
28 / 47
four lies the LLM tells you, every day
Four lies
01 Made-up libraries
"Use the popular npm package called react-fancy-modal!" It does not exist. Never has.
02 Hallucinated APIs
"Just call stripe.payments.cancelByEmail()." That method isn't in the docs. It just sounds plausible.
03 Fake config flags
"Set USE_PRODUCTION_MODE=true." The flag does nothing. The framework ignores it.
04 "This is secure"
It can't verify that. It pattern-matched on what "secure" code usually looks like.
29 / 47
why this happens
Why does this happen?
It's not lying. It's predicting the next word.
Every answer is the model's best guess at what code usually looks like when humans answer this kind of question.
It has no concept of "checked the docs" or "ran the code." If a function name looks right, it'll suggest it.
And here's the kicker: it doesn't know when it's wrong.
30 / 47
steal this
The verification habit
- Did the code actually run? Not "did it generate" — did it run, with no errors?
- Did I click the thing it said works? "It compiles" is not "it works."
- Does the library/API exist? Search npm, GitHub, official docs. Takes 30 seconds.
- Ask "are you sure?" If it backtracks, it was guessing the first time.
31 / 47
the red lines
Never trust the LLM unconditionally with…
💸 Money
Stripe charges, refunds, Plaid transfers. One typo = real dollars.
🗑️ Database deletes
DROP TABLE, DELETE WHERE. The LLM forgets the WHERE clause sometimes.
🔐 Auth & permissions
"Admin checks" the LLM wrote may not actually check anything.
📧 Anything sent to users
Emails, SMS, push notifications. Send it to yourself first.
32 / 47
"npm install of a package that didn't exist… until it did."
war story #4
Cursor told a founder to install a charting library that, at the time, was not on npm. They typed npm install, got "not found," shrugged, moved on.
Two weeks later, an attacker noticed Cursor was suggesting that fake name to thousands of users. They published a malicious package under it. Anyone who did install it shipped malware in their app.
Lesson: before installing a package the LLM suggested, search for it on npmjs.com. Real packages have weekly downloads, GitHub repos, and history.
33 / 47
section 05
Auth: do not roll your own.
34 / 47
the one rule
Auth is the one thing you should not let the LLM invent.
Use a library. Use a service. Both are fine.
"Custom auth I built in an afternoon" is the start of every breach story.
35 / 47
the 2026 short list
Pick one.
| Clerk | Better Auth | Supabase Auth |
|---|
| Type | Hosted service | Open-source library | Bundled with DB |
| Free tier | 50K users | Free forever | 50K MAU |
| Setup | ~10 min | ~30 min | ~15 min |
| Best for | B2C, fastest path | You want to own it | Already on Supabase |
WorkOS is also great if you're B2B and need SSO from day one (free up to 1M users).
36 / 47
why "my own auth" is scary
CVE-2025-29927
March 2025: every Next.js app doing auth in middleware was bypassable.
Attackers could send a single header — x-middleware-subrequest — and skip auth entirely. Walk straight into the admin dashboard.
This is a framework that thousands of teams use, written by experts. They still missed it for years.
Takeaway: auth is genuinely hard. Use a library that's audited and patched. Update it.
37 / 47
the tradeoff
When to pay vs. when to self-host
Pay (Clerk, Auth0, WorkOS)
- You're under 50K users
- You want a UI tonight, not a project
- You'll need SSO/SAML for enterprise
Free until ~50K, then $25/mo + $0.02/user
Self-host (Better Auth, Supabase)
- You want users in your own database
- You're comfortable updating libraries
- You don't want a per-user bill at scale
Free forever; you own the patch cycle
38 / 47
"The password reset that emailed everyone"
war story #5
"Why is everyone in our Slack getting a password reset email?"
The AI wrote a password reset endpoint. It worked. It also reset the password for the email you put in the form, regardless of who was logged in.
Anyone could take over any account. The hack: type their email into the reset form.
Lesson: let an audited library handle the auth flows. You're not smarter than 5 years of CVEs.
39 / 47
section 06
Watch what's actually happening
Errors. Costs. Uptime. The dials you wish you had on day 1.
40 / 47
right now
Do you know what's broken?
Is anyone hitting an error right now?
Sentry will tell you.
How much did your AI features cost yesterday?
Helicone or Langfuse will tell you.
Is your site even up?
Better Stack or UptimeRobot will tell you.
41 / 47
the minimum viable stack
Minimum viable observability
1. ERRORS → Sentry
Free for hobbyists. 5-min install. You'll know when something crashes — including the exact line.
2. AI COSTS → Helicone or Langfuse
One line of code in front of OpenAI/Anthropic/etc. Tracks every call, cost, latency. Spending dashboards.
3. UPTIME → Better Stack or UptimeRobot
Pings your URL every minute. Texts you when it dies. Free tier covers anything you'll need pre-launch.
42 / 47
"Signups stopped 9 days ago"
war story #6
"Wait — when's the last time anyone signed up?"
An env var rotated. The signup endpoint started returning a silent 500. Form submitted, page refreshed, no error to the user.
Nine days passed before someone noticed the signups graph was flat. With Sentry installed, this would've been a phone notification five minutes after the first failed signup.
Lesson: silent failures are the most expensive. Set up error monitoring before launch, not after.
43 / 47
"The infinite retry loop"
war story #7
"GPT-4 was called 11,000 times in the last hour. We have 80 users."
An OpenAI call lived inside a useEffect. The effect re-fired on every state change. Every state change called GPT-4. Every call changed state.
Helicone caught the runaway in 12 minutes. Without it, this would've been a $2K bill before anyone noticed.
Lesson: set spend alerts on every paid API. AI calls in render loops are a normal failure mode, not a freak event.
44 / 47
the point
Don't rewrite it.
Own it.
45 / 47
level up
How to make vibe coding actually better.
- Embed your sessions. Every vibe coding conversation is data. Export it. Embed it. Search it later. You'll stop re-solving the same problems.
- Analyze your AI spend. Export your OpenAI or Claude usage data. Look at what you're actually paying for. Half of it is retries and hallucination loops.
- Run a local model in the background. Have it do proactive research — read docs, scan changelogs, build a reading list for you to flip through while you wait on the vibes to vibe.
- Stop treating the AI like a black box. The more you understand what it's doing, the better your prompts get, the less you pay, and the more you own.
46 / 47
your weekend checklist
Your weekend checklist
- Push your code to GitHub. Private repo. Today.
- Audit secrets. Move every key into env vars. Add .env to .gitignore.
- Make tomorrow's change on a branch. Open a PR. Let CodeRabbit or Claude review it.
- Connect your repo to Vercel/Netlify/Railway. Bookmark the rollback button.
- Pick an auth library. Don't write one.
- Install Sentry + Helicone + an uptime monitor. 30 minutes total.
47 / 47
connect
Thanks, Boulder. Build something this week.
Threshold Labs
getthreshold.com
LinkedIn
linkedin.com/in/ryanastpierre
These Slides
bsw-decks.pages.dev
Tomorrow 9am · "My Covariance Matrix is Better Than Your Shit-Tier LLM" · Boulder Associates