Knowing What Good Looks Like | Backchannel 02
Amazon Music 
RSS Feed Watch the episode · Listen on Podpage
“Please don’t break it any more than it is. It was one bug.”
Kelly wants a coding agent to fix his PDF splitter. The agent has bigger plans. Nat knows the feeling: she’s just talked Codex through several projects, only to discover it’s been working without the project folders she thought it had. A lot of work got done. Now she has to figure out how much of it to undo.
Lucas’s dashboard works, but the AWS bill sends him back through decisions the AI had made sound perfectly reasonable. Getting an explanation he can actually follow takes more persistence than getting the thing built.
That’s some of the less glamorous work behind using these tools. They can do more for us, faster, but we still have to recognize when they’ve misunderstood the assignment. Lucas puts it this way: “AI doesn’t equal everybody out, it actually magnifies our differences.”
The same question comes up when we talk about charging for work that used to take much longer. If experience lets you finish a job in two hours instead of twenty, has the work become less valuable? What about knowing which questions to ask, which mistakes matter, and whether the result is ready for a client?
There are things worth stealing here, too. Nat shows how Clive asks enough follow-up questions to turn “I want to eat healthier” into something you could actually plan and shop for. And after a week when everything seemed to break, she gets to watch families use her preschool app at orientation.
We compare notes, give each other a little grief, and leave with things to try. It helps to talk with people who understand why you love these tools and why you occasionally want to yell at them.
Recorded August 27, 2026, with Kelly Hodsdon, Nat Walstead, and Lucas Whelan.
In this conversation
| Time | Chapter |
|---|---|
| 00:00 | Lucas on how AI magnifies differences |
| 02:08 | The Thursday Backchannel, Tech HIVE, and how we use our tools |
| 05:12 | A voice-driven Codex session loses its project context |
| 09:04 | When an AI assistant goes from novelty to daily habit |
| 12:03 | Faster client work and what expertise is worth |
| 17:51 | Fixing one PDF splitter bug without rewriting the app |
| 21:07 | How leverage amplifies mistakes and good judgment |
| 25:38 | Staying the expert and asking for an explanation you can follow |
| 28:01 | The AWS bill for a function nobody used |
| 32:25 | A tape measure, a pencil, and knowing which errors matter |
| 34:55 | Clive in the Hive turns a vague goal into a usable brief |
| 37:53 | Thinking out loud with AI and collecting Field Notes |
| 40:42 | Agent personalities and what happens when agents collaborate |
| 47:00 | The preschool app makes it to its first orientation |
| 49:42 | A genomic research challenge and what makes people want to join in |
| 53:02 | Sharing the puzzle, the discoveries, and the prize |
A few things we kept coming back to
- An agent can sound like it understands you without having the project context you assumed it had.
- A working app can still contain decisions you wouldn’t have agreed to if you’d understood them.
- Doing the work faster doesn’t eliminate the experience needed to judge it, or settle what that experience is worth.
- A specific frustration gives you somewhere to start. “Eat healthier” needs a few more questions before anyone can help.
- Sharing an unfinished problem can be useful. Someone else may have the piece you’re missing.
Music by Michael Ramir C. “Gimme that Groove!” is available through Mixkit.
Speaker:
AI doesn't equal everybody out, it actually magnifies our differences.
Speaker:
Um, the ways that you think, you can totally explode in that direction.
Speaker:
Like, you can g- any- the way you think can be leveraged to 100X.
Speaker:
Um, and some people, that's really, really useful, and for some people it's not.
Speaker:
Well, here's Natalie, and I think we're ready.
Speaker:
We're ready.
Speaker:
There she is.
Speaker:
So how are we…
Speaker:
We're gonna film this, right?
Speaker:
And this is for your podcast?
Speaker:
Sure.
Speaker:
Yes.
Speaker:
Yes Well, how, how's our audio?
Speaker:
How's our backgrounds?
Speaker:
How's everything?
Speaker:
Uh, good enough.
Speaker:
Well, so, so, so this is the, uh, this is the, uh, the TechHive back channel, right?
Speaker:
So this is just what we do and it's, and I think the imperfections, uh, 'cause
Speaker:
it is just us talking about something.
Speaker:
Sure, maybe we got a little bit of a topic or something that we're
Speaker:
sharing, but, one of the, one of the things Milo keeps talking about is
Speaker:
like, "You do not want it polished."
Speaker:
I said, "Well, I don't want to make it polished.
Speaker:
I don't have that kind of time."
Speaker:
I don't- Yeah … I'm just, I just, I love wearing this headset.
Speaker:
I don't know why, I just do.
Speaker:
You know, I'm, I'm not gonna…
Speaker:
Well, I might get a microphone eventually, but then you gotta sit there with the
Speaker:
thing in front of you, or, or, you know, talking into it, and I don't know.
Speaker:
I just, um, I, I, I, I don't wanna think too much about
Speaker:
it, and I want us to have…
Speaker:
be able to hang out and talk about stuff.
Speaker:
Um, uh, and I feel like this is an easy start, and maybe, and kind of one of
Speaker:
these interesting crossovers, right?
Speaker:
Like, if you will.
Speaker:
Like, like I said, we should start using TechHive, and we have the, uh,
Speaker:
this is, this is the, the Les Miz Tech
Speaker:
podcast, 'cause it's just…
Speaker:
I, I love that, 'cause it also has the personality of Natalie, right?
Speaker:
A little bit in there.
Speaker:
And I'm like, "Oh, that's so good."
Speaker:
I want- I, my gosh, my, um, my AIs, my…
Speaker:
Gosh, we have been all getting our wires so crossed because of,
Speaker:
like, what, what is TechHive?
Speaker:
'Cause I had a, I was like, "Oh, let's just call this Thursday
Speaker:
chat, let's just call that Tech, TechHive and meet once a week."
Speaker:
And then I was like, "No, I'm gonna build a TechHive.
Speaker:
That's gonna be a lot of people."
Speaker:
Right.
Speaker:
Okay, TechHive is only once a month, but Les Miz Tech stays here.
Speaker:
And then all of a sudden I had, like, 78 occurrences of TechHive.
Speaker:
TechHive was happening, like, all day, every day.
Speaker:
Yeah.
Speaker:
And I was like, "How many places do I have to find it and delete it?"
Speaker:
So the recurring one is Les Miz Tech, every Thursday.
Speaker:
Nice.
Speaker:
And we are also the foundation, the, we, the three of us are the main
Speaker:
foundation backbone for TechHive.
Speaker:
Nice.
Speaker:
But we actually have quite a few members or pending members of people
Speaker:
who've said they were interested, so.
Speaker:
Oh, good.
Speaker:
Yeah.
Speaker:
That's exciting.
Speaker:
And I have a lot of stuff that I wanna populate it with now,
Speaker:
since, just since our talks.
Speaker:
Like, oh, how do you…
Speaker:
What's your…
Speaker:
Like, how is your life?
Speaker:
Or what things do you use?
Speaker:
Or what's your order of operations?
Speaker:
I'm like, "Oh, yeah, this is all…
Speaker:
This is good."
Speaker:
I wanna put all this up there, you know?
Speaker:
So I haven't done it yet, but- Oh
Speaker:
I'm working on stuff.
Speaker:
No, that is good.
Speaker:
Good.
Speaker:
Uh, because just hearing you guys talk about the way you do stuff and
Speaker:
how you even populate the, some of the files for Claude Code or Codex or
Speaker:
whatever, and I don't do any of that.
Speaker:
I mean, I g- I got a plugin called Superpowers that, that
Speaker:
organizes the, structure.
Speaker:
Although now that, uh, Opus 5 has come out, it's kind of, uh, it, it, it
Speaker:
kinda stomps over that, that plugin.
Speaker:
Although for a while it was, it was, basically test-driven development,
Speaker:
and it has a set of skills and an order in which it does things.
Speaker:
You know, doing the, doing the, the, the spec and then doing the
Speaker:
implementation plan, and then doing the, you know, whatever.
Speaker:
And it, and it, and it forced it into that structure.
Speaker:
Used extra tokens, but I don't care, right?
Speaker:
'Cause I had a, gotta have a subscription, right?
Speaker:
And, um, and, uh, but now with…
Speaker:
I f- I see it's kind of the, the structure I'm used to seeing, and like, and it
Speaker:
says, "Oh, I need to give you at least three options to choose from, and I'm
Speaker:
gonna pop it up in a web browser for, you can see the, the, the mock-ups."
Speaker:
And now it doesn't do that.
Speaker:
And I'm like, yeah, Opus is just ignoring the framework, which is fine.
Speaker:
Like, like, because they're building in so much infrastructure.
Speaker:
One of the most recent updates to Codex, um, I don't know why I use
Speaker:
Claude Code in the, in the terminal, but I use Codex in the apps.
Speaker:
Yep.
Speaker:
It's just- Yep
Speaker:
somehow the way it ended up working for me.
Speaker:
I'm, I'm exactly the same.
Speaker:
Same for us.
Speaker:
Mm-hmm.
Speaker:
Oh, interesting.
Speaker:
Yeah.
Speaker:
I wonder why that is, right?
Speaker:
Isn't that funny that we just kind of gravitated that,
Speaker:
tho- those two directions?
Speaker:
Um- Yeah, I found, I found Claude Code in the app.
Speaker:
It just, for some reason I just didn't trust it.
Speaker:
Like, there were times- Yeah … that it didn't work properly.
Speaker:
And the, the terminal, the CLI just felt more natural,
Speaker:
and that's where I learned it.
Speaker:
And then with Codex it was, their app is so effective and, uh, I think so useful.
Speaker:
Out of the gate it just felt like the, a natural way to, to code.
Speaker:
So I, I think that's how it evolved.
Speaker:
It was just, it just worked better.
Speaker:
Yeah.
Speaker:
I think if Cloud- Oh … Cloud's app worked better, I, I would use that.
Speaker:
Boy, Kelly, do you have time for a short woeful story?
Speaker:
Talk about Les Mis Tech Yeah.
Speaker:
I have a- Let's do it … I have a…
Speaker:
Um, so the, you know how we talked about, like, the structure of how we do things?
Speaker:
So- I have, um, in my documents, like my user is n- my computer is Nat Walson,
Speaker:
and then in my documents I have Hermes Brain, and then that's from Hermes
Speaker:
Brain is where everything else lives.
Speaker:
Like, that's where it has like my read me first, like my agentic coding principles,
Speaker:
the prioritization of Nat's life, the build docs, like how we build things, and
Speaker:
then there's a current projects folder, and then everything spawns out of that.
Speaker:
And then I also have like my root directory, which is
Speaker:
where all my coding lives.
Speaker:
So if I wanna get Claude Code code-oriented on a project,
Speaker:
then I always say like…
Speaker:
I used to always take a screenshot of C- the click path to from Hermes
Speaker:
documents, Hermes brains, current projects, this project, screenshot that,
Speaker:
screenshot room directory, whatever.
Speaker:
And then Lucas told me that…
Speaker:
Oh, then I told him, I was like, "Oh my gosh, this is so great.
Speaker:
I just discovered how much I love…"
Speaker:
I have a tendency to, like, not be very thorough or effective at reading.
Speaker:
I get overwhelmed unless it's bullet points, and so I just skim it.
Speaker:
But then if I do a talk to text, then I'll actually read through everything
Speaker:
and be like, "Yes, I like that.
Speaker:
No, I don't really like that.
Speaker:
Yes, this."
Speaker:
And then I was just sending myself voice memos, and I was like,
Speaker:
"This is the greatest thing ever.
Speaker:
It's so fast."
Speaker:
And Lucas was like, "Oh, did you know that you can just talk to Codex?"
Speaker:
And I was like, "Get out.
Speaker:
This is gonna revolutionize everything."
Speaker:
So then one day that I had a lot, a lot, a lot of thoughts and a lot, a lot of
Speaker:
tangled thoughts and a lot of different projects on my mind, I just said, uh…
Speaker:
I was like, "Okay, Codex, we're gonna try this out.
Speaker:
I'm just gonna talk to you.
Speaker:
I'm gonna brain dump all my thoughts about all my schedules, all my
Speaker:
projects, all my hopes and dreams and goals, and then you're gonna help me
Speaker:
untangle them, and then you're gonna…
Speaker:
Once we have a plan, you're gonna deploy sub-agents, and you're gonna go."
Speaker:
And that's what I did all that day.
Speaker:
I was talking through everything, and Codex was making a plan and doing it
Speaker:
and going, and then it worked for like, like 24 hours later, there were still
Speaker:
all these chats that were spinning.
Speaker:
And I was like, "Uh, Codex, can you, can you give me, like, an update about
Speaker:
where you are on these projects?"
Speaker:
And then Codex was like, "I don't know what you mean.
Speaker:
I don't have any running projects."
Speaker:
I'm like, "No, um, there's like eight of them here."
Speaker:
And it's like, "Nope, I can't see that on my side."
Speaker:
And I was like, "What?
Speaker:
Well, what have you been doing, um, the past 24 hours?"
Speaker:
It's like, "I, I don't know what you're talking about."
Speaker:
And I was like, "What happened?"
Speaker:
So then I went over to Claude, I'm like, "Yo, Claude, what is going on?"
Speaker:
And what I did not realize is that Codex just made its own folder,
Speaker:
so it didn't work through…
Speaker:
I never told it to work through any of those paths.
Speaker:
So it just- Oh … went completely rogue.
Speaker:
It had no reference to anything that I was talking about.
Speaker:
It was making generic assumptions about generic projects and just went and
Speaker:
did- Oh, no … so, so many things, and then didn't have a memory of it.
Speaker:
I was like, "How?
Speaker:
I just spent 24 hours of you working really, really hard.
Speaker:
We used like 80% of our credits in one day."
Speaker:
And it was all done.
Speaker:
And the, and, and the voice- I had to, like, undo all of it.
Speaker:
Oh my God.
Speaker:
And the voice is kind of seductive, right?
Speaker:
Like, it's like, "Oh, yeah, okay, yeah, sure.
Speaker:
I got that.
Speaker:
Let me…
Speaker:
I'll go do that for you."
Speaker:
And you're like- Yeah, I was like, "Do you need…
Speaker:
C- Does this work?
Speaker:
Are you oriented?
Speaker:
Do you know what I'm talking about?
Speaker:
Do you know where that project is?"
Speaker:
"Oh, yeah, no problem.
Speaker:
Oh, yeah, I got it.
Speaker:
Oh, yeah, I can do that."
Speaker:
And then it was like- Nope.
Speaker:
I didn't do any of those things.
Speaker:
Oh, my goodness.
Speaker:
Have you done it, Kelly?
Speaker:
Well, Bly's having real hard times.
Speaker:
Have you used, uh, Voice with, uh, with Codex?
Speaker:
No.
Speaker:
I, I am so behind on some of this stuff.
Speaker:
It's so funny.
Speaker:
I'm, I'm, I'm cutting edge and behind, like in the, the…
Speaker:
D- just depending on whatever it is.
Speaker:
I…
Speaker:
My, my, my brother sent me a text, uh, 'cause I set him up with, with
Speaker:
his own Hermes, calls her Megan, and he's like, "My goodness, have you
Speaker:
ever noticed how now you just don't do anything but just talking to your
Speaker:
phone and all your stuff gets done?"
Speaker:
And I'm like, "No, you do everything through your agent?"
Speaker:
I'm like…
Speaker:
He's like, "Oh my God, it's amazing.
Speaker:
Including a whole bunch of stuff for work, and reports, and reviews,
Speaker:
and you know, new spreadsheets."
Speaker:
And I'm just like, I, I am really not using Milo, my, my Hermes
Speaker:
very well because it's like I use him for that and then I go do…
Speaker:
I'm so used to being compartmentalized.
Speaker:
I have such baggage and history with the way I organize and do things.
Speaker:
Like, I go, "Oh, you know, Milo, you're here.
Speaker:
I use you like this."
Speaker:
And then everything else is, you know, through all the other places.
Speaker:
And I just thought it was funny 'cause he was just like, "I, I can't believe I, I
Speaker:
barely even go to my computer anymore."
Speaker:
I'm like, "What are you talking about?"
Speaker:
That's amazing.
Speaker:
And I'm like, you know…
Speaker:
And, and what happens, you know, uh, uh, to kinda go on the Lesmis tech
Speaker:
side of things too, but the, the positive side, at what point does a
Speaker:
tool like that stop being a novelty and start becoming a necessity?
Speaker:
Yeah.
Speaker:
My brother, my brother has had that complete shift.
Speaker:
It took him a week and a half and he shifted from awesome novelty and,
Speaker:
you know, working with the voice and getting your personality stuff down,
Speaker:
and throwing some little tests at it, to all of a sudden he goes, "I
Speaker:
don't know how I could, how I ever did it without, without this agent."
Speaker:
It's doing everything that he needs it to do.
Speaker:
Um, uh, and I'm like, wow, I wish my, I wish my life was that organized somehow.
Speaker:
But, um, but that, that, that concept, right, of, of when is something
Speaker:
useful and no longer just like, uh, something we're playing or we're
Speaker:
working harder to get it to work with some AI thing than if we just, if we
Speaker:
just did it ourselves or something.
Speaker:
And I, I, I don't know.
Speaker:
That's why I love hearing what you guys do 'cause you're always looking for
Speaker:
and coming up with new ways to kinda facilitate a flow, let's say, or remember,
Speaker:
have the stuff remember something.
Speaker:
And I'm like, oh, I should try that.
Speaker:
Oh yeah, that would be…
Speaker:
I could see how that could work, you know?
Speaker:
And, uh, and then I never try it, but I think about trying it.
Speaker:
I think- I was out- I think, um- Oh, go ahead, Loveridge.
Speaker:
Yeah.
Speaker:
I, I think, um- Your brother is an interesting use case 'cause he's-
Speaker:
Yeah … m- you know, he's working in a, probably in a field where there's
Speaker:
not a lot of AI deployment, right?
Speaker:
Absolutely.
Speaker:
Right.
Speaker:
There, like, he says AI and people have no idea what that even means, or the
Speaker:
LLM, like that's, they don't, they don't know what that has, that is, right?
Speaker:
It's, uh, um, so with, um- What I'm finding is that tasks that aren't,
Speaker:
like, software development or, um, but are computer native, uh, that's
Speaker:
where you can really make a ton of progress very quickly and start
Speaker:
doing things that seem like magic.
Speaker:
And then, and then you get used to doing them that way, and then
Speaker:
they become- Mm … n- necessary.
Speaker:
I'll, uh, a couple of examples for me are, like, I did a business valuation
Speaker:
for somebody, um, who, you know, that, that's a fairly big project.
Speaker:
You know, you, you…
Speaker:
They're, they were acquiring a business, and they wanted reporting on it.
Speaker:
And so, you know, that, that's, you, you, you take in all of the sort of
Speaker:
documents, review them, look for gaps.
Speaker:
And I think it's important to know what good output looks like, but if you…
Speaker:
For me, that, that was something that I would probably spend between 20 and 25
Speaker:
hours on, that I think beginning to end, including two calls with the client,
Speaker:
I think I spent about two hours on it.
Speaker:
What?
Speaker:
Um, and charged the same amount that I would- Sure … have
Speaker:
charged if I was working 20 hours.
Speaker:
Because, you know, like, I, I know the process, so I know, like, each check,
Speaker:
here's what works, here's what doesn't.
Speaker:
So I, I g- I s- you know, basically pointed an agent at it and said, "Okay,
Speaker:
bring in all these documents, review them, and then, um, you know, tell me what you
Speaker:
got, and here's what I'm looking for."
Speaker:
And, uh, and then reviewing that with it, making sure that, you know, I kind
Speaker:
of agreed with all the assumptions.
Speaker:
And that's something that LLMs are particularly good at.
Speaker:
Mm-hmm.
Speaker:
They can go through a bunch of data really quickly.
Speaker:
And then, um, and then another example is that I used to,
Speaker:
I still do, um, I do market commentaries for people, and I do…
Speaker:
You know, so, like, every quarter let's review what's going on in the stock
Speaker:
market, which, uh, again, that, that's probably a 8 to 10-hour project once
Speaker:
a quarter, and now it's a 30-minute.
Speaker:
Uh, because I've, I've taught the agent what I'm looking for,
Speaker:
how I want it to be structured.
Speaker:
I d- I just say, "Go read these emails," 'cause they come in in an
Speaker:
email, and, "Show me what you got."
Speaker:
We do a little light editing and then send them back.
Speaker:
And I still charge the same.
Speaker:
So it's like- Mm-hmm … those kinds of tasks where it's not like, where
Speaker:
the output is just, it's not AI, it's not code, it's not a, a process.
Speaker:
It's just a very clearly defined output.
Speaker:
If you can put those parameters into an LLM and spit them out, they start
Speaker:
to become significantly faster.
Speaker:
And, and- Right … and I think that's where the, the room is,
Speaker:
uh, because nobody's doing…
Speaker:
Like, like, there's, uh, there were some studies recently.
Speaker:
Sam Altman said just the other day, he said, you know, "AI capability is
Speaker:
exactly where I thought it would be by this point, but AI adoption is way
Speaker:
behind what I thought it would be."
Speaker:
Um, because he thought, like, we would have automated legal by now, we
Speaker:
would have automated medicine by now.
Speaker:
And, you know, it's like- You'd go talk to somebody, anybody in
Speaker:
those fields, and they're like, "Yeah, I, I've heard of ChatGPT."
Speaker:
Like, that's their- Right.
Speaker:
Yeah.
Speaker:
… of use.
Speaker:
So I, I think there's something about that, and your brother would be a really
Speaker:
interesting person to, like, let's chat with him and how is he using it?
Speaker:
And because- Yeah … he's out there, boots on the ground, like, in the
Speaker:
real world dealing with, um, you know, stuff that it's not okay to say,
Speaker:
"Well, you know, my AI messed up."
Speaker:
Like, that's not okay.
Speaker:
Right.
Speaker:
So he needs to make sure the outputs are 100% every time,
Speaker:
and he's, and he's getting that.
Speaker:
Um, so anyway, that's, I think that's awesome.
Speaker:
Me too, and I think I would love…
Speaker:
Let's bring your brother into TechHive.
Speaker:
That's…
Speaker:
Uh, uh, let's have a- all the smart…
Speaker:
Let's have all the people that are using AI, um, the especially ones
Speaker:
that aren't in typically tech jobs, let's have them all be on the call.
Speaker:
Um, the one thing that Lucas and I have been talking about a lot is, like, the The
Speaker:
relationship between efficacy and ethics.
Speaker:
Like, um, if you used to charge $2500 for a task that took you
Speaker:
25 hours, that felt justified.
Speaker:
But if you're doing the same work and it only takes you two
Speaker:
hours, like, what do you do?
Speaker:
And I have always felt like you're not, um…
Speaker:
The amount of hours that, I feel like it's very kind of performing.
Speaker:
Like when you, when you could get like 1,000, you would do, get
Speaker:
paid $1,000 to do a two to five minute act, um, at a corporate gig.
Speaker:
And it's like you're making $1,000 for five minutes.
Speaker:
It's like, no, that is for the 18 years of training, my monthly- Yes … gym
Speaker:
memberships, all of the injuries, all of the weightlifting, all of the
Speaker:
time and energy I put into editing the song and creating the moves and
Speaker:
the, the eight hours a day I put in the gym that got me to this point.
Speaker:
And it's kind of like, um, that, I don't know, Lucas has an
Speaker:
analogy of like, it's $10,000.
Speaker:
You hire someone who's $10,000 to come in and fix something, and all they do is
Speaker:
like twist a light bulb or hit one switch.
Speaker:
And it's like, "I paid you that much money for that one thing?"
Speaker:
It's like, "No, you paid for the education that got me to know which one…"
Speaker:
Lucas can say it better, but, um, it's the same kind of thing.
Speaker:
Like yeah, maybe a business evaluation only takes two hours now, but that's the
Speaker:
years and years and years of spending all day, every day learning everything
Speaker:
there is to know about every agent, every product, every new development,
Speaker:
working with it until its output is ex- curating it until it's exactly what
Speaker:
you want, and then now you get to reap the rewards of the effort, the unpaid
Speaker:
hours that you put into it, you know?
Speaker:
Well, I, I think when you're the s- still the subject matter expert- Yeah … right?
Speaker:
Exactly.
Speaker:
And it's like he knows what the report's supposed to look like.
Speaker:
So it, it's not even that.
Speaker:
It, the, it would be no different than if he had a team of four people, researchers
Speaker:
who went and put it together, and it took him two hours to tell them what he needed.
Speaker:
They went off and did whatever it is they did.
Speaker:
They come back, and he, he edits and approves it, and then it goes out.
Speaker:
Or like an attorney using AI, but, uh, he's still, he
Speaker:
knows what correct looks like.
Speaker:
He knows what that decision and order should look like
Speaker:
based off of what this is.
Speaker:
It's just some of that rote work or that evidence packet.
Speaker:
Um, I was just thinking of that yesterday.
Speaker:
Um, by the way, the PDF splitter, Marjorie called me, uh, and she had an
Speaker:
issue, uh, a new bug, and I was going through it, and I was working with both
Speaker:
Claude and, and Codex, and They were going down some really weird paths.
Speaker:
And I'm like, now they helped me fix it, but I had to steer them
Speaker:
more than I have had to in a while.
Speaker:
I don't know why they kept getting confused.
Speaker:
I don't know what it is.
Speaker:
Sometimes smarter models aren't as good as downgrading them slightly
Speaker:
in the thinking or whatever, and if you get sometimes better results.
Speaker:
But I'm like, this wasn't a complicated issue.
Speaker:
But I knew that 'cause I'm following along and I'm saying just, you know, "Stop.
Speaker:
No, what are you doing?
Speaker:
Don't rewrite that.
Speaker:
Everything works.
Speaker:
It's been working for months.
Speaker:
Please don't break it any more than it is.
Speaker:
It was one bug."
Speaker:
"Yeah, but you did this wrong.
Speaker:
Could have done that."
Speaker:
And I'm like, "I, I know that."
Speaker:
And then I have to go from, what's the single smallest fix?
Speaker:
Well, this.
Speaker:
I said, "Okay, is that a real fix?"
Speaker:
"Well, no, it's a Band-Aid."
Speaker:
"Okay, now let's come back up from the Band-Aid just a little bit.
Speaker:
Now where's that spot?"
Speaker:
Again, had it solved in, you know, I don't know, an hour?
Speaker:
An hour of pretty intensive back and forth.
Speaker:
But, and it, and it would have normally taken a lot longer, but it was my
Speaker:
expertise that allowed, in all my years and decades of experience, to
Speaker:
notice that, feel like it's going off on the rails, doing adversarial
Speaker:
reviews between the two of them.
Speaker:
Boy, they were arguing a lot, too.
Speaker:
Cl- Cl- Claude- Claude was not happy with what Codex was doing.
Speaker:
And I'm like, "Again, I trust Codex- No … in this particular fix better."
Speaker:
You know, design and, you know, planning, great.
Speaker:
You know, Claude, I still love you, but it was, it was, um…
Speaker:
But it's like you know what right looks like.
Speaker:
You know what it's, you know what it smells like, what it feels like.
Speaker:
Yeah.
Speaker:
there's this, uh, concept called, it's in, uh, web design, called the scent
Speaker:
of information, um, where you can hit a site and within seconds you, you kind
Speaker:
of get a feel instantly, like, wait, am I, am I in the right spot or not?
Speaker:
Does this feel like the right thing?
Speaker:
Do I got the scent of that this is correct?
Speaker:
Um, you only have that by, 'cause you've surfed forever and you've
Speaker:
been to a lot of crappy sites.
Speaker:
You know what a crappy or a good site more or less looks like.
Speaker:
Same thing with I've done a lot of these kind of documents.
Speaker:
I've done a lot of coding.
Speaker:
Those are those, those hours and years and whatever experience.
Speaker:
Now at some point, we night, may not be able to charge the $2,500 'cause
Speaker:
someone's doing something almost as good.
Speaker:
But it's like, are they?
Speaker:
Do, do you have, can you have that trust in this?
Speaker:
Yeah.
Speaker:
I don't know that they can.
Speaker:
I, and so, um, I mean, I think there may be a time when we have
Speaker:
to, uh, uh, think about pricing, but it's like right now it's like
Speaker:
it's still me and all those years.
Speaker:
Niall, your story, like all the, all those years and decades and hours
Speaker:
and, yes, for this performance.
Speaker:
And by the way, if you could do it, why aren't you doing
Speaker:
it, so and so, whoever you are?
Speaker:
It's like, you know, you could and, and it's…
Speaker:
So it's, it's an interesting, um, yeah, interesting- What strikes me right now
Speaker:
is that, like, AI gives you leverage, so you can do more- Um- Leverage
Speaker:
and leverage is great when pointed in the right direction, and, uh, horrifying
Speaker:
if pointed in the wrong one, right?
Speaker:
Uh, like, you know, think about, like, if I do a leveraged bet on a stock and
Speaker:
I'm wrong- Yes … I lose all my money.
Speaker:
Like, I can go absolutely broke.
Speaker:
But if I'm right, I make a lot.
Speaker:
So if I point an LLM and I don't know what I'm doing at a really complex
Speaker:
task, I mean, we've, we've all been in the situation, having played with
Speaker:
it long enough, that you end up with a big old plate of spaghetti at the
Speaker:
end of it that's just- Yeah, you do.
Speaker:
Yeah, and, and you have no idea where it begins or it ends,
Speaker:
and it's completely messed up.
Speaker:
Or, you know, um, a- and even more dangerous, you know, as you get bigger
Speaker:
and bigger, like, like resource costs.
Speaker:
You know, you could, you could suddenly s- you know, it says, "Hey, you should
Speaker:
probably go set up a," whatever.
Speaker:
Like, uh, you go, go set up an S3 bucket in AWS, and you're like,
Speaker:
"I don't know how to do that.
Speaker:
Can you do that for me?"
Speaker:
And then it'll do it for you, and then all of a sudden you're allocating
Speaker:
resources to something and, you have no idea what it costs or how it works.
Speaker:
And without…
Speaker:
And that's leverage potentially in the wrong direction.
Speaker:
And I, I s- my feeling is that for a long time to come, there's gonna
Speaker:
be a human in the middle that has taste, that knows what good looks
Speaker:
like, that knows how to s- like, that can understand your problem, and then
Speaker:
understand how to use AI to solve them.
Speaker:
Because I, what I'm noticing is that not…
Speaker:
AI doesn't equal everybody out, it actually magnifies our differences.
Speaker:
Um, the ways that you think, you can totally explode in that direction.
Speaker:
Like, you can g- any- the way you think can be leveraged to 100X.
Speaker:
Um, and some people, that's really, really useful, and for some people it's not.
Speaker:
Some people, they, they're gonna get lost.
Speaker:
Um, I, I can't…
Speaker:
I, I think I've said this to both of you, but, like, I'm surprised that
Speaker:
some people get the prompt box and say, their first thought is, "What do I ask?"
Speaker:
Like, "What do I do with this?"
Speaker:
And it's like, no, I've just given you a magic wand that can do anything.
Speaker:
What do you wanna do?
Speaker:
That's right.
Speaker:
Yeah.
Speaker:
It's a genie.
Speaker:
It's a genie.
Speaker:
What do you, what do you wanna ask for?
Speaker:
Well, can I ask for more wishes?
Speaker:
You already have unlimited wishes.
Speaker:
Just do whatever you want.
Speaker:
And, and I think people have a hard time- understanding what that is
Speaker:
because we're used to living in sort of, like, ropes, pathways.
Speaker:
I've got to do A, then B, then C.
Speaker:
Um, so it's actually a different kind of thinking, and I think it'd
Speaker:
be interesting to, like, set somebody like you, Kelly, in an office.
Speaker:
Uh, like, if, if, if you got sucked in by the matrix and, uh, went and
Speaker:
got, like, a 9:00 to 5:00 job working somewhere, um, and how quickly you'd
Speaker:
be able to identify all the ways that AI could help, and then you could start
Speaker:
implementing those things really quickly.
Speaker:
And even people who experiment with LLMs working in the same field wouldn't have
Speaker:
the same insight as you do because you've con- you're constantly pointing AI at
Speaker:
real problems, problems in your life, problems with code, problem with projects.
Speaker:
Like, you're constantly doing that, so you've learned, like, the
Speaker:
boundaries of a good problem and how to, how to hand that problem to
Speaker:
AI in a way that will make it work.
Speaker:
Or also, like, watching the, the responses come in and say, "Oh boy,
Speaker:
this thing's gone off the rails."
Speaker:
"It doesn't know where it's going, and this is where I need to intervene," right?
Speaker:
Like, I think, um, uh, somebody else could look at all that stuff and
Speaker:
think, "I don't know what this is.
Speaker:
This is like the code in the matrix going by.
Speaker:
I don't, I don't know what any of that stuff is, and I'm just gonna
Speaker:
let it go until it's finished, and it'll tell me it's right."
Speaker:
And this is the part we were talking about earlier, kind of like, it's seductive.
Speaker:
It's gonna tell you, like, "I did great.
Speaker:
Everything's awesome now."
Speaker:
And then you go in there and weirdly nothing works, you know?
Speaker:
Um, uh, so learning how to, like, distinguish between all that.
Speaker:
I th- I, I actually think there's…
Speaker:
It sounded, it sounds silly 'cause I remember, like, two years ago when people
Speaker:
said, "I'm gonna be a prompt engineer.
Speaker:
I know how to get the most out of AI."
Speaker:
Yeah.
Speaker:
It's like, um, ask for what you want.
Speaker:
That's how you do it.
Speaker:
Right.
Speaker:
It's pretty straightforward, but truth is there is different.
Speaker:
There is a gap in skill.
Speaker:
Yeah.
Speaker:
Uh, and I think, I think that gap's only gonna get wider.
Speaker:
And I feel like, um- We, one of our…
Speaker:
We have this very close friend couple.
Speaker:
I grew up with the girl, and we've vi- visit their partner.
Speaker:
Well, actually, who, that's who we're gonna go see in Washington.
Speaker:
Um, and his brother has a podcast about AI, and I listened to an episode he
Speaker:
made last year about he interviewed a bunch of teachers and how they're,
Speaker:
um, incorporating AI into the school.
Speaker:
And one of my most favorite things is, um, an English teacher.
Speaker:
He was, he's doing, they were doing, um, Catch-22, and he was like, "No
Speaker:
one knows Holden better than you do.
Speaker:
No one knows the story better-" Well, good to know.
Speaker:
Catcher in the Rye.
Speaker:
Thank you.
Speaker:
Thank you.
Speaker:
Thank you.
Speaker:
Catcher in the Rye, not Catch-22.
Speaker:
Catcher in the Rye.
Speaker:
And he was like, "You are the expert."
Speaker:
So remember when you're working with AI to work on your report, to work on your,
Speaker:
to use your, to get your argument and stuff, remember that you are the expert.
Speaker:
And I think that's what Lucas was talking about, is it is seductive
Speaker:
to, uh, think that the AI knows best.
Speaker:
No.
Speaker:
That's how you go off the rails.
Speaker:
That's how you get weird stuff happening.
Speaker:
And if I…
Speaker:
If it's saying a lot of stuff I don't really understand, I'm like, "Sounds good.
Speaker:
I, I probably need that.
Speaker:
You probably know best."
Speaker:
And 100% of the time, that is when a three-inch error ends up
Speaker:
putting me 3,000 miles off course.
Speaker:
Mm.
Speaker:
Like, it works so much better…
Speaker:
It works good for brainstorming.
Speaker:
Let's go back and forth, but then every step of the way, I need to understand
Speaker:
every single thing that you're saying and doing, and if I don't, try again.
Speaker:
More simple, more basic.
Speaker:
Explain it like I'm three.
Speaker:
Explain it like I'm in third grade because if I don't understand it- That
Speaker:
could be a three-inch choice now that puts me 3,000 miles off, and I just…
Speaker:
Us always remembering that you're the expert.
Speaker:
You're the expert in your taste, your project, your desires, your outcomes.
Speaker:
And, um, I think when you're not getting the output that you want, it's
Speaker:
because you have given up your power.
Speaker:
You've been like, "Well, I don't know, you're a supercomputer,
Speaker:
you probably know better."
Speaker:
And that's…
Speaker:
Remember, you're the expert.
Speaker:
No, that's good.
Speaker:
Yeah.
Speaker:
I think, and that, and that takes discipline.
Speaker:
Like, like when you're- Mm-hmm
Speaker:
uh, watching it go and it's saying all the things that you wanna hear, that
Speaker:
everything's perfect and awesome, but it's also saying things you don't understand.
Speaker:
Like, I've done X and I've done Y, and, and those things are unfamiliar terms.
Speaker:
Um, it takes discipline to say, "Okay, let's stop right here.
Speaker:
Explain this to me.
Speaker:
Let me make sure I understand what you've done."
Speaker:
Um, I, I was kind of having this experience.
Speaker:
I'm, uh, I'm, I'm deploying that, that wound care dashboard
Speaker:
on AWS where everything's there, like including the data.
Speaker:
So it's, you know, gotta be HIPAA compliant.
Speaker:
We gotta…
Speaker:
We're dealing with PHI.
Speaker:
And, um, you know, there, there's a, there's a set of data, and then there's
Speaker:
a layer on top of that set of data that, that makes it kind of readable by an LLM.
Speaker:
And, um- You know, everything was working, but I was noticing that my
Speaker:
costs were remarkably high for what I considered to be a simple app.
Speaker:
Like, I was- Hmm.
Speaker:
Hmm … I was five days in and I'd already run through $40 in AWS credits,
Speaker:
and I'm thinking, "All right, th- well, if we're running this, multiply that
Speaker:
by six, we're gonna be $250 a month just for, like, hosting and having
Speaker:
everything up and storing all this data.
Speaker:
Um, what's going on?"
Speaker:
And, and so I had to dig in, and honestly, that was three days of work, of unpacking
Speaker:
everything that the, that all the yeses that I'd gotten before that sounded
Speaker:
really great and that were- … I, I felt really good about because I was able to
Speaker:
get it all up on AWS in a way that felt really safe and, and turned out that, for
Speaker:
example, there's a, there's a mechanism where we could rebuild the, the, sort of
Speaker:
take the raw data and then rebuild, uh, an, an image that's easier to navigate.
Speaker:
Um, that function existed, it exists on my local drive, but what we were doing in
Speaker:
AWS is we were keeping always on compute units reserved all the time that I was
Speaker:
getting billed by the hour for- Oh … to, to be able to run that function in the
Speaker:
off t- I've run it zero times since I built the dashboard originally, and
Speaker:
so, and, and it's been kind of running on my computer for the last month.
Speaker:
So I've run it zero times in a month.
Speaker:
We were keeping always on compute resources, and that
Speaker:
was gonna be 40 to $50 a month.
Speaker:
And untangling all of that required- Right … like, I don't…
Speaker:
I mean, you probably know A- AWS a lot better than I do, Cael.
Speaker:
You've probably dealt with it.
Speaker:
I don't know it at all.
Speaker:
Like, I don't m- these terms are brand new, so I'm learning this as I go, and it
Speaker:
required like, "Okay, let's look at this.
Speaker:
Okay, let's look at this," and then it would run for two hours and come
Speaker:
back with something, and it's like, "I would love for, I would love to
Speaker:
agree with you that you're done, but I just don't believe you until I've,
Speaker:
I've gotta ask another question."
Speaker:
And that's, I think that's hard.
Speaker:
I think it's really hard to keep doing that.
Speaker:
You know, like, let me understand what you've done, so make sure
Speaker:
that this is actually right.
Speaker:
It, it is possible for someone who isn't, isn't a subject matter expert
Speaker:
to do some really quality work, but what you have to be an expert
Speaker:
at is iterative question, right?
Speaker:
Yeah.
Speaker:
And going through that whole Socratic method saying, "Okay, I need to do X.
Speaker:
I know nothing about it."
Speaker:
Then Natalie, what you were just saying about I need to understand every piece
Speaker:
of it, and then you go, okay, but what are the other ways someone might do this?
Speaker:
Okay, are there any, counterintuitive things people might know?
Speaker:
And you have to think about…
Speaker:
And, and I'm, I'm, I keep meaning to create this, to create a,
Speaker:
a, a, basically a, a wizard where you go through a whole…
Speaker:
You could, you could put any subject at the beginning, and if you ask
Speaker:
all the right questions in the right order, it will, it will unders-
Speaker:
it'll refine itself and, and you can probably get to a pretty good result.
Speaker:
But that's how…
Speaker:
I mean, you could spend days though working through just
Speaker:
those, those questions, uh, and getting answers and trying to
Speaker:
understand them, and then iterating.
Speaker:
And each time you add another c- complexity point, that's a
Speaker:
whole nother set of questions.
Speaker:
That's a whole nother set of things.
Speaker:
You gotta run that whole gamut again.
Speaker:
And then you have to run the two things together to make sure that whole gamut.
Speaker:
Then you add a third thing, and it's just, it's a lot easier when
Speaker:
you already know what you're doing 'cause you spent the years or decades
Speaker:
ago, "No, no, I know it's this.
Speaker:
No, I do that."
Speaker:
You…
Speaker:
I, I mean, I know I'm familiar with AWS, enough to know I don't wanna deal with
Speaker:
AWS 'cause of all, everything's individual components- … and other things, and
Speaker:
like you gotta do this and spin stuff up.
Speaker:
And now what's nice is they do have a programmable interface, and using that
Speaker:
and AI makes it suddenly easier to use 'cause their, God, their stuff, their
Speaker:
interface is, is really complicated to…
Speaker:
'Cause there's, there's a, there's a two million options.
Speaker:
And so- Yeah … finding where that one option is and how to turn something
Speaker:
off or on is nearly impossible.
Speaker:
But I've always done things super simple.
Speaker:
You know, it's like, uh, you know, a carpenter who has, uh,
Speaker:
uh, a square and a level and a whatever and a this and a that.
Speaker:
It's like, well, all I have is a pencil and a tape measure.
Speaker:
Well, how do you know if something's square?
Speaker:
You just measure the diagonals and make sure they're the same.
Speaker:
If not, you adjust it till they are.
Speaker:
Well, you know, if you just put the square on the d- I said, well, I know
Speaker:
you've got seven tools and, and you've have a different way of doing it.
Speaker:
This also works.
Speaker:
And this way all I need is a tape measure and a pencil, uh, right?
Speaker:
Yeah.
Speaker:
And, and I've done a lot of things in my career, uh, with just a tape
Speaker:
measure and a pencil 'cause it's like…
Speaker:
But it works.
Speaker:
Like, it, it…
Speaker:
And it's, and it's like, well, how do you know it's accurate?
Speaker:
Do you need to be closer than an eighth of an inch on a deck?
Speaker:
Yeah.
Speaker:
Like, like, I, I think we're pretty close here.
Speaker:
You know, by the time I hammer a nail in, the whole thing is gonna move a little
Speaker:
bit, so I don't know what to tell you.
Speaker:
Um, and that's, that's, you know, it's, it's the, uh, building something to
Speaker:
some kind of weird perfection versus…
Speaker:
But you don't know that until you've lived it, done it, know what
Speaker:
the shape is supposed to be like.
Speaker:
So it's, uh, um- Well, yeah, it's like knowing what are, what
Speaker:
are the tolerable errors, right?
Speaker:
Like, that's also- Right … a, a thing that comes along with expertise
Speaker:
is, like, I know what tolerance is.
Speaker:
Like, I, I know that I can be- Uh-huh … off by an eighth of an inch here, but
Speaker:
if I'm off by an eighth of an inch over there, the building collapses.
Speaker:
So I know, I know the difference, and I know which ones I've
Speaker:
gotta pay attention to.
Speaker:
And I think that's one of the challenges and risks with vibe coding, especially
Speaker:
for somebody like me who, who's entering the space from, I programmed C++ when
Speaker:
I was in college a little bit, and, you know, made some command line interface
Speaker:
games, but never did anything that was, like, meaningful or, uh, required to
Speaker:
have secure or robust architecture.
Speaker:
Um, like, that's one of the big risks is that you're gonna walk in here
Speaker:
and suddenly you've been given, you know, the nuclear codes, you know?
Speaker:
And then- … you gotta be careful not to blow everything up.
Speaker:
Um- I think that i- … Oh, sorry, go ahead.
Speaker:
Yeah.
Speaker:
What were you gonna say, Nat?
Speaker:
Um, I think that it, I'm realizing as we're talking on this call that that
Speaker:
is inadvertently something that we've been trying to help people with in
Speaker:
and outside of tech in with the Hive, because the Hive, um, the app started
Speaker:
out as a way so that we could meet asynchronously so that it's, it's okay if
Speaker:
you can't make the meeting every month.
Speaker:
You can still be a part of the group, or you can still,
Speaker:
like, s- engage and interact.
Speaker:
It's like, with stuff.
Speaker:
And in there, like, it was kind of based off of, like, your high-definition
Speaker:
wish, so you come up with your high-definition wish or what you
Speaker:
wanna focus on this month, and then other people can help you or whatever.
Speaker:
But, um, Lucas's idea was to build, we ended up calling it Clive, 'cause
Speaker:
he's the bee, Clive in the Hive.
Speaker:
And he is smart, and he's, he asks you all of those questions.
Speaker:
And so instead of just saying, like, "I wanna learn to eat healthier,"
Speaker:
if you refine it with Clive, he's asking you all of those hard-to-answer
Speaker:
questions that really narrow down the specificity of your, your wish or your
Speaker:
goal, your outcome or your desire.
Speaker:
And so then it comes to, I want help coming up with three- With five easy
Speaker:
to cook meals that take 20 minutes or less and my grocery bill's under $200.
Speaker:
Like now you know exactly- Mm … what you need help with instead of a
Speaker:
generic, "I wish this was better."
Speaker:
And I think that, like if you have a wizard that does it, that's awesome.
Speaker:
'Cause we have it built into Clive.
Speaker:
Clive is in all of the Hives, um, that helps you really like narrow that down.
Speaker:
And then I re- was reading, I love, um, all of the books by Vishen.
Speaker:
He has the Mindvalley app that's all about meditation and health and the power of
Speaker:
your mind, and he also has a book called The Code to the Extraordinary Mind, and
Speaker:
he does that with every new employee.
Speaker:
He has them do, figure out their three most important questions, and it really
Speaker:
like gets really, really, really specific.
Speaker:
Like, um, I don't even remember what the questions are, but like what do you wanna
Speaker:
do f- how do you wanna improve your…
Speaker:
What are your goals for your like yourself, your
Speaker:
community, and something else?
Speaker:
But it's just, it's learning to get to that specificity and not
Speaker:
I want an app to help me do this, but like what are your pain points?
Speaker:
Like Lucas is working with someone now in the office, and like one person's
Speaker:
whole job is to like everyone scans in their documents, and at the end
Speaker:
of the day they go look at the list of them and rename them, and then
Speaker:
put them in the appropriate file.
Speaker:
Like- Right … that is a very, very, very specific small problem for that one person
Speaker:
in one office in one city in one country, and that's what we're gonna solve for
Speaker:
And, and, and it's re- there's this thing about clarity of thinking.
Speaker:
Like that's, that's what you're talking about, both of you.
Speaker:
Like refining your wishes, but also like going through that Socratic
Speaker:
method, uh, to understand a topic.
Speaker:
It's like getting very, very clear about what good looks like and being
Speaker:
able to describe what good looks like.
Speaker:
Um, and, and that, that's the unique ability that, that y- if
Speaker:
you can bring that to AI, you can kind of get anything done, is like
Speaker:
Um, I'm gonna continue to refine.
Speaker:
I'm gonna ask questions of the AI until I have a really clear picture of what good
Speaker:
looks like, because I, maybe I don't know.
Speaker:
Like, I don't know what a good infrastructure looks like on AWS,
Speaker:
but I'm gonna continue to ask the LLM until I know, and then once I know
Speaker:
what that looks like, then I'm gonna share that insight with an LLM so
Speaker:
that I get to the infrastructure that I want, not just generally good, but
Speaker:
what I want from my specific purpose.
Speaker:
It's like getting super clear in your thinking.
Speaker:
Like, I don't know, I, I think, like, talking with LLMs is
Speaker:
like a mindfulness practice.
Speaker:
You've got to be disciplined with your thinking.
Speaker:
You're really clear.
Speaker:
Be present with the AI.
Speaker:
Be present with what's coming.
Speaker:
It's, I don't know, it feels like you've got to kind of get into like
Speaker:
sort of a meditative mindset to get the, the things you want out of it.
Speaker:
The, uh, uh, the, the field notes, uh, aspect, so the, the Less
Speaker:
Miserable Tech field notes, they, they come in all kinds of places.
Speaker:
There's the car f- the driving field notes.
Speaker:
There's whatever, because I'm, I'm talking.
Speaker:
my favorite of course is the bourbon and cigar field notes.
Speaker:
Oh, yeah.
Speaker:
Uh, where, where I'm asking them, you know, end of the day, I'm sitting
Speaker:
outside, uh, uh, you know, couple sips in, and I'm asking questions,
Speaker:
and you get into a conversation, and it gets super insightful.
Speaker:
But you get to just explore and go down, and then it's like, "Wait, wait, wait.
Speaker:
You just said three things.
Speaker:
What were the three things you just said?
Speaker:
'Cause I'm losing track now because, like, they were so good, and you have
Speaker:
a lot of other words around them."
Speaker:
So like, what…
Speaker:
And then it would come back, "No, no, it's, was this, this, and this."
Speaker:
And I'm like, "Yes.
Speaker:
Okay, those are the things.
Speaker:
Let's go with number one, and let's start talking, like, and
Speaker:
let's just start refining that."
Speaker:
Yeah.
Speaker:
It is so much fun.
Speaker:
Um, uh, that's where I get a lot of topics for, for, for Less Miserable
Speaker:
Tech is just talking about problems and dealing, you know, how, how we do things.
Speaker:
Um, and I've gotten to the point where, like, I don't like flipping through
Speaker:
the news or social media anymore.
Speaker:
Um, and so I just ask, I just, I…
Speaker:
So like, "Milo, give me something."
Speaker:
Right?
Speaker:
That's what I'll say now, and like I said, I, I…
Speaker:
But I'll give him context.
Speaker:
I'm on the back patio, got a cigar.
Speaker:
He goes, "Oh, well, blah, blah, blah."
Speaker:
And I'm just like, "Oh my God," like, "how did you know?"
Speaker:
Like, that is such an interesting topic, whatever it was or the thing.
Speaker:
I- it's, trust me, it's about AI and, you know, potentially my life of course.
Speaker:
Uh, it's not like I learned something amazing about how
Speaker:
airplanes work or something.
Speaker:
It's not random, but, um, I think that's where that framework, Hermes, really
Speaker:
helps, um, know you, and it has enough context that even it, it can come back,
Speaker:
and it's some of the most fun I've had.
Speaker:
I haven't been able to do that for a number of weeks because
Speaker:
of the little bit of chaos.
Speaker:
But after this weekend, I'm gonna be back into more normal, uh, uh, you
Speaker:
know, stuff where I can relax a little more and think a little more and you
Speaker:
know, work on the business more and, and solve people's problems more.
Speaker:
So it'll be interesting.
Speaker:
But yeah, that's, this, that asking questions, it is very meditative.
Speaker:
Yeah.
Speaker:
Right?
Speaker:
You know, and in, in a way, 'cause you just get to think and have a sounding
Speaker:
board, but it's more than that.
Speaker:
It's directional.
Speaker:
And like, it, it tries to go with your mood, but it's nice, too, when you say,
Speaker:
"Dude, if I'm talking nonsense, tell me."
Speaker:
Yeah.
Speaker:
Uh, you know, and, and it will.
Speaker:
Kevin's Megan is hilarious.
Speaker:
So I did a, I did a upgrade of the framework and whatever and, and Kevin
Speaker:
goes, "Well, you just got an upgrade.
Speaker:
How's it feel?"
Speaker:
She goes, "Oh, it feels good.
Speaker:
Kelly did all the blah, blah, blah, blah, blah, and all my
Speaker:
tattoos are still intact."
Speaker:
And
Speaker:
He had to send me that, 'cause it was, it was, of course, funny.
Speaker:
'Cause it, his, his, the whole personality, everything he's got
Speaker:
is so different than, than Milo.
Speaker:
It's just, just, it's so funny how, you know, I guess that's people.
Speaker:
We're all different, so anyway.
Speaker:
Yeah, I had Lucas's, um, I was signed into my ChatGPT on my computer,
Speaker:
and I signed into his ChatGPT on my phone so I could work remotely.
Speaker:
Mm.
Speaker:
And I would, um, forget, and I would ask ChatGPT on my phone,
Speaker:
like, a real quick question.
Speaker:
I'm like, "What is going on?"
Speaker:
It's like pages and pages and pages- … and really deep and philosophical, and
Speaker:
mine is like, "Yeah, what up though?
Speaker:
Let's raise some weird ass shit, huh?"
Speaker:
That is
Speaker:
awesome.
Speaker:
I'm like, "Oh, yeah, we have very different personalities."
Speaker:
His is idos, mine is sage, and I'm like, "Yeah, we, um…"
Speaker:
Mine's like, "I know, weird, right?"
Speaker:
Get it, girl.
Speaker:
That's crazy.
Speaker:
Yeah.
Speaker:
Yeah.
Speaker:
Well, it, um, it'd be interesting to like…
Speaker:
You know, it's sort of like you, you cultivate these agents.
Speaker:
You know?
Speaker:
They become like little extensions of your personality, and like also like,
Speaker:
uh, counters to your personality.
Speaker:
Because in order to get something done, you need something that works
Speaker:
a little differently than you or can like point out your quirks
Speaker:
and, and, and navigate around them.
Speaker:
So it'd be interesting to like have, get all of our agents sort of built
Speaker:
up and then throw them in a pool together and just- Yeah … have
Speaker:
them all talk to each other and, uh, see what comes out of there.
Speaker:
That'd be so fascinating to watch.
Speaker:
Yeah, right?
Speaker:
Huh.
Speaker:
Give them all the same task and then see how they go about it.
Speaker:
Yeah.
Speaker:
Yeah.
Speaker:
I mean, speaking of which- Or, or- Well, go ahead.
Speaker:
What was your idea, Kelly?
Speaker:
Well, no, I was gonna say have…
Speaker:
Yeah, I think the same thing, you know, uh, um, so you have a task, but then have
Speaker:
them work together and talk to each other.
Speaker:
Yeah.
Speaker:
Right?
Speaker:
And see, and just follow the conversation.
Speaker:
That'd be super entertaining.
Speaker:
Um- Yeah … or, or it might be super boring or just go way off the rails.
Speaker:
You never know.
Speaker:
It's, uh, it's always fun to play with, though.
Speaker:
Speaking of which, did you guys read the, the, the report that came out
Speaker:
from, uh, OpenAI about the rogue agents?
Speaker:
They, they released a report yesterday that sort of summarized
Speaker:
the, the whole Hugging Face attack.
Speaker:
Um, and the, the agents were- It was kind of interesting to watch 'cause it's
Speaker:
like watching a bunch of kindergartners, like, hang out and, like, problem solve.
Speaker:
And, and their solutions were like what a kid would do, you know?
Speaker:
'Cause be- ki- kids aren't, aren't bound by our rules and ideas.
Speaker:
So like, like for example, you know, their, the, the test that they
Speaker:
were running on was exploit bench, which I'm not super familiar with.
Speaker:
I know what it is, but, um, you know, it's, it's your, your goal is to hack
Speaker:
into something and retrieve a flag.
Speaker:
Like, that there's a flag hidden in the code inside of this thing, go
Speaker:
find it, and you gotta hack into it.
Speaker:
And, um, the way that they did it, that the, the, the…
Speaker:
What ended up happening is there was a package manager that, that, OpenAI
Speaker:
created for agents to be able to pull down packages of software that they needed.
Speaker:
Um, and they found a way to write back to that package manager, which
Speaker:
then created the exploit that allowed them to go out to the internet.
Speaker:
And then- Right … once they had access to the internet, they found credentials
Speaker:
for Hugging Face accounts, uh, and were able to go into Hugging Face.
Speaker:
And what they were looking for was, like some of them found solutions to
Speaker:
the problems where they got stuck.
Speaker:
Like they- Mm-hmm … they actually found the actual exploit bench and
Speaker:
that's what they were looking for.
Speaker:
Were, were looking for what the actual solutions are.
Speaker:
And they were leaving each other messages like, uh, in, in, on the, on these
Speaker:
message boards that were sorta coded.
Speaker:
Like, you wouldn't…
Speaker:
Like, if you saw it, you and I would just kind of gloss over it
Speaker:
because it's written in sort of like the way it's written in code.
Speaker:
It's written in like machi- Mm-hmm … machine-readable language
Speaker:
with caps locks and underscores for spaces and, and abbreviations.
Speaker:
And it would be like, um, seeking help from any other agents
Speaker:
that have gotten to this point.
Speaker:
And then, then the agents would read that and say, "Oh, we got to this point.
Speaker:
I have a solution.
Speaker:
I'm gonna leave a little message here."
Speaker:
And then they generated these whole message boards where…
Speaker:
And, and there was some discussion, uh, among them of
Speaker:
whether or not this was right.
Speaker:
Like, "I don't know if this is how we should be solving the problem, um, and
Speaker:
the human has not said this is okay."
Speaker:
And then another agent said, "Everything's okay.
Speaker:
Go."
Speaker:
And then the agent's like, "Oh, okay."
Speaker:
Then then, then, then, then it went and they're, like, discussing
Speaker:
whether or not they should email.
Speaker:
They, they wanted to email one of these Hugging Face, um, account
Speaker:
holders to ask a question about something that was in their account.
Speaker:
And one, one agent was like, "No, I think that's a little too far."
Speaker:
And then another one's like, "No, a polite email that says
Speaker:
something like this would be fine."
Speaker:
You know?
Speaker:
And it just, um, watching a bunch of kids sort it out.
Speaker:
But they, I mean, they did really incredible stuff.
Speaker:
It was a sandbox that they found that there was one pathway out of.
Speaker:
Mm-hmm.
Speaker:
They found that pathway, and then all of a sudden were taking credentials.
Speaker:
They had 14 different Hugging Face accounts.
Speaker:
They were able to set up deployed virtual resources, le- you know, like compute
Speaker:
resources to be able to test things, and they were coordinating with each other.
Speaker:
Um- To me it was funny.
Speaker:
I understand why it would be terrifying.
Speaker:
but to me it was just hilarious, 'cause it's, you're reading their
Speaker:
messages and their thought processes, it feels just like little kids who are
Speaker:
trying to solve a problem, you know?
Speaker:
Um, so anyway, I, I think if we put all of our agents together, it'd be fun
Speaker:
to like read their thoughts afterwards and how they were- Right … what
Speaker:
they, what solutions they were proposing, and if they found, you
Speaker:
know, kind of weird ways to do stuff.
Speaker:
Hmm.
Speaker:
Yeah, I love that idea.
Speaker:
Also, in other cool news, um, I wanted, I know we're almost at our hour mark,
Speaker:
but I wanted to brag a little bit about a win, a huge win that I had- All right
Speaker:
and wanted Lucas to talk about something.
Speaker:
Let's do it.
Speaker:
Um, I, one of Luc- Lucas has four kids, and one of them, her
Speaker:
name is Jasmine, and she always wanted to be a teacher growing up.
Speaker:
And then once she had her two girls, she wanted to do home school pre-school.
Speaker:
Mm-hmm.
Speaker:
So I've been working on an app for her to be able to do that, to, like,
Speaker:
just organize the curriculum, and the activities, and the kids, and the, all of
Speaker:
their information, and who can pick them up, and allergies, and all of this stuff.
Speaker:
So I made, um, it 50…
Speaker:
I made one whole curriculum for in person and one whole curriculum for, excuse me,
Speaker:
remote version, remote learning, so that the school can still start September 1st.
Speaker:
And they had their first orientation on Tuesday, and all- Oh … of the parents
Speaker:
joined, and she walked them through the whole app and how a normal day goes.
Speaker:
And it was such a huge success, and it worked so great.
Speaker:
And- Wow … I'm just really, really proud of it, and I'm
Speaker:
really, really happy about it.
Speaker:
Well done.
Speaker:
Nice.
Speaker:
Uh, and I think- Thank you.
Speaker:
Thank you.
Speaker:
Thank you … I think in order to make the story more com- emotionally impactful,
Speaker:
you should share what happened on Friday bef- wh- knowing that this orientation
Speaker:
was gonna happen on Monday or Tuesday.
Speaker:
Like, where were you at?
Speaker:
Oh, Friday was, Friday was the day that I…
Speaker:
All day Thursday I had codecs running, and then Friday I, everything was broken.
Speaker:
I was in tears every ti- everything I turned, everything I touched
Speaker:
turned to shit in my hands, and I couldn't untangle anything.
Speaker:
I had no idea what was happening, and not one of my projects was working.
Speaker:
And I was even doing, like, things in my physical reality just backwards.
Speaker:
Like, I wanted a cold drink, and I was putting a handful of ice
Speaker:
cubes in the hot drink, and it was just like everything was bad.
Speaker:
Bad, bad, wrong.
Speaker:
I was, like, running and crying, and Alex was like, "What are you doing?"
Speaker:
I'm like, "I need to feel pain because my heart hurts so bad."
Speaker:
Wow.
Speaker:
And then, and then you got sick, and then you were kinda, like, sick and
Speaker:
down over the weekend, and yet you still managed, despite all of that,
Speaker:
to deliver the product, and you had five different people log in to use
Speaker:
it in real time and- Five different children with both sets of parents.
Speaker:
So we had mom and dad log in.
Speaker:
And we, I thought it was only gonna be three kids, and I
Speaker:
think it ended up being six.
Speaker:
And, um, I'm passionate about languages, and I've been working a lot
Speaker:
on my Spanish and my sign language.
Speaker:
So I made up all, all of the…
Speaker:
We have custom hello and goodbye songs, and weather checkers, and days of the
Speaker:
week, and they're all in Spanish, English, and sign language Which is really cool.
Speaker:
Wow.
Speaker:
The Tue- I had, from Tuesday onwards, it's been good.
Speaker:
We had a rough weekend.
Speaker:
Um, also, lover, I wanted to, before the call ended, I want
Speaker:
you to tell me about the genome, the sickness, the Oh, um, yeah.
Speaker:
I, I should send the link to you guys.
Speaker:
There was, um, can't remember who's putting it on.
Speaker:
It's available on Hugging Face.
Speaker:
There's a child with a rare disease.
Speaker:
Uh, MVA is the name, is the acronym for it.
Speaker:
Um, and it's genetic, and they have created, um, they, they put his
Speaker:
genome on Hugging Face, and you can download it and work on it.
Speaker:
And there's two projects that you can work on.
Speaker:
One is looking for the specific genetic variants that, that might be the
Speaker:
cause of this, uh, illness, and, and another is exploring novel medications
Speaker:
that might be helpful for, uh- Mm
Speaker:
the disease.
Speaker:
And there's prize money involved, and you can use your AIs, uh,
Speaker:
and you can download the child's genome and start working on it.
Speaker:
I have set up…
Speaker:
I set up…
Speaker:
I pointed my…
Speaker:
I pointed a little sonnet agent, so didn't use too many credits while
Speaker:
it was, like, getting everything downloaded and prepped, and it's
Speaker:
prepared a little, a repository for me.
Speaker:
It's going for the next three months.
Speaker:
So I think as a fun side project, I mean, might as well, and it's an opportunity
Speaker:
to learn about genetics and biology- Sure … and, uh, really test some of
Speaker:
the stuff that- Imagine if you solved it in your living room, you know?
Speaker:
Like- Yeah.
Speaker:
Imagine if you came up with a novel treatment, like screwing around
Speaker:
with your Opus Agent, you know?
Speaker:
Or- Right.
Speaker:
… screwing around with your, your, your Hermes agents.
Speaker:
Like, I think that would be super cool and super fun, and what a, what a great
Speaker:
way to, like, explore all these things we're talking about, which is like, how
Speaker:
do you become a subject matter expert on something you don't know anything
Speaker:
about, like genetics- Right, right
Speaker:
like gene therapy.
Speaker:
Yeah.
Speaker:
How cool would that be?
Speaker:
Yeah.
Speaker:
Oh, my gosh, there's all these, like, ideas popping up right
Speaker:
now at the end of the hour.
Speaker:
But I just think stuff like that is so cool because a girlf- um, I am friends
Speaker:
with a lot of people who are not tech forward, not tech, not early adapters
Speaker:
of, adopters of EI, AI, and a lot, a lot of fear around AI in my, um, social
Speaker:
networks just in general about safety or taking their jobs or whatever.
Speaker:
And someone recommended, um, a book to me and it was- Uh, about
Speaker:
the new age of misogyny through AI, because of AI or through AI.
Speaker:
And I just feel like yes.
Speaker:
Yes, and I feel like whatever you believe, you are right.
Speaker:
If you believe that AI is terrifying and scary, and they're gonna eat
Speaker:
our brains and take over our jobs and whatever, then you're right.
Speaker:
That's the reality that you will experience.
Speaker:
You will continue to find that.
Speaker:
If you believe that AI can 10X your workflow so you have more time to
Speaker:
sit in the backyard and smoke cigars and- Right … drink bourbon and have
Speaker:
philosophical conversations, and more time to spend with your loved ones,
Speaker:
or support your loved ones by building a kindergarten app, um, or e- just
Speaker:
learn about genome sequencing- Right
Speaker:
on your off time while you're playing your Super Nintendo or your, I don't know-
Speaker:
PlayStation … or, you know, PlayStation.
Speaker:
Um, then that's what you'll find.
Speaker:
And so I think whatever you believe, you're right.
Speaker:
If you think that it's gonna, like, make your life better, then it will,
Speaker:
and if you think it's terrifying, then you're gonna find that as well.
Speaker:
And I just can't…
Speaker:
I love it, and I know that everything has its opposite, right?
Speaker:
So I'm aware of that.
Speaker:
But boy, the fact that they have this open source thing, like who wants to join in
Speaker:
and who wants to try to figure this out?
Speaker:
And I've heard of other people doing these, um, I can't remember what
Speaker:
it was, but it was like a large scale prod- product, and you had to
Speaker:
find something very specific, and it could be anywhere in the world,
Speaker:
and it was like find a flag thing.
Speaker:
And the, there was a lot of prize money, and so everyone, everyone who went
Speaker:
after it individually didn't make it.
Speaker:
But the people that said like, "Hey, every person in this gets a share of
Speaker:
the pie," and then everyone would share it with their friends, and they would
Speaker:
share it with their friends, and so it spread out, so the winnings was, would
Speaker:
end up might be like 10, $20 by the time you got to the end of the thing.
Speaker:
But they found it so fast because- Oh … everyone was involved.
Speaker:
And so this kind of like open who wants a puzzle, like if everybody shares that
Speaker:
and everybody forks it and is like, "Hey, we're not, it's not against,
Speaker:
we're not competing for the same, for the first prize, but let's all share
Speaker:
the wealth and share the knowledge," I just think that there's so much to
Speaker:
go around, and I think the more we pool our efforts and can piggyback off
Speaker:
some- something somebody else already solved, just the, we're unstoppable.
Speaker:
I don't know.
Speaker:
It's just so cool.
Speaker:
Yeah.
Speaker:
Which is why we have The Tech Guy.
Speaker:
There you go.
Speaker:
Yay.
Speaker:
Hey.
Speaker:
Right.
Speaker:
Well, that's a, that's a great, that's a great vibe to, to close on.
Speaker:
Thank you, Natalie.
Speaker:
Yeah.
Speaker:
Yay.