It wasn’t long after the release of ChatGPT when many of us began to think through how this new technology would change our lives. Since then there has been this idea floating around that we’ll all soon have a bunch of assistants at our beck and call. I’ve used this ‘gaggle of assistants’ analogy myself when talking to people about how our jobs and lives will change.
Recently I’m leaning more towards another analogy: we’re all going to become product managers.
I like thinking about it this way because putting yourself in the mindset of being a good PdM helps you get the most out of an LLM. The required skills have a lot in common:
- you need to know what you can outsource (to your team or LLM) vs what you should do yourself
- you need to understand what it is that you want your team / LLM to do for you
- you have to clearly articulate the requirements for the job
- you have to determine whether your team / LLM has succeeded
- you have to work iteratively with your team / LLM to give corrective guidance and make adjustments
- you should recognize when your team / LLM has a better idea than you do
The examples below are quite imperative, but I fully expect to be more declarative as time goes on and these systems become better.
Examples
In the context of pulling together this seeking-manifold site, I needed to convert the format of links in a markdown file. It was a lot of links, more than I wanted to do modify manually, and they followed a clear pattern.
As long as I have the ability to specify the requirements for this task, a capable LLM can spare me the effort of creating a script to pull it off. Here’s the problem I posed to my LLM:
Please write a python script to open a markdown file, test.md, and make substitutions of the following form:
Replace every line that has the following format:
with a line that has this format:
![[filename.png|alt text]]
Here are three examples:
Replace this:
with this:
![[clip-guided-diffusion.png|CLIP Guided Diffusion]]
<etc.>
Do not modify any other text.
In this case, the script offered up did the job perfectly.
Here is another example, where I wanted to see what in my Obsidian files I’d been working on as a function of time (based on dates in headers):
Please write a python function to take a directory as input and do the following:
1. Recursively find all markdown (`.md`) files within the directory
2. Within each markdown file, find all lines beginning with `## YYYY-MM-DD`, where YYYY-MM-DD is a date in that exact format. The line may have other text following the date, which can be ignored.
3. Create a single list of tuples with (YYYY-MM-DD, `filename`), where filename is the name of the file in which the matching line occurred
4. Sort the list of tuples by date (ascending)
5. Return the list
In this case I had to have a bit of back and forth with the LLM before I got things the way I wanted, but you get the idea. If you have the basic skills of defining the requirements for what you want, you will be well-equipped to work with an LLM.
(Of course I don’t need to tell you to be careful running code generated by an LLM; look it over closely first!)
2024-08-05: Here’s a post by Nicholas Carlini in a similar vein but with a ton of examples of how he uses LLMs for programming assistance.
2024-08-22 update: Here’s a relevant and interesting piece.