Lean And Mean AI (Long Version)
Thought On Scoping AI Applications
Imagery we choose often have poetic meaning behind.
And sometimes, there’s no point in analyzing hidden meanings because some metaphors are intended to be enjoyed at face-value with a story or two.
I’ll discuss how to go about defining AI’s scope and some realistic challenges that exist.
Flowers With No Scent
This is a true story from ancient Korea.
This is a story about a queen, not married to a king type of queen but a full monarch. Story is set in a time when she was a crown princess.
One day, she received gifts from diplomats of a nearby country. They gave her a painting, with flowers drawn on them that she have never seen before, and a pouch full of seeds with a note saying “See painting for when they bloom“. The flowers were very beautiful and everyone in the royal court knew the painting represented how great things would be once the crown princess sits on the throne. Such a thoughtful gift.
The crown princess, having never seen the type of flowers before, asked one of her gardeners about it. Instead of giving her a clear answer, the gardener gave a vague response about it being a peculiar choice.
When spring came, the flowers bloomed. And the crown princess went to the garden where the flowers were and decided to smell them. Weirdly, there was no scent. Feeling puzzled, she went to see the painting again. This time she noticed that there were no bees drawn on the painting, which seemed off-putting.
She probed her gardeners about it. One of whom, finally ended up telling her that the type of flower she received doesn’t have any scent. They also added on that sometimes people compare others with beauty but with nothing to give to the flower.
The crown princess was furious: It was clear that the diplomats were really just questioning her abilities. She felt foolish to be unaware of the true message for so long and vowed to be the best she could be. That would be the best form of vengeance.
Choosing Scope Of AI Is Intentional
There are many potential take aways in this story. But maybe there’s a plot twist. Perhaps the painter of the flower really loved the flower and didn’t care about what other people said. Maybe the diplomats lost the original painting and the seeds on their journey so had to replace them and hoped no one would ever notice. Maybe whoever sent the diplomats really just wanted to deliver a thoughtful advice - on what not to become - without being overreaching. So many untold stories and “what ifs”.
It matters what the crown princess experienced & how she felt.
History records it as an experience of shame, anger, and then resolve to be better. It almost doesn’t matter exactly what the untold stories of the painter or the diplomats are. It just matters how the flower was delivered and what types of things incurred as a result. The story is intended for us to make sense of the crown princess’s resolve to be a good ruler.
But we can’t control how people interpret things.
People tend to project. Whatever the crown princess was feeling - being the first female heir to the throne in history of her kingdom, others questioning her abilities, etc… - wasn’t new to her. So whatever gift she received, she would’ve seen it as an opportunity to pronounce how the rest of the world thinks of her or to grow in her abilities.
So, we could just focus on helping people get to their end goals.
This is why I compare AI to a flower - each has a meaning to them, it takes time and care to grow them, and the way we deliver flowers to others really matters1.
Or it might be that, when I go grocery shopping, I walk by this flower shop that’s always busy. And I notice people buying flowers of different sizes, types, and quantities depending on what they want.
Third explanation for the flower metaphor could be that it’s inspired from that time I sent a picture of a flower to a date and got dumped because she thought I was being facetious. Apparently I shouldn’t send pictures of flowers, I should actually bring one.
One of the 3 explanations for why I chose flower to represent AI is made up but it doesn’t really matter.
Because AI just a tool.
Flower can be a great tool to convey meaning to someone. And AI can be a great tool to convey information (and therefore meaning). What makes AI better than flower is that you can control how it conveys things.
Only thing to read into here is that each “flower” can have a very well-defined purpose and intentions. Choosing the scope of your AI application and is not much different: You want to be intentional about what it does, how it ingests and presents information, and for whom.
2 Ways To Scope
There are two ways to go about choosing what your AI application does.
Features to Data Requirements:
One is starting with specific features in mind and then narrowing down the data requirements for fine-tuning and context window.
For example, pretend I want to make an AI app that tells me all the facts about history of Paris in plain words that anyone can understand. This is such a broad topic that I might be out of luck without defining exactly what kind of history and what time periods. Instead, if I aim at urban legends in Paris in the years 1800-1900, that’s a very well defined time period where one could probably find some well-defined source of data somewhere.
Data To Experience:
Second way is to start with specific data sources you have access to - either because it’s your proprietary data or you just have access to it in easy and convenient ways - and then figure out what you can make with it.
Going back to urban legends in Paris in the 19th century example, imagine being a librarian of a history archive and you really want to figure out a way to preserve a piece of subculture and history. Perhaps you have this brilliant idea to turn your database into a storytelling machine. Then you realize that maybe you really want to target teenagers, because they’re the ones who will most likely to pick up a new trend and spread them quickly. And then the next is figuring out how to tell great stories.
Who Is Using What Tools To Do What Tasks, How, And Why?
Whatever path you choose to scope out your AI application, the goal is always the same. To figure out Who [Target Users] is using What Tools [Target Application] to do What Tasks [User Needs], How [User Interface], and Why [User Wants].
For example, consider again the case of getting teenagers excited about urban legends in Paris.
Sooner or later you might figure out that teenagers [Target Users] across any part of the world tend to spend lot of time on TikTok [Target Application] to consume information of any type [User Needs]. You also figure out that they are looking for new experiences and learn new things of any kind [User Wants].
Luckily it turns out that your archive isn’t just an encyclopedia of urban legends, but a collection of accounts of various writers telling the story in their own narrative voice.
You know you’re off to a great start. You know that you just need to start making contents that talk about Parisienne urban legends and put them on TikTok.
You’re just not sure what the videos need to look like - how long, tone, visuals, and more. If you decide that AI can be helpful here, figuring out the exact how is about how you build your AI application.
Iterating on AI Applications
Let’s keep working with the example of Parisienne urban legends. There are 3 steps to repeat to iterate on an AI application to make it better over time.
Feasibility Study: Do you have access to the right information?
Finding AI’s “Voice”: Does the AI talk in a way that is relevant and relatable?
Tying Evaluation Back To AI’s Voice And Data: What does your evaluation say about how well you are representing the “optimal voice” for your AI and what kind of information you maybe missing in your data?
Idea is pretty simple -
Make Sure To Have Enough Information
If your data doesn’t contain anything you can use, you can’t build any AI. In the case of Parisenne urban legends, good AI application could be coming up with scripts and visuals for TikTok videos. And in such a case, AI needs to see data that has information on how TikTok videos are usually scripted and how people have attempted to put visuals on short-form videos to deliver a message.
Data With Unique Voices Helps
If you just have data on how others have done it, your AI will just end up getting the same voice as all the other ones. If you’ve seen enough AI generated videos on YouTube that’s pretty much copy and paste of each other, you might know what I’m talking about. But luckily if you have access to an archive full of famous writers writing about urban legends in their own voice, maybe you can create an AI with a pretty convincing voice. Or if you are a good writer yourself or know enough good writers, maybe you and the other good writers can be your humans in the loop for reinforcement learning. There’s no correct answer here.
Good Evaluation Is Common Sense Evaluation
Evaluation in AI must be the most obvious yet the least understood thing about AI.
Well, how do you evaluate your car? If it gets there fast enough without killing anyone, maybe it’s car worth purchasing. If it has comfortable seats, a pretty big trunk, nice stain resistant leather, and comes with wheels that can withstand mud and ice, you got yourself a good outdoors vehicle.
In the case of AI talking about Parisienne Urban Legends, or really any AI that helps make contents, good ways to evaluate are in how engaging the viewers find it. Any comment by a real-person helps. Anyone real sharing it with others they know helps. But there’s also hidden aspects of how viewers find it that just aren’t captured.
It’s pretty straightforward to tell how a good evaluation impacts the AI’s Voice - if people like how they’re being talked to then something is going right. If not, then maybe it’s a good idea to change the voice. How evaluation impacts what data you should have is sort of the same, with 1 extra step.
And that step is just about determining what’s missing to give your AI the voice it deserves? Are you missing seriously good writers that can write scripts for you? Are you just missing what words and plots that teenagers react to in stories? Answering these questions depends on what you’re shooting for, of course, but the principles are the same.
Challenges Need Changes
Lot of content platforms track pretty much everything about you so they can get around the issue of not having enough context about you. But as an AI developers who works outside of platforms, what can actually be done? Well the least you can do is to be ready to face the strange yourself.
Getting Personal Without Asking Personal Questions
Most important thing is always going back to what users want. So we just have to ask questions about why they do something.
I can’t really give you a formula on how to ask “why”. During my time at Nasdaq, we just talked directly to customers because they shared the problems we were solving. But if you are developing something for a large number of people, many you will never meet or ever talk to, what do you do?
Best I can say here is that when people feel safe and comfortable, they want to say stuff to you. Figuring out how to do this is the power of evaluations.
Who’s Even Talking Here?
To be fair, I’m trying to figure out how to get my readers to talk to me. Well I can’t write better stuff if they don’t give feedback.
All of them keep reading and keep ignoring my threads and emails. Only 1 response to my survey so far and 1 reader who reached out to me to give feedback.
If people don’t start giving feedback, that means I’d have to start posting bunch of senseless polls on my articles. Who knows, things can keep getting darker and darker to a point where I end up just selling information like everyone else. I might not even know the difference between art and just information.
Maybe it’s about time I think about that list of RAG stack that developers would definitely love to use.
Anyways, can people help here?
Again, there’s no hidden meaning here.