Thinking like a human: agentic workflow in action
Imagine you're cooking a dish with many ingredients and steps, but you have to follow the recipe from start to finish without tasting it or making any adjustments along the way. Sound risky, right? Well, that's pretty much how we use LLMs these days.
We give them a prompt and they generate the full response, one word at a time, without going back to revise anything. It's called "zero-shot" prompt. You'd think the end result would be a disaster, but LLMs actually do a shockingly good job, whipping up high-quality output even with this challenging approach. It's pretty amazing how capable they are, considering the constraints!
In real life we go through a recipe multiple times, making improvements at each stage. For example, we might follow a steps like:
Plan out the overall menu and decide which dishes to include.
Figure out if any additional ingredients need to be purchased or if any prep work can be done in advance.
Cook a first version of the dish, following the recipe closely.
Taste the first attempt to identify any flavors that are too strong, too weak, or not quite harmonious.
Adjust the seasoning, cooking time, or techniques based on the feedback from the taste test.
And so on.
This iterative process is essential for most chefs to create a delicious meal. So how can we help LLMs to do better job than just zero-shot prompt and whipping out higher-quality output.
How
To do that, first we need to understand how our human brain thinks. Our brain operates using two types of thinking: System 1 and System 2.
System 1 thinking:
Operates very "fast"
Functions "subconsciously"
Is "automatic" most of the time
Handles everyday decisions like brushing teeth or taking a shower
Can be error-prone (like forgetting to comb your hair—I don’t have that problem anymore 😉)
System 2 thinking:
Feels slower.
Involves conscious deliberation.
Requires effort, akin to climbing a mountain on some days.
Is suited for complex decisions, such as job changes or selecting a college.
Tends to be reliable, as decisions are thoughtful and not made in haste.
Daniel Kahneman explores these concepts in his book, Thinking, Fast and Slow, which portrays similar ideas about System 1 and System 2 thinking.
What if we can apply similar System 2 thinking and iterative approach like what Chef applies in creating the best recipe on how we work with AI? Andrew Ng describes this as an "agent workflow" where system goes through various stages, such as Reflection, Tool use, Planning and Collaboration. There are other researchers are calling out similar step. Here is my point of view of what that agent framework would look like. As depicted here, the user interact with the system which has agent workflow built in where these agents are using not only LLM but also applies system 2 thinking, create a plan for it, applies step by step process for it and eventually orchestrate the entire process to generate a result which is better output than simply derived from LLM by asking a question.
With an example:
Let's consider an example. You are a hiring manager and looking to fill up a role of software engineer for your company. Since it's been a while since you have hired a software engineer, you will need to create a job description to give it out to your HR team. As you are using LLMs, like chatGPT, Claude in your day-to-day work, you thought OK, let me ask one of this LLM to do it for me. Not a bad idea, tbh.
In above image, I have created a job description for software engineer to be hired at Cerebras Systems. Those who want to see as a flat file, here is the GitHub gist.
In reality, you will not want to use above job description as-is and update it to reflect your team, the role, the company and you may ask your HR partner to help you out with editing some of the content to support what's trending in the industry, market and for that particular role.
What if we can do exactly that using some of the different tools instead of you and your HR partner modifies it? Let's get agents to do that work for us!
Okay, what do I mean by agents here? Its bunch of software which is going to do either sequential work or hierarchical steps on behalf of us and integrate it with the same LLM you used earlier to create the raw job description.
To create the job description, we are going to apply at least 3 steps (or 3 agents), which are done not only sequentially but also in an iterative manner.
We will need to do some research and analysis.
Based on the research and analysis, we will need to write the job description
Once it's created, we need to edit to make it look better and succinct.
Search and Analysis: This could be a task where the software will need to search the internet for company's culture, values, mission, also look for what kind of skills required in 2024 to be a software engineer, what are the industry trends, challenges and opportunities exist in this role and what kind of qualifications required for it.
Write: This is an iterative process in conjunction with "search and analysis agent" will provide various outputs from it's search result and the writing agent will be applying it's core skill "writing"! How it writes? Of course using LLM.
Edit: Once everything is written up, the "editor agent" comes and edit for clarity, make it more engaging, align with company vision and do the content refinement such as grammar issues, etc. How it edit? Of course using LLM.
At the end, finally we will have a job description which is customized from the original job description created by LLM to hopefully better version of it.
You can also see above output as GitHub gist.
Details:
What you see above is a small application I have created to apply agentic workflow what we reviewed above - where this application takes an input such as, company description, company domain, hiring needs (software engineer), and additional benefits. From there, it launches various agents, such as searching the internet, collating the information from internet, writing the job description and finally editing to make it better. To keep it simple, I have use the same example of hiring software engineer for Cerebras Systems.
To build this application, I have used crewAI agent framework and the code for you to run it yourself in your local environment is available on GitHub (https://github.com/keyom-ai/agentic/tree/main/job-description).
Did it really work?
Those who are wondering how good job this agentic workflow did. Here are some comparisons:
As you can see from above picture, except few lines (in blue represents no major change), most of the output of the chatGPT+agents looks better or concise. It removed the "fluff" what was created originally by chatGPT plus also enhanced the output to attract the talents to consider Cerebras Systems as a company.
Conclusion:
These are very early days of agent based workflow to solve certain problems. We have recently seen where team Devin created an entire AI Engineer workflows to augment software engineering work. Similar work are in progress and it's an ongoing research topic in the AI community. We are continue to see major progress happening in this area throughout the year and I am optimistic that by using some of these agentic workflow, we will be able to improve our life from mundane tasks which are going to be handled by AI.
What do you think?
p.s. those who prefers the video version of this article, you can watch it on YouTube as well.