This is a submission for the Agent.ai Challenge: Full-Stack Agent (See Details)
What I Built
In building agents I came across an annoying problem. LLM's are often terrible when it comes to the concept of time. They're often bad at:
- Sorting items by date
- Consistently understanding relative words like Yesterday, Tomorrow, Next week, etc.
- Understanding the concept of what is in the past and what is in the future - date and time wise, it's not going to tell your fortune.
Some LLM's behind the scenes are provided the current date and time in a system prompt. That's great, that doesn't always apply to every LLM though. The understanding of relationships is also interesting and I think the problem is that "Tomorrow" "yesterday" are variables that humans understand as being relative, but would appear in written language of training data that is already "old" or in the past from the current time and date.
Additionally LLM's just like how they're not natively good at math, have a hard time with dates and times because probably because they're basic math. It's likely partially an issue of tokens can be broken up multiple different ways for a date/time and the LLM isn't seeing the actual words and numbers, it's seeing basically id numbers that represent these concepts.
So the strategy is that I take away some of the "math" and give the LLM a little extra understanding of time so that it can work with time the way humans do.
Demo
This agent is fully designed to be used by your agent. All you need to do to use it within your agent builder, click add action, go to "advanced", choose invoke agent. Enter datetime. 1 optional parameter will appear for setting a time_zone (defaults to UTC if time zone is not provided).
You can go directly to the agent and click the dropdown to see the available time zones, but basically it's all standard time zones. For example for New York you could enter:
America/New York
Set the output variable to something like current_date_and_time, then when you use an LLM action, you can insert that variable somewhere in your prompt.
That's it! Now ask it to do something it could only do if it understood dates and times and it should perform better.
In order to make trying it easier I built a demo agent that does the above
Ask it questions that would require understanding of date and time such as reordering tasks, use relative words, try asking it what month it will be x months from now, etc, regardless of the LLM model you use it will perform better than without the agent.
Check out the actual current Date and Time agent
Why does this meet the criteria for full stack
This is a utility for other agents to use. Behind the scenes what this agent does is hits a serverless function to grab the current date and time and provide just enough information to better enable LLM's to properly handle dates and time.
I added this agent entry in-case the fact that it's an agent invoked by my Agent Inspector Agent - would disqualify the Agent Inspector from the Full Stack competition.
Agent.ai Experience
I've enjoyed building on Agent AI, it's a cool tool that is among the pioneers of supporting agents working together.
Top comments (2)
Hi Jon,
It was nice to learn about such an agent being created to cater to this niche use case. However, I would love to hear more about the real-world applications of this model and how we can use it. Also, do other LLMs struggle with solving the same issue?
By the way,
I have created an agent as well and would love to get your feedback on it.
Blog Post: dev.to/sattavatakshat/social-genie...
Agent Link: agent.ai/agent/Social-Genie-Ai-Soc...
I've found that all of the LLM models currently supported by Agent AI experience issues with handling dates and times. Open source models not backed by Google/Meta seem especially prone to issues.
I suspect this may get fixed over time at the API level for LLMs but in the mean time using this agent improves date and time handling.