Everyone is throwing the term AI around over the last year, even a hole in the wall restaurant owner is telling his potential investors, AI is running his kitchen!
I am sure you all must be tired of hearing AI everywhere you go or see. I am slammed with AI advertisement wherever I looked daily, but I would be lying if I say I am not interested. Of course, I am! However, as with any technology, I needed a use case for getting into AI. I am using the term AI in a very broad sense as in “Mode of Transportation” versus aircraft, ship down to a skateboard. All can be considered mode of transportation; a mean to get your from point A to point B. What exactly AI encompass is a topic in itself for another day.
Actually, my involvement with AI goes back a long way, as in decades ago. In my senior year in college, back in the 80’s, Artificial Intelligence was the rage in the Computer Science domain. But then AI is all about leveraging Lisp to process huge amount of textual data. There were no GPU, LLM, agentic Ai etc. Eventually, AI faded from my memory as life priorities consumed most of my focus, until the last couple of years when AI busted onto the scene like the best thing since sliced bread.
Over the past year, AI technology started seeping into my daily life. Work place started pushing AI tools, such as Copilot, AI training., and internal websites interaction are being serviced by AI agents. I started using Copilot instead of google for doing my search. Yeah, don’t laugh, that’s not what AI. But it is a start. What I noticed right off the bat was, with google, you ask for a piece of toast, google spite out “here are all the things that is related to toast”, copilot, “How do you want your bread toasted? Do you want white, wheat bread? Do you want a thicker or thinner slice?”. Based on your response, Copilot will tell you how it is going to make your toast and also may recommend what spread goes well with it. Your conversation with Copilot can take off into another direction.
My initial impression was, google search is like a vending machine, you make selection and it gives you what you asked for. Copilot is like ordering through a human order taker. The downside to using Copilot is, if you are not specific in your question, you will get unexpected or unwanted result.
Here’s an example of taking Copilot response as the absolute bible. I was investigating a performance issue within an application my team support, going through the logs and working with the vendor technical support team. This process usually takes a couple of days or more. These logs are huge, thousands of lines. A SME on the line of business side, started chatting me on the side pasting snippets of output from Copilot. He pointed to the 1.2 millions instances of the same error in the log file. Since I was working closing with tech support and we are narrowing our focus on a potential root cause, I decided to cut out the noises. But I did bring this error he highlighted to the support engineer. The engineer deem that as a non issue and common appearance in the log, or rather a false flag. Out of courtesy, I responded later that, the error is a false flag.
Unfortunately, he didn’t leave it at that, and send an email to management with his analysis highlighting the 1.2 million occurrences of that error, that needed to be investigated regardless. I asked where did the 1.2 million number comes from. He replied that, copilot provided that by scanning the log files. As mentioned, log files usually contained 100s to thousand of lines and all logs roll over daily. So having 1.2 million errors in a day is not normal. I check the latest logs and the error he highlighted appears a few times but as the vendor support engineer mentioned, this error is relevant only to an older version of the software.
Of course his email caught his management attention and escalated because his argument was 1.2 millions errors is not something we should ignore. I had to stop our troubleshooting and craft an email response with evidence from the vendor that the error was a artifact from an old version that is still live in the current application. I asked where is he is getting that high of a number. His response was he asked Copilot to scan for all instances of the error. Then it dawn me that copilot was scanning all the historical logs in the folder. So I had to explain to him that the 1.2 million is not the actual number but copilot is treating duplicates as separate incidents.
The old adage – “Garbage in Garbage Out” is still valid. Using AI requires a basic understanding of knowing what you are asking and not taking the output blindly. AI is there to do the heavy lifting in terms of processing Terabytes of information in seconds which human can’t accomplish. But as of today, AI is not able to mimic human emotion, rational reasoning. We need to responsibly review the result for AI before making a decision based on it. Remember asking the right question is key! The result is only as good as the question being asked! If you just asked for a car, don’t expect to get a ferrari or even just an ordinary car that works!
