Human vs. LLM Responses: What Every AI App Builder Should Know
Humans respond to questions from other humans very differently than how large language models (LLMs) like ChatGPT respond to humans.
Most of us know this instinctively.
But if you’re building AI or LLM applications that are expected to meet the needs of humans in complex, demanding situations - and you want your users to trust and rely on those applications - it’s critical that you understand the differences.
Based on our experience working with three of the world’s largest AI application builders over the past decade, here are six ways LLM responses differ from human responses:
01
Simplistic responses
Unlike LLMs, humans excel at adapting their communication style to their audience. Imagine an experienced physicist asking a complex physics question about gravity. An LLM might offer a basic textbook answer, while a fellow human physicist could engage in a nuanced discussion exploring advanced concepts. Similarly, a seasoned Java programmer wouldn't appreciate a beginner-level response to a question about polymorphism from an LLM, because a fellow programming expert would never do that.
02
Remembering the past
Human memory thrives on long-term information integration and synthesis. We weave past conversations into current ones, creating richer responses. LLMs, however, often struggle with short-term recall, limiting their ability to build on previous exchanges. Imagine discussing a movie – a human can connect the dots between multiple scenes or cross-reference earlier scenes with ease, while an LLM might struggle to remember anything discussed more than a couple of questions ago.
03
Emotional intelligence
Humans are masters of reading between the lines. When interacting with a fellow human being, most of us intuitively grasp the intent, skill level, and even emotional state based on subtle cues like tone and volume. LLMs that are not explicitly trained using datasets enriched with context, tone, and other parameters will most likely miss these nuances, leading to potentially frustrating interactions.
04
Establishing empathy
Empathy is a cornerstone of human interaction. We can sense and respond to another's emotions, offering support or humor as needed. LLMs, unless, once again, meticulously trained and fine-tuned, lack this ability. Imagine confiding in someone about a bad day – a human listener might offer solace or practice active listening, while an LLM might respond with a ‘flat’ sentence. While these situations can provide fodder for movies about robots, if you’re an AI/LLM application builder, this can erode your user’s willingness to engage with your application.
05
Continuous learning
Unlike the static knowledge base of an LLM, humans are continuous learning machines. Many of us treat every interaction as an opportunity to learn and expand our understanding. This allows us to draw on past interactions and accumulated knowledge to make each new interaction more meaningful. LLMs can struggle here without a very deliberate approach to continuous training and fine-tuning.
06
Navigating biases
LLMs trained on unfiltered data can inherit harmful biases, leading to unfair or offensive responses. Humans, while not immune to bias, can actively challenge these preconceptions and strive for fairness. Imagine asking about career advice – an LLM might perpetuate unconscious biases, while a human can offer guidance that’s solely based on objective criteria and the other person’s aspirations.
So what does all of this mean for an AI/LLM application builder?
No one expects LLMs to function - yet - like a human.
However, taking into account the specific use case and target audience, AI/LLM application builders can mitigate some of these limitations. A robust training and fine-tuning strategy that uses domain-specific datasets created, vetted, and evaluated by humans, we have found, can be particularly effective.
