I often feel that the LLM-based AI I use is getting close to what I need, but there is something missing. While sometimes the AI surprises, often the response seems too generic, lacks oomph, or is just not good enough. Ideally, we’d like our AI assistant to fully appreciate our context and needs and deliver a spot-on output that we can use. This is roughly alluded to as the “alignment problem.” However, if we consider AI (LLMs) as a tool, its alignment partly depends on the limitations of the tool itself and partly on the user – who should be aware of the tool’s affordances (what it can and cannot do) and have the capability to engage and exploit these affordances. Using a knife to open a screw might work but….
Looking at the tool itself, at the cost of oversimplification, if we consider the AI as a simple system I-P-O model, there are three aspects of the LLM that can improve the quality of the output (O), i.e., improve alignment. First, the input (I) is about the quality of the prompt…with a larger context window (input size) leading to better output. On the (P) side we have the data and the model. The quality and size of the training data is important. More specialized data will give better outputs within the specialized context, while broader training data might increase versatility, but also increase hallucination in areas where data is sparse. Further, the model itself has relationships between data — and the size of the model (more relationships) results in more refined output. So, based on these three aspects, every instantiation of AI might have some inherent limitations.
Looking at the user side, a prompt can give an adequate output on an LLM, but either refining the prompt (prompt engineering) or continuing the conversation with the LLM can obtain more customized output…better aligned with user needs. However, achieving such alignment takes resources either in the form of cognition, time, or training. So, the return on this investment is going to demarcate users. What is the extent of alignment/investment…. will some users accept a mediocre or generic response (which reduces productivity but increases efficiency), or will they work towards alignment (increasing productivity, but reducing efficiency)?
The path of least resistance is to accept the output of an LLM as “adequate” or believe that it is “the best it can do.” If the output is believed to be adequate, then we are accepting some level of mediocrity. Perhaps mediocrity is an acceptable level of compromise in achieving efficiency (i.e., I’ll accept the draft email that the LLM spit out…it’s good enough, and I can move to other, more important things) over trying for better alignment. The second (“it’s the best it can do”) may be true for the reasons laid out earlier on the limitations of LLMs. But it casts aspersions on the ability of AI to deliver an aligned product. If the output is perceived as the best possible, then there is no incentive to cognitively invest in conversation or better prompts to improve the output.
