To LLM, or Not to LLM: How Designers and Developers Navigate LLMs as Tools or Teammates
In the rapidly evolving landscape of technology, large language models (LLMs) are becoming integral components of design and development workflows. However, the decision-making process regarding their implementation is not simply a matter of technical capability; it involves nuanced considerations of roles, responsibilities, and organizational accountability.
Research Overview
A recent study published on arXiv (arXiv:2604.15344v1) employed a constructivist grounded theory approach, interviewing 33 designers and developers from three prominent technology organizations. The findings reveal that professionals do not evaluate LLMs solely based on their capabilities but rather assess the role these models might play within their existing workflows.
Framing LLMs: Tools vs. Teammates
The study highlights two primary framings through which designers and developers perceive LLMs: as tools or as teammates. Each framing carries distinct implications for decision-making and accountability.
- LLMs as Tools:
- When viewed as tools under human control, LLMs are generally accepted in workflows.
- This framing allows for seamless integration within existing governance structures.
- Participants felt that the clear delineation of control mitigated concerns about accountability.
- LLMs as Teammates:
- The perception of LLMs as teammates introduces ambiguity in agency and responsibility.
- Practitioners expressed caution, especially when outcomes were not clearly attributable to human or AI actions.
- Concerns were raised about how shared responsibility could impact organizational accountability.
Collaborative Configurations
Despite the challenges posed by teammate framing, some participants described scenarios where LLMs effectively contributed to collaborative reasoning. In these instances, LLMs were positioned within explicit oversight structures, enabling productive interactions while maintaining clarity in accountability.
Implications for Design-Time Reasoning
The findings of this study suggest a shift in how practitioners should approach the integration of LLMs into their workflows. Rather than viewing the decision as a binary choice—whether or not to use LLMs—the focus should be on sociotechnical positioning during the design phase. This perspective emphasizes the need to consider how roles are framed and how those framings impact decision authority, accountability ownership, oversight strategies, and organizational acceptability.
Conclusion
As organizations increasingly embrace LLMs, understanding the complexities of their integration becomes paramount. By framing the question of LLM usage as a sociotechnical challenge rather than a straightforward technical decision, designers and developers can better navigate the evolving landscape of AI tools and their implications for human work.