The "Prompt" is no longer enough: welcome to the era of orchestrated AI!
Imagine the scene: a company invests tens of thousands of euros in a revolutionary AI agent designed to automate customer support. Two weeks after deployment, they completely backpedal: the agent hallucinates, misses key context, and annoys users. This is the daily reality for 95% of companies that stumble with AI and get zero ROI. This is what researchers call the GenAI Divide.
To cross this barrier, our teams chose to treat AI not as a magical tool, but as a genuine system engineering subject.
In 24 months, Malt has deployed and stabilized more than 80 assistants and workflows in production. At the latest AI Malt Academy, Léo Martin (AI & Automation Engineer at Malt) shared our internal methodology. His golden rule for moving from a simple experimental "POC" to industrial efficiency? "Test, Train & Throw."
The mirage of immediate ROI: the LinkedIn illusion
What you see on social media: An autonomous agent that navigates company tools perfectly and delivers flawless answers on the first try.
The reality on the ground experienced at Malt:
- 6 months of manual copy-pasting from Slack before coding a single line of automation.
- Massive rollbacks because a feature was automated too quickly.
- Hundreds of iterations on prompts and formatting.
Léo Martin shares an essential golden rule: "Test, Train & Throw." At Malt, 40% of AI initiatives are voluntarily stopped along the way. Let’s be honest: in advanced engineering, knowing when to abandon a path that does not create value is not a failure—it is a proof of maturity. An AI project that never evolves or encounters no obstacles often hides window-dressing innovation. To build a solid infrastructure, you must accept testing, learning, and sometimes pivoting.
The vision: The two-headed organization (sommeliers vs. chefs)
To cross the GenAI Divide, Malt structured its teams into two distinct expert profiles, supported by our cross-functional teams:
- The AI Champions ("the sommeliers"): Present at a rate of 1 to 2 people per team, they know the "wine list" (available AI tools), guide their direct colleagues toward the right solution, and evangelize usage daily.
- The AI Builders ("the chefs"): At a rate of 1 to 3 experts per major department (Marketing, HR, Sales...), they are the ones who "cook the dishes." They combine data, APIs, and models, and design architectures from end to end.
- The Central Pillar: This entire network of Champions and Builders is actively supported and framed by cross-functional AI, Tech & IT teams, which guarantee the security, governance, and robustness of the global infrastructure.
The methodological trick: the "sandwich" 🥪
Léo Martin points out that to successfully deploy a workflow in a company, technical talent alone is not enough. You need to bring together 3 key profiles from the very first week:
Between exploding demand and operational reality
This transition toward industrialization does not come out of nowhere. Data from the latest Malt Tech Trends shows an unprecedented acceleration: demand for the creation of AI agents has surged, while workflow automation projects on tools like n8n have experienced spectacular growth of +1,390%.
Yet, the market faces a double challenge: search volume is exploding, but team trust still needs to be built. A complementary study highlights a major paradox: 90% of employees admit to using ChatGPT on their personal accounts, while only 55% trust the internal tools provided by their company.
It is to resolve this friction and capture this "Shadow AI" that Malt's robustness framework was designed.
The methodology: Malt's robustness framework
To transform the chaos of raw data into business value, Léo uses a pragmatic approach divided into four technical pillars.
1. The imperative of the non-scalable: the mandatory manual phase
Before automation comes a fine understanding of the business need. For example, for Malt's Customer Care department, teams started with 6 months of 100% manual processing.
Expert tip: Do not try to code immediately. Use this manual phase to identify real bottlenecks and map out edge cases. If the initial data is of poor quality (Garbage in, garbage out), the agent will only amplify the mess.
2. Architectural clarity: assistant, workflow, or agent?
Ne demandez pas un "Agent" quand un simple script séquentiel déterministe suffit. Léo recommande de catégoriser précisément le besoin:
3. Native distribution: go where your users are
The classic mistake is forcing employees to open a new tab or yet another web interface. Your main competitor in a company is not OpenAI; it is the reflex to open chatgpt.com on the side. The solution? Push AI directly into the existing workflow: a Slack bot (like Malty AI at Malt), a Chrome extension, or a native integration in Salesforce or Notion.
4. Recursive observability: monitor to avoid flying blind
You cannot improve what you cannot see. From day one of production, the architecture must integrate a strict logging system (like Langfuse) to allow Product Managers to modify prompts and datasets without depending on developers. Evaluation must be hybrid: combining classic Machine Learning metrics and an LLM-as-a-Judge validation layer for the semantic aspect.
6 Lessons learned in "The Hard Way"
The experience gained by Malt across its 80+ production workflows can be summarized in 6 major key learnings:
The results: what is the return on investment?
Industrializing AI represents a strategic investment, but contrary to popular belief, the cost is not where you expect it. At Malt, the budget breakdown highlights a clear reality: the vast majority of the investment is human (dedicated to the time of Builders and the Core Team), while the purely technological line (Gemini API, Claude, platforms like n8n or Dust) represents only a minor fraction of the overall budget.
The return on investment was achieved in 12 to 18 months, with major gains equivalent to 12.5 Full-Time Equivalents (FTE):
- Customer Care = automated pre-drafting of support tickets. Precious minutes saved per ticket and a drastic drop in handling time.
- Ticket Routing = automated sorting and semantic orientation of support requests. The vast majority of incoming flows are sorted 100% autonomously to the right contact person.
- Finance = automated analysis and validation of complex financing requests. Nearly a quarter of files qualified and processed end-to-end without friction.
- Sales = automated aggregation of Salesforce data for pre-sales. Significant time saved for each sales representative before customer meetings.
AI is no longer an R&D topic, it is infrastructure
A company's competitive advantage no longer lies in the choice of its language model, which has become a commodity accessible to all. Value is found in the intelligence of the orchestration, data governance, and the ability to support humans.
For freelancers and companies alike, boundaries are blurring: the roles of Growth Marketer, Product Manager, and engineer are merging into a pivotal profile: the AI and automation engineer. Today, this is the most strategic hire for scaling organizations.
Want a replay? This AI Malt Academy awaits you
Want to dive into the code, discover behind-the-scenes details of our internal workflows, and avoid production deployment pitfalls?
👉 Watch the full replay of Léo Martin's Malt Academy on our YouTube channel.
About Léo, AI & Automation Engineer at Malt
Léo Martin is an AI & Automation Engineer at Malt. He designs and deploys multi-agent architectures, automation infrastructures, and LLM Ops pipelines that run the company daily. His expertise helps bridge the gap to industrialization, turning technological adoption into a sustainable engine for ROI.
Need to automate your business processes? Consultez les profils de nos experts IA sur Malt.