Eye On A.I.

• #298 Ryan Kolln: How Appen Trains the World's Most Powerful AI Models

  This episode is sponsored by AGNTCY. Unlock agents at scale with an open Internet of Agents.  Visit https://agntcy.org/ and add your support. How do the world's most powerful AI models get trained and trusted at scale, and what does that really take from data to deployment? In this episode, Appen CEO Ryan Kolln joins Eye on AI to unpack how rigorous human evaluation, culturally aware data, and model-based judges come together to raise real-world performance. In this episode of Eye on AI, host Craig Smith speaks with Ryan Kolln, CEO of Appen, about building evaluation systems that go beyond static benchmarks to measure usefulness, safety, and reliability in production. They explore how human raters and AI evaluators work in tandem, why localization matters across regions and domains, and how quality controls keep feedback signals trustworthy for training and post-training. Ryan explains how evaluation feeds reinforcement strategies, where rubric-driven human judgments inform reward models, and how enterprises can stand up secure workflows for sensitive use cases. He also discusses emerging needs around sovereign models, domain-specific testing, and the shift from general chat to agentic workflows that operate inside real business systems. Learn how leading teams design human-in-the-loop evaluation, when to route judgments from models back to expert reviewers, how to capture cultural nuance without losing universal guardrails, and how to build an evaluation stack that scales from early prototypes to production AI. Stay Updated: Craig Smith on X: https://x.com/craigss  Eye on A.I. on X: https://x.com/EyeOn_AI 

  --------  

  51:01

  --------

  51:01
• #298 Sunita Sarawagi: How AI Can Learn the World and Still Follow Logic

  This episode is sponsored by AGNTCY. Unlock agents at scale with an open Internet of Agents.  Visit https://agntcy.org/ and add your support. Why should AI that learns from the messy real world still obey strict logic, and what does it take to make that reliability hold up in production? In this episode of Eye on AI, host Craig Smith sits down with Sunita Sarawagi to unpack how large scale learning can be combined with explicit rules and constraints so models stay trustworthy. We cover when world ingestion fails without structure, how to encode domain logic alongside LLMs, and which hybrid or neurosymbolic approaches reduce hallucinations while preserving flexibility. You will hear how to design a reliability stack for real users, detect out of distribution inputs, and choose evaluation signals that reflect outcomes rather than accuracy alone. Learn how product teams layer formal logic on top of generative models, decide what to hard code versus learn from data, and enforce business policies across agents, tools, and knowledge graphs. You will also hear how to run safe experiments, track prompt and model changes, prevent regressions before they reach customers, and plan for compute and infrastructure at scale with metrics like completion rate, CSAT, retention, and cost per resolution.   ​​Stay Updated: Craig Smith on X: https://x.com/craigss  Eye on A.I. on X: https://x.com/EyeOn_AI

  --------  

  24:16

  --------

  24:16
• #297 Jeff Lunsford: How Agentic AI Will Redefine Every Digital Interaction

  Why will agentic AI redefine every digital interaction, and what foundation do enterprises need to make it safe, trusted, and real time? In this episode of Eye on AI, host Craig Smith sits down with Jeff Lunsford to unpack how a neutral customer data platform like Tealium becomes the control plane for agentic systems. We cover how to collect and unify first party data responsibly, enforce consent and identity across channels, and feed the right context to models so agents can act with confidence in the moment. You will hear how real time profiles, event streams, and deterministic identity power personalization, automation, and transactions across web, mobile, ads, email, and customer support. Learn how leading enterprises are preparing for agentic commerce that could double digital interactions, why governance and privacy must be embedded into delivery teams, and which standards enable safe transactions and payments with agents. You will also hear how to build an "agentic front door" for your business, design guardrails and spending allowances, choose where to run reasoning and inference, and measure impact with metrics like conversion rate, ROAS, CSAT, and cost per resolution. ​​Stay Updated: Craig Smith on X: https://x.com/craigss  Eye on A.I. on X: https://x.com/EyeOn_AI 

  --------  

  47:55

  --------

  47:55
• #296 Yeop Lee: How Coxwave is Redefining AI Evaluation

  This episode is sponsored by AGNTCY. Unlock agents at scale with an open Internet of Agents.  Visit https://agntcy.org/ and add your support. How is Coxwave Redefining AI Evaluation? In this episode of Eye on AI, host Craig Smith is joined by Yeop Lee, Head of Product at Coxwave. Together they explore how teams move beyond accuracy-only metrics to outcome focused evaluation with Coxwave's Align. We look at how Align measures satisfaction, trust, and task completion across chat, email, and voice, how LLM as judge pairs with human review, and how product teams search conversations to find hidden failure patterns that block adoption. Learn how leading companies design an evaluation stack that guides prompts, agents, and UX, which pitfalls to avoid when shipping updates, and which metrics matter most for success, including completion rate, CSAT, retention, and cost per resolution. You will also hear how to run experiment tracking with model and prompt change logs, set up governance that prevents regressions, and choose between SaaS and on premise deployments that meet security and compliance needs. Stay Updated: Craig Smith on X: https://x.com/craigss Eye on A.I. on X: https://x.com/EyeOn_AI

  --------  

  43:04

  --------

  43:04
• #295 Fergal Reid: Why Your Bots Fail and How Agents Fix Your Customer Support

  This episode is sponsored by AGNTCY. Unlock agents at scale with an open Internet of Agents.  Visit https://agntcy.org/ and add your support. Why do so many chatbots fail in the real world, and how can AI agents actually fix customer support? In this episode of Eye on AI, host Craig Smith explores how teams move beyond scripted bots to production-grade AI agents that resolve real issues across chat, email, and voice. We look at what makes agents reliable at scale, how to configure them safely, and how to manage them like digital workers alongside your human team. Learn how leading companies approach agent onboarding and governance, which pitfalls to avoid, and which metrics matter most for success, including resolution rate, CSAT, and cost per resolution. You will also hear how to enable actions like refunds and returns through secure procedures, design human handoff that customers appreciate, and build an omnichannel rollout plan that scales responsibly. ​​Stay Updated: Craig Smith on X:https://x.com/craigss Eye on A.I. on X: https://x.com/EyeOn_AI

  --------  

  43:49

  --------

  43:49

Weitere Technologie Podcasts

Trending Technologie Podcasts

Über Eye On A.I.