통과율만 보면 안 보이는 실패: YouTube 광고팀의 프로덕션 Eval 운영법
프롬프트를 잘 쓰면 에이전트가 말을 듣는다는 기대는 데모까지만 유효하다. 프로덕션에 올리면 같은 프롬프트, 같은 입력에도 실행마다 결과가 달라진다. 어제 통과한 케이스가 오늘 실패하고, 어제 실패한 케이스가 오늘 통과한다. 이 상태에서 “출시해도 되는가"를 판단하려면 감이 아니라 측정 장치가 필요하다.
더 보기태그
- A2a
- Act
- Agent
- Agent-Debate
- Agent-Finance
- Agent-Harness
- Agent-Skills
- Agentic-Payments
- Ai-Agent
- Ai-Architecture
- Ai-Blockchain
- Akash
- Apify
- Base
- Benchmark
- Bitcoin
- Bittensor
- Capital-Markets
- Ccip
- Cctp
- Chainlink
- Circle
- Coding
- Coding-Agent
- Coinbase
- Compliance
- Context-Engineering
- Cross-Chain
- Cryptography
- Decentralized-Compute
- Depin
- Dex
- Diffusion-Policy
- Distillation
- Eip
- Eip-3009
- Ensemble
- Erc-8001
- Erc-8004
- Erc-8041
- Erc-8126
- Erc-8183
- Erc-8196
- Erc-8226
- Ethereum
- Evals
- Evaluation
- Fhe
- Fido2
- Fintech
- Goldman-Sachs
- Google-Cloud
- Gpt-5-6
- Groth16
- Harness
- Heloc
- Hermes-Agent
- Identity
- Imitation-Learning
- Knowledge-Management
- Kya
- Langfuse
- Langgraph
- Lerobot
- Llama-Cpp
- Llm
- Llm-Agents
- Llm-as-Judge
- Llm-Observability
- Local-Llm
- Maci
- Math
- Mcp
- Memory
- Metadata
- Mev
- Moa
- Model-Routing
- Moe
- Mpp
- Mtp
- Multi-Agent
- Observability
- Okf
- Onchain
- Open-Source
- Open-Standard
- Opik
- Orderbook
- Passkey
- Physical-Ai
- Post-Quantum-Cryptography
- Post-Training
- Privacy
- Prompt-Engineering
- Provenance
- Python
- Quant-Finance
- Quantum-Computing
- Qwen
- Rag
- Reasoning
- Reinforcement-Learning
- Render
- Reputation
- Retrieval
- Reverse-Kl
- Rl
- Robot-Learning
- Rwa
- Search-Agent
- Security
- Self-Improvement
- Semaphore
- Skills
- Small-Language-Model
- Smart-Contract
- Smolvla
- Snark
- Solidity
- Sp1
- Stablecoin
- Swe-Bench
- Tokenization
- Tracing
- Trading
- Ucp
- Uncensored
- Usdc
- Vla
- Voting
- Wallet
- Webauthn
- X402
- Ylds
- Zero-Knowledge
- Zk-Proof
- Zk-Rollup