I still remember the Slack message that arrived at 2:47 AM last March. A machine learning engineer at a healthcare AI startup, someone I’d interviewed six months prior about their ambitious diagnostic model, was having what could only be described as an existential crisis.
“Our fraud detection model just started flagging every transaction from zip codes beginning with ‘9’ as high-risk,” he wrote. “We can’t figure out why. It wasn’t doing this yesterday. We’ve rolled back twice. Same behavior. We think someone poisoned our training pipeline but we have no audit trail. No signatures. Nothing. We don’t even know when the data changed.”
Read More from DZone.com Feed