Across clients and programs, I find myself repeating a mantra: “Make it, make it right, then make it right and fast.”
Data-driven projects (AI, Data Science, Reporting, Automation) have inherent uncertainty which can lead to unanticipated failure modes. Whether that is an ASI becoming SkyNet (as AI doomerists claim) or a report just including a bug, uncertainty is what the experienced Data & AI leader grapples with.
I’m interested to hear how you deal with runtime uncertainty. My approach follows a bit of DMBOK, a bit of MLOps: (1) tollgate development to mitigate risk (sometimes the data isn’t up to the task for the AI or ML need) (2) be realistic and clear with what an ML model can handle — all models are wrong but some are useful! (3) encode ML model performance or build requirements as unit/integration tests (4) build with observability at the forefront