NannyML

OSS Python Library for Detecting Silent ML Model Failure

5.0•9 reviews•

78 followers

OSS Python Library for Detecting Silent ML Model Failure

5.0•9 reviews•

78 followers

Visit website

NannyML estimates real-world model performance (without access to targets) and alerts you when and why it changed. The performance estimation algorithm, confidence-based performance estimation (CBPE), was researched by core contributors.

This is the 2nd launch from NannyML. View more

NannyML Regression v0.8.0

OSS Python library for detecting silent ML model failure

Detecting silent model failure. NannyML estimates performance for regression and classification models using tabular data. Alerts you when and why it changed. It is the only open-source library capable of fully capturing the impact of data drift on performance

Free

Launch tags:

Open Source•Artificial Intelligence•GitHub

Launch Team

pymaster210

NannyML

Maker

Hey, it’s Wiljan from NannyML. Indeed, a lot has happened since our last launch here. I have been mostly talking to data scientists for the past few months. So we can understand their pains and needs better, allowing us to build a better product for them. We have also been talking to business users, model owners, and other stakeholders. AI models are almost never isolated services and most of the time part of a bigger business process. One of the questions I often get is why we need to monitor AI models after they have been deployed. I thought I share my answer here: AI models are trained using historical data. Yet, they are tasked to make predictions on data of the present. Today's data might be very different from yesterday's data. The world is constantly changing and so does the data it generates. Assumptions and patterns that were true during training won’t hold forever after deployment. AI models are bound to deteriorate. It’s a natural process and a matter of when not if!

Report

2yr ago

Hakim Elakhrass

NannyML

Hunter

Hey Product Hunt! Hakim here from NannyML 👋 Today we are launching NannyML v0.8.0. In the past 6 Months we have added: - Regression Support (with Direct Loss Estimation) - Command Line Interface - Database exports and cloud integration - Official docker container - A ton of new univariate data drift tests (Jensen-Shannon Distance and L-infinity distance) - A bunch more improvements All of these things bring you closer to having a complete view of your ML performance in production, catching failures, and resolving them as fast as possible. Most notable is regression support. NannyML now allows you to detect silent model failure on all ML models with tabular data that are classification or regression, integrated anywhere, with any stack. After the launch of NannyML Open Source 6 months ago here on product hunt, we have put our heads down and kept iterating on the library with your input. 8 major releases have happened since, alongside 15 minor releases. Your input has been vital to this velocity and has allowed us to continuing to build a library loved by 1000’s of data scientists worldwide. I hope you like these new features and I am looking forward to hearing all of your feedback! - H

Report

2yr ago

Niels Nuyttens

NannyML

Maker

Hey, Niels the only engineer here. It’s been amazing to see how much our team has achieved since our last launch. We launched mission-critical features such as support for regression and expanded the range of drift detection methods. We turned a library into a CLI and a container, exporting our metrics into databases and dashboards, pushing forward to give NannyML a well-deserved place within the fast-moving MLOps landscape. We’ve completely overhauled and reinvented ourselves to make NannyML as easy to use as we can. I’m already looking forward to tackling the next set of challenges together with this group of exceptional people. Give NannyML a try, I hope you love it as much as I do.

Report

2yr ago