Primary Photo for {0} {1}

The Dependency Graph Your Tests Can't See

Presentation byKayodeA Adeniyi

A model I was working with scored R squared 0.998 on public US data. One of the input columns had been calculated from the target by the same agency that published both, so the model was rediscovering a formula it had already been handed. Tests don't catch this. Nothing throws, nothing fails, the numbers look great. So I built a linter for it. You write a manifest of what derives from what, the tool walks the graph, and CI goes red when a feature sits on a derivation path to or from your target. It returns four verdicts, including one for columns whose provenance nobody has traced yet. The code is Python. The idea applies anywhere you have a dependency graph and a CI job. I'll show the leak, the manifest, the linter, and a run going red live.
Repo: https://github.com/Adeniyikayodee/derives-from-tutorial

Presented with these Guilds
Cover Photo for JavaScript Monthly London Meetup
Primary Photo for JavaScript Monthly London Meetup

JavaScript Monthly London Meetup

Javascript evolution has sped up (a lot) in recent years and event the most veterans developers find it hard to keep up with the latest trends. This meetup group aims to bring you monthly bite-sized updates on the world of Javascript along with a healthy dose of nice people, beer and pizza.

Please use your full name when registering, as some of our venues require a full list of attendees beforehand. You have an idea and you want to be a speaker?

We are always looking for more speakers - submit your talk here (https://docs.google.com/forms/d/e/1FAIpQLSdFaatfveOUbrmer47jYb5J4J4ttxAFc1CgTjUDltBXmDOJmg/viewform)

1.4KMembers
Similar Presentations
Primary Photo for {0} {1}

The Dependency Graph Your Tests Can't See

Presentation byKayodeA Adeniyi

A model I was working with scored R squared 0.998 on public US data. One of the input columns had been calculated from the target by the same agency that published both, so the model was rediscovering a formula it had already been handed. Tests don't catch this. Nothing throws, nothing fails, the numbers look great. So I built a linter for it. You write a manifest of what derives from what, the tool walks the graph, and CI goes red when a feature sits on a derivation path to or from your target. It returns four verdicts, including one for columns whose provenance nobody has traced yet. The code is Python. The idea applies anywhere you have a dependency graph and a CI job. I'll show the leak, the manifest, the linter, and a run going red live.
Repo: https://github.com/Adeniyikayodee/derives-from-tutorial

Presented with these Guilds
Cover Photo for JavaScript Monthly London Meetup
Primary Photo for JavaScript Monthly London Meetup

JavaScript Monthly London Meetup

Javascript evolution has sped up (a lot) in recent years and event the most veterans developers find it hard to keep up with the latest trends. This meetup group aims to bring you monthly bite-sized updates on the world of Javascript along with a healthy dose of nice people, beer and pizza.

Please use your full name when registering, as some of our venues require a full list of attendees beforehand. You have an idea and you want to be a speaker?

We are always looking for more speakers - submit your talk here (https://docs.google.com/forms/d/e/1FAIpQLSdFaatfveOUbrmer47jYb5J4J4ttxAFc1CgTjUDltBXmDOJmg/viewform)

1.4KMembers
Similar Presentations