We had more than a hundred open vulnerability alerts. Five needed fixing that day, and nothing told anyone which five.
A new vulnerability is published somewhere almost every hour. Most of them will never matter to you. A few will matter a lot, and quickly. The hard part is not finding vulnerable dependencies. Tools have done that for years. The hard part is noticing, in time, the one that needs you today.
We build and run our own products, and we advise other businesses on how to run theirs. So this week we did to ourselves what we would do for a client: we asked how long it would take for a newly published, actively exploited flaw in one of our dependencies to reach a person who could fix it. The honest answer was: we did not know, and possibly never.
1. What we already had, and what was missing
GitHub’s Dependabot reads every lockfile in a repository (npm, pip, Docker, GitHub Actions) and compares it against the GitHub Advisory Database. When an advisory is published that matches a package you use, it opens an alert, usually within minutes. Our Dependabot pull requests that upgrade a package are merged automatically into our integration branch.
That sounds complete. Three things were not:
- Coverage. Dependabot alerts are a per-repository switch. In most of our repositories it was off, including nearly all the Python services. Every open alert we had was for JavaScript. That was not because our Python was clean. Nothing was checking it. Turning it on for every repository took one loop over the GitHub API.
- Judgement. Of the alerts that were open, the large majority were in build and test tools: bundlers, test runners, dev servers. They are real, and they are worth fixing, but not at 11 p.m.
- Someone being told. Dependabot notifies a repository’s watchers, by default. In a company with dozens of repositories, that is nobody in particular.
2. Severity alone is the wrong filter
The obvious fix is to page on anything rated critical. We tried that on paper against our own list, and it fails both ways. It would have paged us for a critical advisory in a test runner that never ships to a customer. And it would have stayed silent on a medium advisory that attackers are actively using.
Two public signals do a much better job.
CISA’s Known Exploited Vulnerabilities (KEV) catalogue. The US Cybersecurity and Infrastructure Security Agency publishes a list of vulnerabilities with evidence of exploitation in the wild. It is free, machine-readable JSON, and it changes a few times a week. If a CVE you depend on is in it, someone is using it now.
EPSS, the Exploit Prediction Scoring System. Maintained by FIRST, it gives each CVE a probability of being exploited in the next 30 days, recalculated daily. GitHub already attaches it to the advisory, so you do not need another API.
3. The rules we page on
We split every open alert by one more fact Dependabot gives us: whether the package is a runtime dependency (it ships to production) or a development one (it runs in CI and on laptops).
| Runtime | Development | |
|---|---|---|
| In CISA KEV | Text now | Text now: CI runs beside deploy rights |
| Critical | Text now | On the board |
| High, EPSS of 10% or more | Text now | — |
| High | On the board | — |
| Anything else, EPSS of 10% or more | On the board | — |
| Everything else | Counted only | Counted only |
“On the board” means it appears on our operations console’s Alerts page, with a link, but nobody’s phone rings. “Counted only” means the automatic upgrade will handle it, and the console shows a single line saying how many there are.
The texts and the page use one alert per repository and advisory, however many lockfiles carry the package. The text says what to do, not just what is wrong: “Critical, in production code · upgrade to 3.4.17 · open today”. Once a critical alert is older than 24 hours, or a high one older than seven days, it also says “past its fix window”. Those are the fix times in our security policy, so the alert holds us to our own promise.
4. Where it runs: not a new system
The first design that comes to mind is a new scheduled function, a new database table, and a new notification topic. We did not build that, because we already had all three.
Our operations console has an alert sweep: a single AWS Lambda function that runs every minute. It reads every source that can need a person (CloudWatch alarms, failed deployments, data-erasure deadlines, unusual client activity, changes to an administrator’s sign-in, customer feedback). It decides what is an alert, remembers when each one started, and texts and emails whoever subscribed. That code has already been through several rounds of hard review. A blip must not page twice, a message must not be lost when the function dies halfway, and a source that cannot be read must not look like all clear.
Vulnerabilities became its seventh source. That is a reader and a set of rules, not a system:
- Every minute, it asks GitHub for the organisation’s open Dependabot alerts. Each page carries an ETag, so a minute with no new advisory gets a 304 Not Modified, which costs nothing against the rate limit.
- Once a day, it reads the KEV catalogue and keeps only the CVE identifiers, a few kilobytes. If CISA’s site is down, it keeps yesterday’s list and says on the page that it is judging on severity alone. It never goes quiet because a feed was unavailable.
- When GitHub closes an alert, because the upgrade reached the main branch or someone dismissed it with a reason, the console clears it two minutes later and tells whoever it told when it started.
The result: a newly published advisory that matches a runtime dependency on our main branch reaches a phone about a minute after GitHub opens the alert. If an alert we are sitting on turns up in the exploited catalogue, it reaches a phone within a day.
5. Two small traps worth knowing
A new alert source must not be silent for existing subscribers. Our subscriptions store the list of sources each person ticked. Add a source, and every subscription saved before it simply does not include it, so nobody hears about the new thing you built. We now record which sources a person was offered when they saved. A source added later is on for them until they say otherwise. While fixing this, we found the previous new source had the same gap.
“Could not read” is not “nothing found”. If the GitHub app has not been given permission to read Dependabot alerts, the API answers 403. The console says Not connected in words, and keeps whatever it judged before. A security check that fails quietly is worse than none, because it looks like a clean bill of health.
A checklist for your own estate
- Turn Dependabot alerts on everywhere, and for new repositories by default. Check: in a large organisation, it is almost certainly off somewhere.
- Separate runtime from development dependencies before you decide what wakes anyone up.
- Page on exploitation, not on severity. CISA KEV and EPSS are free and public.
- Put the fix in the alert. “Upgrade to 3.4.17” gets done. “CVE-2026-XXXXX, CVSS 9.2” gets a meeting.
- Let automation handle the rest. Automatic upgrades for everything that does not page are what make paging on very little safe.
- Treat a blind check as an alert of its own.
If you would like us to look at how your team would find out about the next exploited dependency, get in touch.