TrackBack: Finding Out Which Vulnerabilities Actually Matter

Run a scan on a typical Python project and you might get 50 alerts. Fifty. Most developers look at that number and either ignore the whole thing or waste days investigating issues that were never reachable in the first place.

We thought there had to be a better way. The Problem: Too Many Alerts, Not Enough Signal Here's what happens in practice. Your project uses requests, which depends on urllib3.

A CVE gets published for a parser bug deep inside urllib3. Your scanner flags it. Now what?

You could: Drop everything and upgrade immediately (even though your code might never touch that parser) Read through the advisory, find the vulnerable function, trace through your codebase manually Ignore it and hope for the best None of these are great options. The first wastes time. The second wastes even more time.

The third is... well, let's not talk about the third one. What We Built TrackBack answers one question: can your code actually reach the vulnerable function? It gives you one of three verdicts: REACHABLE — your code calls it, here's the exact path NOT AFFECTED — no path exists, here's an OpenVEX statement you can use UNCERTAIN — something dynamic (like getattr with a runtime value) blocks the analysis That third category is important.

Most tools would either give you a false negative (claiming you're safe when you might not be) or a false positive (claiming everything is reachable). We decided to be honest instead. If we can't prove it either way, we tell you exactly what blocked us.

How It Actually Works The tool chains together a few ideas that individually aren't novel, but together solve the problem: Step 1: Find the vulnerable function Python security advisories rarely say "the bug is in urllib3.util.url.parse_url". They say things like "improper input validation in URL parsing" and link to a fix commit. We download that commit, look at what changed, and ask Qwen2.5-Coder (running locally via Ollama) to identify which functions are actually vulnerable.

The model reads the diff and the advisory text together. Every function name it returns gets checked against the real source code — no hallucinations make it through. Step 2: Build a call graph We parse your project and all its dependencies using Python's ast module.

This gives us a graph of who calls whom: imports, re-exports, from x import *, classes, inheritance, self, super(), all of it. We also track conditions. For example, yaml.load is only dangerous without a safe loader. yaml.safe_load calls load internally, but with SafeLoader, so it's not vulnerable.

The call graph knows this. Step 3: Search for a path Breadth-first search from your app's entry points to the vulnerable function. If we find a path, that's your proof — we print every step with file and line number.

If we don't find one, you're not affected. If we hit something dynamic (getattr, importlib.import_module, eval), we mark it uncertain. We also do a second pass looking for possible paths by following method names on unknown objects.

Those show up with ?-> on the guessed steps and get reported as uncertain, not reachable. What We Learned Be honest about what you can't prove. Early on, we were tempted to mark uncertain cases as "not affected" to keep the numbers clean.

That would have been a disaster. A wrong "safe" verdict is the worst mistake this tool can make. The model is a tool, not an oracle.

We use Qwen2.5-Coder to read diffs and identify functions, which it's good at. We don't use it for verdicts. Those come from static analysis every time, so the same input always gives the same answer.

Real projects do weird things. We found that Jinja2 compiles templates into Python code and runs it with exec. Our static analysis can't see calls from that generated code, so any project using Jinja2 gets marked uncertain for Jinja vulnerabilities.

That's the right answer. Dead code matters. If the only path to a vulnerable function goes through a function that nothing in your app calls or references, you're not affected.

The tool checks for this and tells you when a function is dead code. The Results We tested TrackBack on eight small example apps covering different cases: transitive dependencies, safe wrappers, dynamic imports, dead code, and more. 15 out of 15 verdicts matched what we expected.

Zero false "not affected" verdicts. The tool runs offline (it caches everything from OSV and PyPI), generates JSON reports, produces OpenVEX documents, and has a web interface that draws the call path step by step. What's Next We built this during Hacktoberfest 2026 Hack Day Sonipat for Problem 03.

It works. You can clone it, scan a project, and get real answers in seconds. There are limitations — we don't infer types, we don't analyze C extensions, and framework code that runs on its own (like a web server's request handler) isn't called from your app, so the verdict only covers your direct calls.

But for the core problem — "is this CVE something I need to fix right now?" — it gives you evidence instead of guesses. That feels like progress.