GitHub `pushed_at` Misled on 14 of 35 Scraping Repositories; Bots Were Confirmed on Three

GitHub's repository page exposes pushed_at, a timestamp that can move when any branch receives a push. That can diverge from the newest commit on the default branch. The default-branch date is a repository-maintenance signal; it is not necessarily the code installed by a package manager.

Package managers normally resolve registry artifacts or module versions. That is why this audit checks repository activity and the published artifact separately: either can be current while the other is stale.

So I pulled 35 repos that still show up in recommendations, and read the number GitHub doesn't put in the header: the date of the newest commit on the default branch.

The discrepancy is real: 14 of the 35 have a pushed_at running more than 180 days ahead of the latest default-branch commit, topping out at 1,802 days. Bots are positively confirmed in three of those 14; after archived and human-activity filters, two confirmed bot cases remain among nine candidates. The more useful finding is that repository and published-artifact recency can diverge.

What was measured, and on what

Official reference: GitHub repository API.

Every figure here was read from a live API response between 15:44 and 15:53 UTC on 2026-07-27 and cached. The 35-row dataset, repository list, built rows, and fetch/build scripts in artifacts/ preserve the audit inputs and transformation code.

Four data categories were collected; registry and activity requests were conditional rather than one uniform four-call sequence:

• GET /repos/{owner}/{repo} — stars, archived, pushed_at, license, default_branch.
• GET /repos/{o}/{r}/commits?sha={default_branch}&per_page=1 — the newest default-branch commit, used as a repository-maintenance signal.
• GET /repos/{o}/{r}/activity?per_page=30 — for every repo where those two disagree, what actually moved pushed_at.
• GET /repos/{o}/{r}/releases plus the PyPI and npm registries — when the artifact last shipped, which turns out to matter more than either.

staleness_days is the elapsed time from the default-branch commit to the as-of moment. The gap between pushed_at and that commit is the illusion, measured in days. Anything over 180 days gets flagged.

Two disciplines are worth stating because they changed the results.

Package attribution was verified, never inferred. A package whose README mentions a repo is not that repo's package. Every mapping had to be confirmed against a structured field — the registry's own repository/project_urls entry, or a manifest committed inside the repo. Six plausible-looking mappings failed that test and their download counts are deliberately not attributed.

One of those rejections is worth the whole rule. curl-cffi pulls 35,763,529 downloads a month and looks, from a distance, like the Python binding for lwthiker/curl-impersonate — a repo cold for 875 days. Attributing it would have produced a number forty-four times larger than newspaper3k's, and it would have been false: curl-cffi's own PyPI metadata points at lexiforest/curl_cffi, a separate, actively maintained project that last shipped 2026-04-03. The most spectacular number available here was the wrong one.

A seventh case is stranger: steel-dev/steel-mcp-server declares @steel-dev/mcp-server in its own package.json, but npm returns 404. It was never published, so "still being installed" cannot be true of it at all.

Where a number couldn't be obtained, it says so. Go, JVM, .NET and PHP tools have no PyPI or npm presence, so they carry N/A (no PyPI/npm package) — never zero. Seventeen of the 35 publish no GitHub Releases whatsoever; that's recorded as none, not as missing data.

These fields are deliberately not collapsed into a single “health score.” A stale default branch, a recent side branch, a missing GitHub Release, and an old registry artifact answer different questions. The row-level evidence is available in the 35-row dataset, with the repository inputs and built records alongside it. Read them as triage signals that determine the next check, not as four votes on whether a project is alive.

The sampling caveat, up front

This is a hand-built list of tools I suspected were coasting on reputation. It is not a random sample of the scraping ecosystem, and "31 of 35 are stale" is not an ecosystem rate — it's close to a measure of how well I picked. The interesting result isn't the stale count. It's that even on a sample selected for the phenomenon, the specific mechanism I was testing explained a minority of cases, and could be positively confirmed in fewer still.

The illusion is real, and this is its worst case

sjdirect/abot, a .NET crawler with 2,308 stars. GitHub reports a push on 2026-07-17, ten days before the as-of date. The default branch was last touched 2021-08-09.

That's a 1,802-day gap. Five years. The header says last week.

Fourteen of the 35 repos show a gap over 180 days:

Fourteen out of thirty-five. Real, worth knowing, and a minority of a sample chosen to contain it.

Bots are confirmed on three flagged repositories—and two remain after filtering

The folk version of this story always names dependabot. I checked, by pulling the activity feed for each flagged repo and classifying every ref pushed after the last default-branch commit. That "after" matters: events predating the last commit say nothing about what inflated the gap, and counting the whole feed instead quietly changes the question.

Confirmed bot-driven, on the post-commit method: three. scrapinghub/splash (4 of 4 post-commit events on dependabot/pip/*), geziyor/geziyor (5 of 5 on dependabot/go_modules/*), apache/any23 (16 of 16 on dependabot/maven/*). One of those three, any23, is formally archived, so it never enters the filtered set — leaving two confirmed cases that also satisfy every other condition.

Outright wrong: two, and both are more interesting than the bot story.

dragnet-org/dragnet's 1,520-day gap comes from a human pushing a branch called mp/py3.10 — an unmerged Python 3.10 port. Somebody tried to bring it forward and stopped. That's not automated noise inflating a timestamp; that's a visible, dated record of a rescue attempt that failed. Arguably it's the most useful signal in the whole dataset, and the "dependabot did it" framing would have erased it.

Mixed, and bigger than either: two. crawlab-team/crawlab has 12,250 stars — the second-most-starred repo in the sample — and a 488-day gap on main. Its feed carries dependabot branches and 24 post-commit pushes from humans, all to develop and test. A user reading the header sees February 2026 and assumes health; a user reading main sees October 2024 and assumes death. Both are wrong. Development moved off the default branch, which is a thing projects do and GitHub's summary view has no way to express. sjdirect/abot is the other mixed case: the push that set its headline pushed_at really was dependabot, but a human pushed upgrade1 in 2024, which is why it drops out of the filtered set later.

Rhizome-Conifer/conifer is the ambiguous one, and I initially read it wrong. Its default branch is main, not master, and main has been still since 2023-10-12 — the only main event in the feed is a branch creation in January 2025, consistent with a rename. Meanwhile a single account pushed to conifer-twilight and twilight/read-only on 2026-07-22, five days before the as-of date. That's genuine human activity, but "actively developed" is more than the refs support: one contributor, on branches named read-only, is at least as consistent with a managed sunset as with ongoing development. What can be said is narrower and still worth saying: the 1,013-day gap is not bot noise, and it is not evidence of abandonment either.

Unknown: seven. php-html-parser, simhash-py, internetarchive/wayback, boilerpipe, waybackurls, wpull and crawler4j all have a verified gap and an activity feed that comes back empty.

The tempting explanation is retention — GitHub's activity feed doesn't reach back forever. The cache refutes that for most of them. The oldest event anywhere in these 123 responses is 2023-03-10, and five of the seven have a pushed_at comfortably inside that window: php-html-parser 2024-08-09, wpull 2024-04-29, waybackurls 2024-05-01, internetarchive/wayback 2024-03-01, simhash-py 2023-05-15. Whatever moved those timestamps should have been in the feed and wasn't. Retention accounts for only boilerpipe (2018) and crawler4j (2021).

So the honest statement is thinner than a tidy explanation: for seven repos the gap is a verified fact and its cause is not established — the endpoint returned nothing, and for five of them I can't tell you why. "Dependabot did it" is an assumption for all seven.

Tallied up, across the 14 flagged repos:

The rows sum to 14. They classify what moved pushed_at; they do not independently establish whether a project is abandoned.

What survives the full filter, and what "survives" means

The original claim needs four things at once: stale over a year, pushed_at inflated past 180 days, not archived, and no sign the inflation came from human work. Nine of 35 candidates clear all four, listed here beside the two exclusions that matter most:

That number needs a qualifier the filter can't carry. Only two of the nine — splash and geziyor — have positive evidence that bots drove the inflation. The other seven clear the fourth condition by having no evidence in either direction. They are cases the claim survives, not cases that confirm it. And the single strongest confirmation in the whole audit, any23 at 16 of 16 dependabot events, is excluded because the repo is archived.

Note also that abot, the 1,802-day case, is not among the nine. Its record contains a human push, so it fails the fourth condition — the most dramatic illusion in the dataset is not a clean instance of the mechanism it illustrates.

For 21 of 35, GitHub reported the staleness plainly

Here's the finding that did the most damage to my thesis. Fourteen repos have a gap of exactly zero, and seven more sit under 180 days. For 21 of 35 candidates, pushed_at is the last default-branch commit. GitHub is not obscuring anything.

Including some of the deadest things in the sample:

BuilderIO/gpt-crawler has 22,374 stars and its header has said the same thing since 2025-07-07. Nothing is concealed, and install volume continues.

That forces the claim down to something narrower: GitHub obscures staleness in a minority of cases, and in the majority it states it plainly while installs continue anyway. Why they continue is outside what this data can answer — the numbers here count installs, not decisions. But no UI change addresses the second group, which is the larger one.

Repository activity and published artifacts can diverge

The single most striking row in the dataset breaks the framing entirely.

codelucas/newspaper — 15,126 stars — is active. Its last default-branch commit is dated 2026-07-21, against an as-of date of 2026-07-27, and it was authored by the maintainer. Every repo-level check passes.

The package everyone installs is newspaper3k 0.2.8, published 2018-09-28. That's 2,858 days old, and it pulls 813,513 downloads a month.

The default branch is current, while the PyPI artifact has not shipped since 2018. This proves a release gap, not why the package has not shipped or whether a pipeline is broken. It is the kind of risk a repository-only maintenance check would miss, because the registry artifact is what normally runs after pip install newspaper3k.

Once you look at packages instead of repos, the pattern is everywhere. Across the 17 packages whose repo attribution was verified, 16 last shipped over a year ago, and those 16 account for roughly 2.28 million installs a month out of 2.30 million total:

tls-client deserves its own line: 790,305 installs a month from a repo cold for 905 days, in a category where staying current is the entire job. Browser TLS behaviour changes; a library that stopped tracking it in early 2024 is running on early-2024 assumptions.

Two cautions on this table. Registry download counts include CI runs and mirrors and de-duplicate nothing, so they measure install volume, not humans. And the rolling windows don't share an end date — npm's close 2026-07-24, pypistats' are relative to fetch time — so the total is a sum of slightly offset months and deserves to be read as "about 2.28 million", not to the digit.

The name collision worth knowing about

internetarchive/wayback is the dead Java OpenWayback, cold for 1,916 days. wayback on PyPI is a different project entirely — edgi-govdata-archiving/wayback — and it is healthy, having shipped 0.5.1 on 2026-06-19, five weeks before the as-of date. Same name, opposite condition, no relationship. This was one of the six rejected attributions, and it's the one most likely to bite a real user: searching the name gets you both, and nothing on either page says which one you found.

Archived, deprecated, and still installed 127,232 times a month

Six repos in the sample carry archived: true, which GitHub renders as a full-width banner. My first read was that this proves people ignore loud warnings. The cache says the story is worse than that.

Official reference: npm download-count API documentation.

Official reference: npm's deprecation documentation.

@modelcontextprotocol/server-puppeteer pulls 127,232 installs a month out of modelcontextprotocol/servers-archived. But its npm repository field is null — there is no link from the package page back to the repo, so the banner isn't something an installer skipped past. Most people fetching this package never had a path to it.

What npm does publish is the deprecation. The package's latest version carries deprecated: "Package no longer supported. Contact Support at https://www.npmjs.com/support for more info." — which npm prints to the terminal on install. So the warning is delivered, in the place the user actually is, and 127,232 installs a month proceed regardless. That's a stronger finding than the banner one, and it points somewhere different: the signal is not missing, it's arriving inside a wall of install output that nothing forces anyone to read.

browserbase/mcp-server-browserbase shows what a clean shutdown looks like: its final default-branch commit, on 2026-07-20, is literally "Mark repository as archived and unmaintained (#198)". The maintainers announced it, dated it, and flagged it in the API. Its 20,389 installs are worth reading carefully, though — the npm window runs 2026-06-25 to 2026-07-24, so 26 of those 30 days precede the archive commit. That figure is mostly pre-announcement demand, not defiance of it. What happens next is genuinely unknown from this snapshot, and I'd want a second reading a month out before claiming anything.

57 stars, 204,025 installs a month

Janpot/microdata-node has 57 stars and pulls 204,025 downloads a month from a release dated 2020-05-11.

At 57 stars, microdata-node has little repository visibility relative to its registry volume. The 3,579-to-1 install-to-star ratio is consistent with transitive use, CI repetition, mirrors, or direct machine consumption. This audit fetched no dependency graphs and cannot choose among those explanations.

The row is a prompt to inventory indirect exposure, but proving transitive use requires reverse-dependency or lockfile evidence that this audit did not collect.

Stale is not the same as broken

An honest audit has to say this: none of this measures whether anything is broken. It measures whether anyone is home.

Some of these are simply finished. SetSimilaritySearch implements set-similarity algorithms; those don't rot. simhash is a 2007 paper. boilerpipe's content-extraction algorithm behaves the same in 2026 as in 2015 — whatever its accuracy against modern pages, the code hasn't drifted out from under you.

What rots is anything with a moving target on the other end:

• Browser automation — each Chrome release can break it.
• HTTP client behaviour that mirrors real browsers — browsers change, and a frozen library stops matching; tls-client sits here.
• Site-specific parsers and per-site extraction rules — every site redesign is a bug.
• Anything wrapping a third-party API — the vendor changes the schema and you find out in production.
• Anything wrapping an LLM — model deprecations move faster than any of this.

So "1,606 days stale" is a five-alarm fire for one category and close to irrelevant for a hashing utility. No breakage was tested here and no claim is made about it; sorting your own dependencies by which category they fall into costs nothing and does more for you than the staleness number alone.

The four checks that actually answer the question

None of them is the repo header.

Run checks 1, 2, and 3 as fast triage. Then escalate when repository metadata is absent, package mapping is ambiguous, activity moved to a non-default branch, or the registry artifact diverges from the repository. Unauthenticated GitHub limits and registry latency make a one-second promise inappropriate.

If a check turns up something cold in a category that moves, the maintained alternatives in this space are documented in our own testing: Trafilatura for the content extraction that dragnet and boilerpipe used to do, Scrapy or Crawlee for crawling frameworks, Crawl4AI and Firecrawl for LLM-oriented extraction, and Scrapling where resilience matters. Our open-source scraper pillar tracks the wider set. Those are first-party reviews; run the four checks yourself before trusting any recommendation, including ours.

Limits of this data

• Sample selection. Hand-picked for suspected abandonment. No ecosystem rates can be read off it.
• No breakage testing. Not one of these 35 tools was run against a live site. Staleness is a maintenance signal, not a functional verdict.
• Download counts include machines. CI, mirrors, no de-duplication, and windows that don't share an end date. Install volume, not users, and not decisions.
• Seven unknown causes stay unknown. The activity feed returned nothing, and for five of the seven retention doesn't explain it. Leaving the cell empty beats filling it with the popular guess.
• staleness_days uses committer dates. Rewritten or backdated history would distort it. None was detected, which is not the same as none existing.
• One as-of date. 2026-07-27. Several of these repos will have moved by the time you read this — newspaper in particular commits regularly. Re-run the four checks; don't cite my dates.

Where a managed service changes the shape of this

Every check here exists because with a self-hosted library, you own the staleness. If a frozen HTTP client stops behaving like a current browser, that's your incident, at whatever hour it surfaces.

Author note: Thunderbit is our managed scraping product. A managed service transfers some maintenance responsibility to a vendor, but coverage, response time, lock-in, and vendor continuity become part of the risk model. Thunderbit was not evaluated in this repository audit.

The honest trade: you give up the ability to read the source, pin a version, and fix it yourself at 2 a.m. For a team already maintaining scrapers, the open-source route is often the right call — the four checks are how you make that a decision instead of an assumption.

Try Thunderbit for Web Data Extraction

The short version

I built a list to prove that GitHub hides abandonment. The illusion is real for 14 of 35 repos and spectacular in one case: sjdirect/abot shows a push last week against a default branch frozen since 2021, a 1,802-day gap.

But the mechanism is narrower than the story. Nine repos clear every condition the claim requires, and only two of those nine — splash and geziyor — have positive evidence that bots did the inflating; the rest clear the bar by an absence of evidence. Two flagged repos are humans trying and failing to revive a project. One, crawlab, has 12,250 stars and simply moved development to develop. For seven, the activity feed came back empty and retention doesn't explain five of those, so the cause is unestablished rather than assumed. And for 21 of 35 repos, GitHub reported staleness accurately.

The worst case in the dataset passes every repo-level check. codelucas/newspaper was committed on 2026-07-21; newspaper3k, last shipped 2018-09-28, went out 813,513 times that month. Across the sample, 16 packages with releases over a year old account for roughly 2.28 million installs a month.

Check the default-branch commit, registry release date, archived flag, and npm deprecation field as initial triage. Resolve package-to-repository attribution and non-default development before drawing a conclusion.

Try Thunderbit for Web Data Extraction Get Started Free

FAQs

What is pushed_at and why doesn't it mean "last updated"? pushed_at is the GitHub API field behind the repo page's activity timestamp, and it refreshes when anything is pushed to any branch. The newest default-branch commit is one repository-maintenance signal, while package managers normally install registry artifacts or resolved module versions. In this audit, 14 of 35 repositories showed the two GitHub dates more than 180 days apart.

Is it always dependabot inflating that timestamp? No, and that turned out to be the weakest part of the folk story. Counting only events that landed after the last default-branch commit, bots are confirmed in 3 of the 14 flagged repos (splash 4 of 4, geziyor 5 of 5, any23 16 of 16). For 2 it's outright wrong: dragnet's gap comes from a human pushing an unmerged Python 3.10 port. Two more are mixed, including crawlab, where 24 human pushes went to develop and test while main sat still. And for 7 the activity feed returned nothing at all, so the cause is unestablished — retention explains only two of those seven.

Does a stale repo mean the tool is broken? Not on this evidence — nothing here was run against a live site. Staleness matters in proportion to how fast the target moves: browser automation, HTTP clients that mirror browser behaviour, site-specific parsers and API/LLM wrappers age quickly, while algorithmic libraries like SetSimilaritySearch or simhash can be years old and entirely fine. tls-client is the sharpest case in the sample, at 790,305 installs a month from a repo cold for 905 days.

How can a repo be active but the package still be dead? That's codelucas/newspaper, and it's the most consequential thing this audit found. Its default branch was committed on 2026-07-21, days before the as-of date, but newspaper3k on PyPI last shipped 0.2.8 on 2018-09-28 — 2,858 days — and still pulls 813,513 downloads a month. Repo-level checks all pass; the artifact you install is eight years old. Always check the registry's latest publish date separately from the commit log.

Do archived repos solve this? GitHub shows a banner. Not reliably, and @modelcontextprotocol/server-puppeteer shows why. Its npm repository field is null, so there's no link from the package to the archived repo and no banner to skip. What npm does deliver is the package's deprecated string — "Package no longer supported" — printed at install time, and 127,232 installs a month proceed through it. browserbase/mcp-server-browserbase announced its shutdown properly in its final commit; its 20,389 installs mostly predate that commit, so they say little either way.

How do I check my own dependencies quickly? Start with the default-branch commit date, registry version/upload date, npm deprecation field, and repository archived flag. Then verify package-to-repository attribution, inspect non-default branches when activity diverges, and use dependency graphs or lockfiles before calling exposure transitive. Name collisions such as the unrelated Java and PyPI wayback projects make that escalation necessary.