Detecting AI Astroturfing in Regulatory Dockets

Most compliance teams think the solution to synthetic comment flooding is buying a better text classifier. That consensus is dead wrong. Relying on syntax analysis to catch generated text is a trap that inevitably leads to Administrative Procedure Act violations. You are staring at a docket with 10,000 comments submitted in 48 hours, and the uncomfortable truth is that your agency mandate requires you to consider them—even if half are synthetic. The firehose of generated noise is not a text problem. It is a coordination problem.

How can you tell if a comment is AI-generated?

You can tell if a regulatory comment is AI-generated by analyzing structural metadata rather than relying on text classifiers. Authentic grassroots campaigns exhibit natural variance in submission timing and semantic diversity, whereas synthetic astroturfing leaves mathematically verifiable signatures in timestamp bursts and identical metadata clusters.

The immediate crisis of synthetic flooding breaks traditional manual review. Human analysts simply cannot read thousands of pages a day. Astroturfing is the practice of masking the sponsors of a message to make it appear as though it originates from and is supported by grassroots participants. When bots execute this at scale, the legal framework gets stressed. The Administrative Procedure Act demands agencies respond to public comments, but it does not define how to handle a firehose of synthetic noise designed to artificially manufacture consensus.

In Perez v. Mortgage Bankers Association, the Supreme Court stated that an agency must consider and respond to significant comments received during the period for public comment. You cannot just discard the noise. Ignoring mass campaigns creates a legal trap. Recognizing this, Section 2(d) of the Executive Order on Modernizing Regulatory Review directs the Administrator of the White House Office of Information and Regulatory Affairs to consider guidance or tools to modernize the notice-and-comment process. This specifically includes addressing mass, computer-generated, and falsely attributed comments.

To manage this, the Administrative Conference of the United States issued Recommendation 2021-1, titled “Managing Mass, Computer-Generated, and Falsely Attributed Comments.” This document provides the baseline for separating unique legal arguments from duplicated volume. The APA paradox remains: you have a duty to respond, but you must first isolate the actual human intent from the generated filler.

Is it possible to detect AI-generated content?

It is possible to detect AI-generated content in public dockets by shifting from brittle text analysis to structural fingerprinting. By combining timestamp burst analysis with semantic clustering, compliance teams can mathematically prove coordination and isolate synthetic volume without triggering Administrative Procedure Act violations based on flawed text detection.

Existing agency guidance assumes generated comments can be caught via text classifiers. The real vulnerability is structural. Text classifiers fail on human-edited prompts and create an APA trap where you dismiss content based on detection errors. The pattern here is clear, and it is one that top-ranking guides miss: coordination leaves a heavier footprint than syntax. Combining timestamp burst analysis with ACUS semantic clustering creates a mathematically verifiable astroturf signature. This bypasses the need to prove the text itself was generated by a language model, thus avoiding the legal risk of guessing wrong.

Building this pipeline requires moving away from brittle models toward structural fingerprinting. Here is the exact sequence we use to isolate the noise:

1. Extract submission timestamps from the docket API to identify burst anomalies where dozens of comments arrive within seconds of each other.
2. Parse metadata fields for identical author patterns, focusing on sequential IP addresses or duplicated form fields.
3. Generate sentence embeddings for the comment text to map the semantic distance between submissions.
4. Apply density-based clustering to group semantic duplicates and identify the core arguments being repeated.
5. Cross-reference these semantic clusters with timestamp bursts to confirm coordination and flag the batch for manual legal review.

I have to admit what almost broke our early pipeline. We initially filtered out a genuine grassroots campaign because they used a shared template from a nonprofit advocacy group. Our semantic clustering flagged their identical phrasing as synthetic. We had to rebuild the feedback loop to weight timestamp variance higher than text similarity when the submitter email domains were diverse. Real writing has scar tissue, and forensic pipelines are no different.

To visualize the tradeoffs, we map our methods against the legal risks:

Triage Workflow: Structural vs. Textual Detection

Detection Method	Technical Vulnerability	APA Legal Risk
Text Classifiers	Fails on human-edited prompts	High risk of dismissing valid human intent
Timestamp Burst Analysis	Misses slow-drip campaigns	Low risk, relies on verifiable metadata
Semantic Clustering	Flags genuine shared templates	Moderate risk without domain variance checks

The frontier is shifting from detecting synthetic text to verifying human provenance. As the volume scales, the burden of proof changes.

Perhaps the White House will resolve to fight fire with fire and issue guidance that formally incorporates AI into not only sorting comments but also responding to them.

— Yale Journal on Regulation

Tools and frameworks for structural triage

Structural triage for regulatory dockets requires combining density-based clustering algorithms with vector databases and official government frameworks. Compliance teams should deploy open-source embedding models alongside established administrative guidelines to separate synthetic volume from unique legal arguments without relying on proprietary black-box classifiers.

You need tools that handle scale without obscuring the forensic chain of custody. We use HDBSCAN for density-based clustering because it does not force you to guess the number of clusters in advance. Sentence Transformers handle the embedding generation, converting raw text into vector space so we can measure semantic distance. Elasticsearch sits at the center, allowing analysts to query the metadata and vector clusters simultaneously.

Accessing the raw data is its own hurdle. Due to aggressive automated scraping, programmatic access to FederalRegister.gov and eCFR.gov is limited to access to their extensive developer APIs. If you attempt to scrape the site directly, they require users whose requests are flagged as potentially automated to complete a CAPTCHA bot test as part of their security measures. You must build your ingestion pipeline around their official API endpoints to maintain a clean data feed.

This structural approach mirrors the forensic rigor we apply when authenticating leaked aviation audio, where preserving the metadata chain is just as important as the audio itself. Whether you are analyzing black box recordings or regulatory dockets, automated enhancers and black-box classifiers destroy the forensic trail. For institutions needing to scale this kind of verification, deploying autonomous AI research teams for institutions ensures the triage workflow runs continuously without human bottleneck. The same logic applies when admitting synthetic cockpit audio in court; mathematical isolation of generative artifacts is the only defensible standard.

How we hit it

Our internal publishing metrics demonstrate how consistent structural analysis and forensic research perform in search indexing. By adhering to strict data provenance and avoiding synthetic filler, we maintain a reliable indexing cadence that reflects the rigorous standards we apply to public interest investigations.

Transparency in our own workflow is non-negotiable. This site has published 63 articles in the last 90 days. We track our search visibility closely to ensure our research reaches the compliance staff and journalists who need it. Currently, 44% of the 62 pages inspected via the GSC API in the last 90 days are fully indexed. The median time from publish to confirmed Google indexing on this site is 8 days, across 27 measured posts. These numbers reflect a deliberate strategy of publishing deeply sourced, structurally sound analysis rather than chasing daily trends.

This leaves us with an open question for the legal community. If a language model generates a highly unique, legally sound comment based on a human's vague prompt, does the APA's duty to respond apply to the synthetic output or the underlying human intent? The law has not caught up to the reality of prompt engineering.

If you want to test these concepts on your own docket this week, try these two experiments:

1. Run a semantic clustering algorithm (like HDBSCAN on sentence embeddings) over 1,000 recent comments to identify the core semantic seeds of supposed mass campaigns. Measure how many distinct legal arguments actually exist beneath the volume. 2. Correlate submission timestamps with API rate limits to find the exact cutoff where human typing speed is mathematically violated. Plot the time delta between sequential submissions from the same IP block to visualize the bot threshold.

MOBILIZR -- Writing at mobilizr.org