Subreddit Tracking Subreddit Comparison Summary Methodology Support me on Ko-fi BETA
Summary of Data Collection Approach

This project tracks posting activity across various political- and news-oriented subreddits using daily snapshots.

The goal is not to recreate Reddit’s full historical dataset, but instead to capture publicly visible posts on a day-to-day basis. Each day, a script collects the currently visible posts and adds them to a growing dataset. Posts made within three hours of the data collection are temporarily excluded to provide a moderation window. Those posts may be collected during subsequent collections.

The dashboard focuses on a few key questions:

  • What percentage of users is needed to account for X% of posts?
  • How do these subreddits compare against each other?
  • Are there patterns in how frequently users post?

The following list provides a summary of how the data is collected. A link to the script and the full datasets (pseudonymized) are provided below the list.

  1. An initial historical sample was collected for each subreddit to establish a starting dataset. The collection uses Reddit's JSON-formatted "new posts" listing and pagination to retrieve approximately the 1,000 most recent posts available for each subreddit at the time of collection.
  2. The datasets are updated daily so newly submitted posts can be incorporated over time and the available historical coverage can gradually expand.
  3. Posts published within the three hours immediately preceding collection are temporarily excluded, giving moderators time to remove spam, rule-breaking material, or other unwanted submissions.
  4. Each daily collection intentionally overlaps existing data to reduce the possibility of missing posts when subreddit activity changes during the collection process.
  5. Reddit post IDs are used as unique identifiers, allowing overlapping collections to be merged without creating duplicate records.
  6. Viewable data has a lag of 5 days to allow the daily script and myself to identify and account for possible data collection errors. Data within that five-day period is considered provisional data.

Full pseudonymized data: Browse the CSV files on GitHub

Comparison to Arctic Shift

Arctic Shift and this tracker use different collection windows. Arctic Shift archives a post when it is initially published, then retrieves it again after approximately 36 hours to update selected fields and record changes such as deletion or editing. This tracker temporarily excludes posts published within the three hours preceding each daily collection. Once an eligible post is collected, it is retained in the dataset, and the dashboard displays the data after a five-day lag.

In the theoretical example below, each color represents the same post at the same horizontal position on both timelines. Arctic Shift moves forward with publication, capturing posts as they appear and retrieving them again approximately 36 hours later. My tracker begins at the daily collection time and looks backward through the available posts. Posts published during the preceding three hours are temporarily excluded from that collection, while older posts are collected and retained. Temporarily excluded posts may be collected during a later daily run if they remain available on Reddit.

Theoretical comparison of Arctic Shift and Live Reddit Tracker collection timing A two-lane timeline showing seven color-coded theoretical posts in matching horizontal positions. The Arctic Shift timeline points forward, capturing all seven posts near publication and retrieving them again approximately 36 hours later. My tracker timeline begins at the daily collection time on the right and points backward. It temporarily excludes the four most recent posts inside its three-hour moderation window and collects the three older posts. A separate forward sequence shows that collected posts are retained and become viewable after five days. Theoretical Post-Capture Timeline Matching colors and positions represent the same theoretical posts Arctic Shift Live Reddit Tracker Earlier publication Later publication All example posts captured ~36-hour update 3-hour cutoff older posts eligible Daily collection script runs (now) Older posts collected and retained Most recent 3 hours temporarily excluded from this collection After collection: Collected posts Retained in dataset Viewable after 5 days Same posts on both timelines Excluded from this collection Collected by tracker Timeline not to scale

Arctic Shift process: collection documentation.

Data Collection Notes

The following collection interruptions or recovery operations may affect interpretation of the dataset. Posts still visible when collection resumes can generally be backfilled, but posts removed during the interruption may not be recoverable. An empirical comparison of routine daily collection with Arctic Shift across all 13 subreddits found that post counts differed by approximately 15% on average. Counts for periods backfilled using Arctic Shift should therefore be treated as estimates.

Date Affected subreddits Note
9/15/26–9/19/26 All tracked subreddits Collection interrupted. Available posts were backfilled, but some removed posts may be absent.
9/14/26–9/15/26 r/politics Backfilled posts using Arctic Shift data.