Ag Data Commons
5 files

USDA - Web Log Analysis Dashboard Dataset

posted on 2024-02-15, 19:55 authored by Siyi HuangSiyi Huang, Russell Brown, Cynthia Parr

These are the monthly summarized server access logs for the following databases: Ag Data Commons, Dr. Duke's Phytochemical and Ethnobotanical Databases, SoyBase, FoodData Central, Life Cycle Assessment Commons, and GrainGenes. Server logs were initially collected and stored using Elasticsearch, and summarized and analyzed using Python, then reformatted in tabular format for use in Tableau.

The dataset contains the KPIs visitors, USDA Researchers, web crawlers, downloads, pageviews and countries. secondary_source_table.csv also contains counts of pageview and download requests that passed through Ag Data Commons to other databases. It can be used to assess Ag Data Commons’ impact on other databases.

Resources in this dataset:

  • Resource Title: Main Table.

    File Name: main_table.csv

    Resource Description: Fields include: database, year, month, spider, non_spider, visitors, usda_researchers, non_usda_researchers, download_count, download_avg_size, pageview_count, download from ADC, pageview from ADC, usda_downloads, non_usda_downloads, usda_pageviews, non_usda_pageviews

  • Resource Title: secondary_source_table.

    File Name: secondary_source_table.csv

    Resource Description: Fields include: database, year, month, secondary_source, downloads, pageviews

  • Resource Title: Top Countries Table.

    File Name: top_countries_table.csv

    Resource Description: Fields include: Database, Year, Month, Country, Hits

  • Resource Title: Top Download Table.

    File Name: top_download_table.csv

    Resource Description: Fields include: database, year, month, download_request, downloads

  • Resource Title: Top Page View Table.

    File Name: top_pageview_table.csv

    Resource Description: Fields include: database, year, month, pageview_request, pageviews




Data contact name

Parr, Cynthia

Data contact email


Ag Data Commons


  • Not specified

ISO Topic Category

  • farming
  • health

National Agricultural Library Thesaurus terms

life cycle assessment; data collection; ethnobotany; chemical constituents of plants; databases

Pending citation

  • No

Public Access Level

  • Public

Preferred dataset citation

Huang, Siyi; Brown, Russell; Parr, Cynthia (2019). USDA - Web Log Analysis Dashboard Dataset. Ag Data Commons.