USDA - Web Log Analysis Dashboard Dataset

These are the monthly summarized server access logs for the following databases: Ag Data Commons, Dr. Duke's Phytochemical and Ethnobotanical Databases, SoyBase, FoodData Central, Life Cycle Assessment Commons, and GrainGenes. Server logs were initially collected and stored using Elasticsearch, and summarized and analyzed using Python, then reformatted in tabular format for use in Tableau.

The dataset contains the KPIs visitors, USDA Researchers, web crawlers, downloads, pageviews and countries. secondary_source_table.csv also contains counts of pageview and download requests that passed through Ag Data Commons to other databases. It can be used to assess Ag Data Commons’ impact on other databases.

• Resource Title: Main Table.

  File Name: main_table.csv

  Resource Description: Fields include: database, year, month, spider, non_spider, visitors, usda_researchers, non_usda_researchers, download_count, download_avg_size, pageview_count, download from ADC, pageview from ADC, usda_downloads, non_usda_downloads, usda_pageviews, non_usda_pageviews




• Resource Title: secondary_source_table.

  File Name: secondary_source_table.csv

  Resource Description: Fields include: database, year, month, secondary_source, downloads, pageviews




• Resource Title: Top Countries Table.

  File Name: top_countries_table.csv

  Resource Description: Fields include: Database, Year, Month, Country, Hits




• Resource Title: Top Download Table.

  File Name: top_download_table.csv

  Resource Description: Fields include: database, year, month, download_request, downloads




• Resource Title: Top Page View Table.

  File Name: top_pageview_table.csv

  Resource Description: Fields include: database, year, month, pageview_request, pageviews