Please complete the NeXC Participant Feedback Survey each day or several times a day during the conference. Thank you!
The aim of the Twitter Scraping Tool was to create an online platform that serves as an early notification system for crop pests, diseases and environmental factors. The platform outputs are targeted to relevant experts and track the occurrence of pest, disease and environmental factors noticed and logged by trusted experts and key individuals in the field. This online asset is scalable, robust and stable as well as being flexible to grow as the scope of work expands, technology and ideas improve.
Specifically:
- The tool gathers (scrapes) data from the twitter platform. The data that will be gathered in the first instance includes: Crop, Disease, Pest, Abiotic factors and Location if available, along with the date and time of tweets.
- It has ability to retrospectively analyse Twitter to enable sampling of report-rich timeframes – especially for case study or proof of concept purposes.
- If an image is uploaded, this is stored for future records and observation.
- The tool will display the twitter user’s profile name (twitter handle) and allow them to be categorized as a Verified, Trusted or Unknown reporter. This determines the display ranking on the data feed, where experts are shown 1st and unknowns are shown last. User status infer the data quality. User’s status be able to be updated from unknown to trusted or expert status, as their expertise increases.
- A reporting dashboard collates data based on the number of pings and status’ of a twitter handle allowing data range sorting.
- Data is sampled according to Twitter’s sampling restrictions.
- The platform supports multi-tenancy, is easy to navigate, quick to load and can be used on mobile devices.
