Pod.Cast annotation system

In July, 2019, Pod.Cast was developed by Microsoft employees Akash Mahajan, Prakruti Gogia, and Nithya Govindarajan volunteering through Microsoft’s annual hackathon. Since then, the tool has labeled many rounds of Orcasound data. On-going development is led by Akash and Prakruti, mainly through AI4Earth & OneWeek hackathons.

Pod.Cast uses a machine learning model to accelerate annotation of biological signals in audio data through web-based crowdsourcing. The software lets a user:

  1. Predict the time bounds of signals in a recording based on a machine learning model.
  2. Visualize the audio data as a spectrogram and listen via a web-based playback UI that is synchronized with the spectrogram.
  3. Validate the predicted annotations and optionally add annotations manually, collaborating with a “crowd” of other human annotators.
  4. Save annotations from each labeling “round” in a .tsv file.
Pod.Cast UI (2020) with audio data from Orcasound

Open-source code & open data

Implementations

  1. Pod.Cast with Orcasound data and a killer whale call model. (VGG-ish model trained first on global killer whale calls from the Watkins Marine Mammal Library, and then on multiple rounds of Southern Resident Killer Whale call data from Orcasound hydrophones.)

Collaborators & contributions

  • Lead developers
    • Akash Mahajan (2019+, Microsoft*)
    • Prakruti Gogia (2019+, Microsoft*)
  • Other developers
    • Nithya Govindarajan (2019, Microsoft*)
  • Annotators / beta-testers
    • Scott Veirs (2019+, Beam Reach, Orcasound)
    • Val Veirs (2020+, Beam Reach, Orcasound)

* Note: this is volunteer-driven & is not an official product of Microsoft.

Support & credits:

More info:

Leave a Reply

Your email address will not be published. Required fields are marked *