Due date: Oct 1

Part 1: Feature engineering for identifying Individuals with ASD

Download the contents from here.

Find the Assignment1.ipynb. The notebook uses a dataset collected for a project aimed to identify people with Autism Spectrum Disorder. Research has indicated that an autistic individual often shows atypical fixation and gaze patterns. In this project, the researchers have demonstrated that by monitoring a user’s gaze as they watch videos, they can identify individuals with an autism spectrum disorder. They recruited 35 autistic and 25 non-autistic individuals and captured their gaze using an off-the-shelf eye tracker connected to a laptop.

The notebook already has code to read the recorded data and load gaze points for each participant for each video. There are 4 videos in the zip file right now for the reference. And the features for those 4 videos are provided in the feat.zip. You will find this file in the assignment_1.zip. Use google colab to run the .ipynb notebook and upload the feat.zip. The goal of the assignment is to use the loaded data to extract meaningful features, normalize your data, ensure there is no class imbalance, try out different machine learning algorithms, and assess the effect on performance as you change some parameters of the algorithms.

The notebook has specific questions and grading strategy in more detail.

Submit your notebooks in ipynb format.

Part 2: Audio-based activity recognition

Develop a machine learning pipeline to detect activities and events using sound. The assignment will involve data collection, pre-processing/signal conditioning, feature extraction, using an existing ML implementation, and analysis of results.

Data Collection (5% grade):

Collect 20 samples each for five classes:

  1. Microwave (run for 30 seconds, and include door opening, closing, and beeps as part of each recording) Microwaves are not supposed to run while empty, it will break if you try this.
  2. Blender (run for 30 seconds). Do not run the blender at high-speed, it can overheat and break.
  3. Fire alarm or any other kind of siren (can be from videos or apps. Please don't trigger your alarms!)
  4. Vacuum Cleaner (run for approx. 30 seconds and try to move the vacuum cleaner around as it will change the sound profile a bit)
  5. Music (approx 30 seconds for each sample, the music of your choice). Vary the song (e.g., ten songs with two samples each)

For any device you might not have (e.g., please don't trigger an actual fire alarm), find a recording on the Internet (maybe on YouTube) and record its sound on your phone. Make sure not to use the audio file directly from the Internet. Play it back and make your recording of the audio file because you want the general variability between recordings for your 20 samples. You do not need to choose 20 different examples of a sound. For example, if you can't access a blender, don't search for 20 blender sounds on the Internet. Find one sound and record it 20 times. Identifying 20 different blenders as "blender" is a much harder problem for a homework.

For recording the sounds, you can use these apps (feel free to try out others, but make sure that you are recording uncompressed WAV files):

iOS: Voice Record

Android: Wav Recorder