Embedded Files
Inspiration
This project was developed for the IEEE Technothon, a three-week design challenge. I decided to creatively tackle a unique tradition at Texas A&M: removing your hat inside the Memorial Student Center (MSC) to show respect for past alumni. Despite signage, visitors often forget. I wanted to build an automated, direct, but polite reminder. The goal was an AI-powered turret that detects a hat-wearer, physically points at them, and announces, "Please don't forget to remove your hat. Thanks and Gig 'em!" until the hat is removed.
Fabrication steps
AI & Bias Mitigation:
AI & Bias Mitigation:
Fine-tuned an Ultralytics object detection model on over 5,000 images. Identified an ethnic bias in the initial dataset and successfully sourced a more diverse dataset to ensure high-accuracy detection across all demographics.
Computer Vision & Fixing AI Bias
Going into this hackathon, I had zero experience with computer vision. I spent the first two weeks deep-diving into Ultralytics and pre-trained YOLO models. I compiled a dataset of over 5,000 images to fine-tune the model, but during testing, I noticed a critical flaw: the model was highly accurate for individuals of Asian descent but struggled significantly with other demographics. I realized my open-source dataset was heavily skewed. Recognizing this algorithmic bias, I immediately scrapped it, sourced a much larger, globally diverse dataset, and retrained the model. This correction drastically improved the model's mean Average Precision (mAP) and overall confidence scores across all users.
20251111_124514.mp4
This was after one of my first sucsessful training sessions; after 50 epochs it was able to classify hats pretty accurately, though struggled in low light environments.
20251110_233523.mp4Hardware Pivot:
Hardware Pivot:
Rapidly prototyped the physical turret under a strict 3-day deadline. Transitioned from a 12V DC motor to a servo motor to achieve absolute positional control without endless tracking loops.
Mechanical Constraints & Prototyping
With the AI functional, I had only three days left to design and build the physical turret. My initial CAD design utilized a 12V DC motor for rotation. However, I quickly realized that without an encoder, the DC motor lacked positional feedback. The turret couldn't determine its absolute angle, causing the tracking logic to enter an endless loop of overcorrecting left and right.
To solve this, I pivoted to a servo motor. While this limited the field of view to 180 degrees, it allowed me to send precise, absolute angle commands for accurate pointing. With only two days remaining, I completely redesigned the turret mount in CAD and 3D printed the final assembly overnight.
Network Architecture:
Network Architecture:
Configured an ESP32-CAM to host a local network, streaming live video to a laptop for heavy AI processing, which then transmitted precise targeting angles back to the turret.
System Integration & Final Results
Because the ESP32-CAM lacks the processing power to run the Ultralytics model locally, I implemented an edge-computing architecture. I programmed the ESP32 to act as a local access point and host a web server. My laptop connected to this local network, grabbed the live video feed, ran the AI inference in Python to calculate the bounding box coordinates, and transmitted the corresponding servo angles back to the ESP32 in real time.
20251112_220337.mp4The final presentation was a massive success. I took a highly original idea, learned computer vision from scratch, managed hardware constraints, and integrated a full IoT network, ultimately placing 2nd out of 11 teams!
Page updated
Google Sites
Report abuse