In just 48 hours, Jared Drueco and his team won 1st place at HackED 2025 by building tabletop fighting robots that use computer vision to translate human movements into robotic actions in real time. The system tracks body poses via a laptop camera and maps them to robotic motions using servo motors and microcontrollers. Here’s how they did it:
- YOLOv11 and OpenCV: Implemented for real-time body movement tracking.
- C++ Embedded Software: Developed for servo control, using sockets for wireless communication between the laptop and robot.
- Custom Hardware: Redesigned classic fighting toys with servo motors and ESP32 microcontrollers for joint manipulation.
- PyQt Desktop App: Created for user onboarding and robot connection.
Devpost: https://lnkd.in/gT7n7Uwx
GitHub: https://lnkd.in/gBBbPdPK
Practice-Verified Codes and Commands
1. YOLOv11 Pose Detection Setup
Install YOLOv11 and OpenCV for pose detection:
pip install opencv-python git clone https://github.com/your-repo/YOLOv11.git cd YOLOv11 python detect.py --source 0 # Use webcam for real-time detection
2. C++ Socket Programming for Wireless Communication
Example of a basic socket server in C++:
#include <iostream>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>
int main() {
int server_fd, new_socket;
struct sockaddr_in address;
int addrlen = sizeof(address);
server_fd = socket(AF_INET, SOCK_STREAM, 0);
address.sin_family = AF_INET;
address.sin_addr.s_addr = INADDR_ANY;
address.sin_port = htons(8080);
bind(server_fd, (struct sockaddr <em>)&address, sizeof(address));
listen(server_fd, 3);
new_socket = accept(server_fd, (struct sockaddr *)&address, (socklen_t</em>)&addrlen);
char buffer[1024] = {0};
read(new_socket, buffer, 1024);
std::cout << "Message from client: " << buffer << std::endl;
close(new_socket);
close(server_fd);
return 0;
}
3. ESP32 Microcontroller Code for Servo Control
Example Arduino code for controlling a servo motor:
#include <ESP32Servo.h>
Servo myServo;
void setup() {
myServo.attach(9); // Attach servo to pin 9
}
void loop() {
myServo.write(90); // Move servo to 90 degrees
delay(1000);
myServo.write(0); // Move servo to 0 degrees
delay(1000);
}
4. PyQt Desktop App for User Onboarding
Example PyQt5 code for a basic GUI:
import sys
from PyQt5.QtWidgets import QApplication, QLabel, QWidget
app = QApplication(sys.argv)
window = QWidget()
window.setWindowTitle('Robot Onboarding')
window.setGeometry(100, 100, 300, 200)
label = QLabel('Connect to Robot', window)
label.move(100, 80)
window.show()
sys.exit(app.exec_())
What Undercode Say
This project exemplifies the fusion of computer vision, embedded systems, and software engineering to create an interactive and innovative solution. The use of YOLOv11 and OpenCV for real-time pose detection demonstrates the power of modern AI frameworks in bridging human actions with robotic responses. The C++ socket programming showcases the importance of efficient communication protocols in IoT and robotics, while the ESP32 microcontroller code highlights the role of embedded systems in hardware control.
For those looking to dive deeper into similar projects, consider exploring the following resources:
– OpenCV Documentation: https://docs.opencv.org/
– ESP32 Arduino Core: https://github.com/espressif/arduino-esp32
– PyQt5 Tutorials: https://www.tutorialspoint.com/pyqt5/index.htm
In the realm of cybersecurity, understanding embedded systems and IoT communication protocols is crucial. For instance, securing socket communication can be achieved using SSL/TLS encryption. Here’s a quick command to generate a self-signed SSL certificate for secure communication:
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes
Additionally, Linux commands like `netstat` and `tcpdump` can be used to monitor network traffic and ensure secure communication:
sudo netstat -tuln # List all open ports sudo tcpdump -i eth0 port 8080 # Capture traffic on port 8080
For Windows users, PowerShell commands like `Test-NetConnection` can help diagnose network issues:
Test-NetConnection -ComputerName 192.168.1.1 -Port 8080
This project not only highlights technical prowess but also underscores the importance of teamwork and innovation in solving real-world problems. Whether you’re a beginner or an expert, the integration of AI, robotics, and software engineering offers endless possibilities for exploration and growth.
References:
Hackers Feeds, Undercode AI
