now that I passed my thesis project, my advisor was kind enough to help me turn my thesis into a publishable work.
What’s the Plan
By the time I’m writing this, I’ve already spent a considerable amount of time revising state of the art solutions for fall detection. Specifically, there’s a IEEE Survey article that help a lot with summarizing various approaches. Advances in Skeleton-Based Fall Detection in RGB Videos: From Handcrafted to Deep Learning Approaches. You can read it for more information. In any case, my advisor suggested we take a different approach since there a numerous papers that already get to 100% accuracy in various databases. Now, the approach is to analyze the context of a given situation to predict one of two scenarios. Number one is to give a probability that a fall is going to occur, and second, after detecting the fall, we need to describe how severe the fall was, nothing too fancy but certainly a level up from just detecting whether someone fell or not.
My progress
So far I’ve been a bit stuck. I started using pytorch, since I already have experience with it, the problem is that using it in Google Colab, training any CNN based architecture with images of a resolution of even 256x256 will most definitely kill my session before the accuracy has reached 30%. So i decided to train YOLO models, in combination with the sdk that Roboflow offers for the latest versions it helped a lot. I made a simple database from existing images in the Roboflow universe, and I have some promising results.
The images above are a result from the same YOLOv8 model trained on the same dataset, obviously there’s room for improvement but before I continue babbling about what I can improve or what I plan to do, let me explain what the system is supposed to do. I talked about identifying two different scenarios, first we need to identify trip hazards or “dangerous” items/object/behaviors that can result in a fall. I tested this idea in chatpgt.
Basically the system has two parts. The item/object/behavior detection and the second model will describe or give a probability that a fall is possible or even imminent. The idea is simple and it works, however I do expect to encounter problems and to evolve this idea. I don’t know what problems I’ll encounter, I’m engineer and like any good one you should always expect a problem in a near future. Anyway, What gave me the idea of this system is a paper I saw on Satellite surveillance
Using Roboflow is great, but I need to figure out how to customize the model if I want to make any other progress. Another issue is that while the Satellite surveillance paper works, it’s intended use is for still images, not videos. So my next challenge is as follows:
- Investigate research about caption generation and object tracking based on videos
- Keep playing with roboflow to boost my research.
I was actually planning on writing this article about a paper I need to read for my next session, but this article is plenty long. I’ll write about it another blog this week.
