Why is this Solution required ?
In large residential and multi-purpose complexes CCTV cameras and security deployments are commonplace. In spite of these security measures, car burglary and car damages have been reported frequently. This shows that existing security systems have shortcomings. Particularly the CCTV doesn't raise alarm when there is unusual activity in the parking area. It is a common practice to use CCTV footage for post-incident investigations.
Solution Definition
We need a solution to promptly alert the car owner in real-time through an email with stranger’s image. The email should be delivered to the owner’s smart phone whenever a car burglary is going to happen in the car parking area. Car owners could then immediately inform the security staff and take actions to prevent a car break-in and also nab the burglar.
Solution components
The three key components of this solution are NVIDIA Jetson Nano H/W, AWS cloud services(S3, Lambda function, SES) and a smart phone with email configured on it. The current jetpack version on Jetson Nano is 4.6. Ensure more software components such as cuDNN, Gstreamer workflow frame work, Tensor RT SDK for High Performance ML inference, are available on Jetson Nano. Another key requirement is that the jetsoninference, a GPU optimized library is configured. The instructions for the same are provided in the link. The installation/configuration/execution was done on the cpu/OS layer(we didn't use the container image, though that was an option). In order to test this solution, it is assumed that the user has access to AWS cloud. For test purposes the free tier on the AWS cloud is sufficient.
How does the solution work?
Edge/AI, cloud and mobility are at the core of this solution. AI with powerful pre-trained models like SSD Mobilenet v2 running on an edge device can accurately detect persons from the incoming video stream.
A solution diagram in Figure 1 depicts the role of each component in the workflow. A video of a parked car from the parking is streamed into an object detector application which in turn scans the video frame by frame. If the application detects any human near the car, a bounding box is drawn around the human(intruder).
The application takes snapshot image of this intruder and uploads it onto AWS S3 bucket. As soon as a new object is Put onto the s3 bucket, an AWS service called Lambda gets triggered and it sends an email alert to the car owner with the intruder’s image. For email, AWS SES service is used.
Solution deployment steps:
On the AWS cloud, Login to AWS console with suitable credentials:
Create a s3 bucket named YOUR-BUCKET-NAME. Point to the same bucket name in the AWS API call in detectnet-snap-2s3.py
Configure AWS Lambda function, by specifying the Destination to send email to CAR-OWNER-EMAIL-ID as soon as the detectnet application puts a snapped picture.
AWS Simple Email Service (SES) is used to send transactional email to recipients
Configure email on smart phone. The email id is CAR-OWNER-EMAIL-ID.
Configure users AWS credentials on jetson nano ~/.aws directory
On Jetson Nano: Launch the following script
On Jetson Nano: ./detectnet-snap-2s3.py CarParking-II-2022-03-05-11PM.mp4 -threshold=0.5 --input-flip=clockwise
The CarParking...11PM.mp4 above is a sample video stream from the car parking area. This is done to demonstrate the application. In real scenario live, video stream will be used. This requires only very minimal changes to the detectnet python code
On CAR-OWNER mobile: CAR-OWNER receives an email alert with intruder's image.
Solution Benefits
Real time alert based on AI/object/human detection helps in preventing burglary and nabbing the culprit
Incrementally enhance existing CCTV surveillance and prevent and protect assets
Since this is an EDGE/AI based solution, the H/W and S/W investments are low
Acknowledgements: I thank Dustin Frankilin of Nvidia for quickly responding to my emails and answering my questions.