Normally, AWS developers use Jupyter notebooks hosted in Amazon SageMaker to experiment with GluonCV models. Jupyter notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations, and narrative text. In this tutorial you are going to create and run Jupyter notebooks directly on an AWS DeepLens device, just like any other Linux computer, in order to enable rapid experimentation.
Starting with version AWS DeepLens software version 1.4.5, you can run GluonCV pretrained models directly on AWS DeepLens. To check the version number and update your software, go to the AWS DeepLens console, under Devices select your DeepLens device, and look at the Device status section. You should see the version number similar to the following screenshot.
To start experimenting with GluonCV models on DeepLens, complete the following steps:
To do so, you need the IP address of AWS DeepLens on the local network. To find the IP address, select your device on the AWS DeepLens console. Your IP address is listed in the Device Details section.
You also need to make sure that SSH is enabled for your device. For more information about enabling SSH on your device, see View or Update Your AWS DeepLens 2019 Edition Device Settings.
Open a terminal application on your computer. SSH into your DeepLens by entering the following code into your terminal application:
When you see a password prompt, enter the SSH password you chose when you set up SSH on your device.
sudo python3 -m pip install –-upgrade pip sudo python3 -m pip install notebook sudo python3.7 -m pip install ipykernel python3.7 -m ipykernel install --name 'Python3.7' --user sudo python3.7 -m pip install gluoncv
jupyter notebook --generate-config
c.NotebookApp.ip = '0.0.0.0' c.NotebookApp.open_browser = False
Save the file (if you are using the nano editor, press Ctrl+X and then Y).
Open up a port in the AWS DeepLens firewall to allow traffic to Jupyter notebook. See the following code:
sudo ufw allow 8888
You should see output like the following screenshot:
http://(DeepLens or 127.0.0.1):8888/?token=sometoken
For example, the URL based on the preceding screenshot is
You should see something like the following screenshot.
Choose New to create a new notebook.
To capture a frame from the camera, first make sure you aren’t running any projects on AWS DeepLens.
import awscam import cv2 ret,frame = awscam.getLastFrame() print(frame.shape)
Alternatively, you can press the Run button in the Jupyter toolbar as shown in the screenshot below:
You should see the size of the image captured by AWS DeepLens similar to the following text:
(1520, 2688, 3)
The three numbers show the height, width, and number of color channels (red, green, blue) of the image.
%matplotlib inline from matplotlib import pyplot as plt plt.imshow(frame) plt.show()
You should see an image similar to the following screenshot:
Now that you have an image, you can use GluonCV pre-trained models to detect people and poses. For more information, see Predict with pre-trained Simple Pose Estimation models from the GluonCV model zoo.
import mxnet as mx from gluoncv import model_zoo, data, utils from gluoncv.data.transforms.pose import detector_to_simple_pose, heatmap_to_coord
people_detector = model_zoo.get_model('yolo3_mobilenet1.0_coco', pretrained=True) pose_detector = model_zoo.get_model('simple_pose_resnet18_v1b', pretrained=True)
yolo_mobilenet1.0_cocopre-trained model is trained to detect many types of objects in addition to people, the code below narrows down the detection criteria to just people so that the model runs faster. For more information about the other types of objects that the model can predict, see the GluonCV MSCoco Detection source code.
img = mx.nd.array(frame) x, img = data.transforms.presets.ssd.transform_test(img, short=256) class_IDs, scores, bounding_boxs = people_detector(x)
detector_to_simple_posefunction that takes care of cropping and resizing for you.
pose_input, upscale_bbox = detector_to_simple_pose(img, class_IDs, scores, bounding_boxs) predicted_heatmap = pose_detector(pose_input) pred_coords, confidence = heatmap_to_coord(predicted_heatmap, upscale_bbox)
ax = utils.viz.plot_keypoints(img, pred_coords, confidence, class_IDs, bounding_boxs,scores, box_thresh=0.5, keypoint_thresh=0.2) plt.show(ax)
After completing steps 1-6, you should see an image similar to the following screenshot.
If you get an error similar to the ValueError output below, make sure you have at least one person in the camera’s view.
ValueError: In HybridBlock, there must be one NDArray or one Symbol in the input. Please check the type of the args
So far, you experimented with a pose detector on AWS DeepLens using Jupyter notebooks. You can now collect some data to figure out how to detect when someone is hunching, sitting, or standing. To collect data, you can save the image frame from the camera out to disk using the built-in OpenCV module. See the following code: