Artificial Intelligence of Things (AIoT) specialist Fiberpunk, which is aiming to “make 3D printing smarter,” has designed an Espressif ESP32-based camera which can sit on a network and be easily picked up by a central system as a feed for YOLO- based object detection models: Sentry Pro.
“We have some YOLOv models that need to be put on [a] local PC computer for watching and doing things,” Fiberpunk’s founders Noah Chen and WeiTing Lan explained. “We looked at a lot of cameras, but they didn’t work for us because they couldn’t do some important things we needed. We made something called Sentry, which is free for anyone to use.
“It lets you do things like find cameras on your computer network,” the team continues, “and connect to other devices like sensors or motors. It’s also easy to set up and use. We’ve used Sentry to do some cool things, like detecting when a 3D printer messes up, or finding if there’s a fire. We’ve even used it to count the number of goats and identify birds in a yard!”
The Sentry camera itself is based on an Espressif ESP32-SU microcontroller module, giving it a Wi-Fi radio for connection to the local network. There’s a single front-facing camera with a 120 degree field of view (FOV), plus UART and I2C interfaces for control of or expansion via external hardware — allowing a single Sentry to both provide a video feed and act as a sensor node or relay to control a connected device.
The AI ​​part of the team’s AIoT vision, though, doesn’t occur on-device. Instead, Fiberpunk has created a centralized application which can rapidly scan the network for and connect to local Sentry cameras then use them as an input for a YOLO-based machine learning algorithm.
The cameras tie in to the company’s Nexus AI platform, detecting print failures and fires. (📷: Fiberpunk)
“[Sentry can] detect [3D] print issues and signs of potential fires,” the company explains of its out-of-the-box capabilities, which are naturally focused on 3D printing. “If Sentry identifies potential problems, it can notify users with images and warning messages. The detection can also improve over time as Sentry collects more data to enhance the computer vision model.”
The company is selling pre-assembled Sentry Pro cameras through the Fiberpunk store at $33.99 each, a $6 discount during what the company calls “final testing,” while it has made the firmware available on GitHub under the reciprocal GNU General Public License 3. The AI ​​portion of the puzzle, henceforth, is based on the company’s Nexus AI platform — available as a free plugin for OctoPrint.