The TrueAI PPE Advantage
- TrueAI was trained exclusively on construction-specific data, making it better at identifying PPE
- Unlike static models, TrueAI uses a feedback loop that lets users flag inaccurate detections
- TrueAI automatically analyzes images captured as frequently as once per minute
- TrueAI is built on AWS, bringing proven scalability, uptime, and enterprise-level security
TrueAI PPE Delivers Smarter, Construction-Trained AI for Safer Jobsites
Transcript
Alright.
Hello, everybody out there.
Welcome to this exciting webinar.
AI that knows
the job site, smarter PPE monitoring built for the field. This is presented by TrueLook and AWS.
So, good afternoon or good morning if you’re on the West Coast. We’re gonna kick things off. But first, I have some quick housekeeping tips for everybody. If you have any questions, please put them in the chat, and we will circle back at the end of the webinar as time permits and try and answer all of your questions.
And then also, just another tip. There’s a little full screen icon in the bottom right of your screen. If the screen is too small, if you click that, it’ll go full screen. You’ll see my face larger than life, unfortunately, but you will be able to see the demo much better.
So definitely click that, for the immersive experience.
So awesome. Let’s kick things off and get started with some quick introductions.
And, my name is Roger Yarrow. I am CEO and cofounder of TrueLook.
TruLook is a provider of professional construction camera systems that offer live streaming, time lapsing for project documentation, and a suite of security features. Our mission at TrueLook is total job site visibility.
My background is technology more than construction, but I am a home DIYer. I have a garage full of tools and all ten of my fingers still.
So I am happy here, to talk about safety and as well, I’m joined by some very smart people who I will let introduce themselves.
Mark, you’re next.
Hi. I’m Mark, software engineer at TrueLook. Our team worked with AWS to deliver TrueLook PPE. Pranav?
Hey, everyone. My name is Pranav Murthy. I’m a senior generative AI data scientist working for AWS. I’m a part of the Amazon SageMaker AI team, and I have many years of experience building AI models for computer vision and natural language domain. Over to you, Surya.
Hey. My name is Surya Curry. I’m on the same team as Pranav. I work for AWS as a senior data scientist as well.
I have a particular interest in computer vision. I’ve been doing that for a for a while now. So, this has been an exciting collaboration. I’m happy we’re all happy to show it to you, all of you.
Great. Awesome. So, those are the smart guys.
But, definitely,
I’m gonna run us through what we’re gonna chat about through today.
We have an agenda because everybody loves a good agenda.
We’ll see that in a second here on the screen. Beautiful. And what we are gonna talk about is the what, why, and how of protective personal equipment known as PPE. Then we’re gonna introduce TrueLook PPE, which is why we’re all here today, to show you how this is a great, creation of this collaboration. And then after that, I’m gonna lean on my smart colleagues again here to explain further how we built this and why we think it’s better than a lot of what’s available today. And then you’ll get to see it in action and how you can take advantage of it for yourself.
So, yeah, let’s dive right into it.
As everybody knows, PPE is personal protective equipment.
This can be hard hats, safety glasses, gloves, high visibility vests, steel toe boots, hearing protection, fall harnesses, respirators.
The list goes on and on. PPE exists for obvious reasons for the obvious reason of keeping workers safe. It helps prevent injuries or worse, and this is why PPE use is mandated by OSHA and other organizations.
It requires employers to assess workplace hazards, provide appropriate PPE, train workers on its proper use, and ensure it is worn and maintained consistently to protect.
When this requirement is not complied with, workers are at risk, but you can also get hit with fines, lawsuits, and delays that cost real money and time.
Safety is ultimately part of culture, and the culture of a job site should incorporate safety. A strong safety culture tells your crews, your clients, and all stakeholders that you care about people and about doing things the right way.
So this all sounds like common sense on paper, but it’s much harder in reality. Getting consistent PPE use across big complex job sites every single day isn’t as straightforward as it sounds.
PPE enforcement is everyone’s job. It is a shared responsibility of the safety manager, superintendent, foreman, and the individual workers. The rules are set, but someone has to enforce them.
Enforcement, however, can be a challenge. There’s several methods out there that already exist to to basically spot check PPE compliance, but these are traditionally points in time where you’re checking.
For example, the safety walk through. These can help, but they only show what’s happening in that moment. As soon as the safety manager leaves, as soon as the inspector leaves, conditions can change, and and it may not be safe anymore. You could have just inspected a moment of safety, but you’re not having continuous safety.
Drones are a great tool for inspections and capturing progress for photos and all other kinds of of documentation, but they’re scheduled. They only give you visibility as often as you fly them, and they certainly can provide the consistency you need for regular PPE checks.
And then there’s also cameras, which is our world, TrueLook.
Cameras provide a wealth of valuable footage and all sorts of use cases doing a wide variety of tasks. But, ultimately, if you’re relying on looking through hours of footage, manually scrubbing video, trying to find, is is everybody doing the right thing. It becomes very time consuming, and it’s still prone to human error.
Important issues can easily get missed. So here we have TrueLook PPE.
This is TrueLook solution to the challenge of how do we continually make sure, you’re compliant. And TrueLook leverages its cameras with TrueAI PPE to help with compliance.
It’s going to provide constant monitoring to help you identify when and where gaps in compliance are occurring. Every minute, the camera looks at all the workers in view for their high visibility gear and helmets.
Every image that’s captured is labeled and turned into data to identify trends and problem areas. This can help to also identify opportunities where additional training or oversight is needed to ensure workers are kept safe. This is not about trying to get people in trouble. It’s about making sure that the right steps are taken to keep both your teams and your company safe. I’m gonna pass things to Mark, to explain a bit more.
Thanks, Roger.
When we started out on this project, we evaluated many frontier and open source a o AI models as well as third party services.
So why didn’t they work? The answer is that when you have an image of someone standing directly in front of the camera perfectly in frame, looking straight at it with a big cheesy grin, it’s not too difficult for these models to determine if someone is wearing a hard hat or a safety vest. But that’s not how our workers present on our construction sites.
They are often viewed top down at a distance, obscured by objects, each other, and even themselves if you can imagine someone tiling on the ground.
Typically, the lighting is varied, and depending on the season, they’re wearing different forms of PPE. So you can, think of a hard hat with and without a sunshield or T shirts versus jackets.
So in summary, these models were not trained on data that is representative of the complexity in our construction sites.
This is why we trained our own model on our data simply to realize better precision in PPE detection by using representative data.
On the topic of scalability, our solution is deployed to AWS SageMaker and is scalable beyond our needs. The detection can be as often as every sixty seconds, and this is configurable by the end user.
In the feedback loop, even though our model performs better than models we’ve evaluated, it is not perfect. We have to improve confidence, and it will sometimes get things wrong.
However, quality improvement will be monthly using, firstly, the images flagged in our user feedback functionality that we’ll demonstrate later and, two, mining TrueLook north of one billion images. We plan to train and deploy an improved model on a monthly cadence.
Finally, the technology works out of the box on all of Trulox cameras already in the field simply by enabling in the app. I’m gonna pass it over to Pranav with AWS.
Thanks, Mark.
So the Amazon Seismic AI data science team partnered closely with, TrueLook, engineering team to design this three stage training pipeline that outperforms state of the art models, for construction safety.
In the first stage, we started off by selecting a model that’s capable of detecting a broad range of target classes. Right? We needed the model to be aware of, of people, places, and things. So for instance, it was able to identify, people, vehicles, everyday objects. This gave the model a baseline understanding of, of all the objects that are typically observed by us humans in lot of different scenarios.
The second stage, we took this, base model, and we adapted that model to deconstruction domain.
We, retrained it on construction specific classes where the model learned to detect people and objects in job sites with high accuracy. In other words, we shifted it from recognizing generic scenes to recognizing construction scenes.
In the last stage or the third stage of the pipeline, we fine tune the model even further using TrueLook own dataset.
With just a handful of images, we were able to unlock significant performance boost.
Now that the model has learned to identify classes like hard hats, boots, people in TrueLook, specific construction zones really well. Together,
TrueLook and the Amazon SageMaker AI team confirmed that this three stage approach was the most effective, after running multiple experiments to validate the results.
But what’s the next step? Right? The next step was to operationalize it. Operationalizing it. Locking it, locking the process into place with continuous integration and continuous deployment pipelines. Now I’ll hand it over to Surya who will walk us through the Amazon SageMaker AI automated process.
Yeah. Thank you, Pranav.
So as Pranav said, the next step here was to take what Pranav and, the data science team built to operationalize it and to have the ability to experiment with multiple different hyperparameters, different instances, different GPU types.
And we built an entire workflow with SageMaker Pipelines that would help, True dot ai retrain models automatically as new data becomes available because TrueLook ai does have the ability to go in. And if it if there is a wrong prediction, customers can flag it and it will automatically go into this retraining pipeline and it will always make the newest and the greatest model available for for, TrueLook customers.
And we could enable multiple experiments to run-in parallel with the SageMaker AI pipelines that we built. And we’ve also built the ability to gate models, experiment with their, hyperparameters, and also be able to find, evaluate them and then make available only the top quality models.
Next slide, please.
So with this pipeline, what we’ve enabled is we’ve enabled a system that analyzes images near real time every sixty seconds.
We also have the ability to handle thousands of detections across multiple job sites.
This is backed by AWS’s, enterprise grade security.
It is backed by AWS’s ability to scale as new requirements come. And because this is backed by some of the some of the best GPUs that, AWS can offer, it is extremely fast.
And like I’ve mentioned multiple times right now, if we only deploy the the models that have the highest accuracy to production.
I think I’m gonna hand it over to Mark at this point.
Okay.
So let’s talk about enabling, first. As of today, TrueLook PPE can be activated by contacting your account executive.
However, there’ll be a release next week where assistive admins can enable it for themselves. So I’m gonna walk through that part.
So assuming TrueAI is enabled at the, customer project and camera level, the TrueLook PPE is enabled at the time lapse level.
So using, this account here, I’m going to work on PPE testing too, and we’re going to click on the edit drawer and then the edit time lapse.
Now before we actually enable PPE detection for a time lapse, there are some selection criteria, specifically time lapse selection criteria we need to pay mine to. They’re pretty simple, but what we’re looking for is around at most one standard time lapse per camera.
We want the field of view to be obviously directed at human activity, But, ideally, we want the people that appear within the frame to appear at a higher resolution. So what does that
mean practically?
We need to be able to, within the time lapse image, discern facial features ideally, but differently head from neck from torso, from legs.
This will be optimal. We’re not we’re not looking for extreme angles. Extreme top downs won’t work as well as something like this, which is a mild top down.
And preferably, the lighting is is favorable, but the model has been trained on quite a variety of data.
Okay. So in the bottom left, once we enable allow PPE detection and click save on the right, obviously, it’s already enabled. All of the time lapse images, however they’re configured, and in this case, every seven minutes, will be routed to the TrueLook PPE back end, and all detections will be persisted, within our database.
So let’s take a look at the visualization of that data now.
In order to do that, what we’ll do is in the breadcrumb navigation at the very top of the page, we’re going to click it click on the, time lapses and then the TrueLook menu item.
This will present, the violations timeline. So
let’s talk through this.
Clearly, on the x axis, we have days.
On on the y, violation counts. And there are three lines, one for each of the violation classes the current model detects, and, obviously, this will be expanded over time.
So they are no safety best, no high visibility upper body, and no hard hat.
Now if we were to hover over the fifth of September last Friday, we can see summary counts for each of the by election classes. And for this time lapse, we’re looking at thirty six in SAT best, two, no high visibility upper body, and no one, no hard hat.
To finish off this view in the top left, we have the date filter, which enables the user to filter by standard and custom date ranges.
Then all the way over to the right, we’re able to filter by PPE enabled time lapses so we could, look at different without having to navigate through, from the time lapse list again.
So look. Let’s look at the next feature now. If we hover over the fifth, click on the line graph point here, we’re presented with the violation viewer.
Now this looks very much like our time lapse viewer, and and and does have similar features.
Obviously, the the time control at the bottom allows us to to click through the time lapses, and suddenly there is view, etcetera. So but what we’re gonna work through is this control in the top right. First, I’m going to zoom.
You know, I’m this gentleman hanging out of the door here, and then go for it. So looking at the top right, we have this violation control.
The very first toggle enables us to turn on all of the time lapses irrespective of whether there was a violation of not or not.
By default, you’re only viewing time lapses for which there were violations detected. But if we toggle it off, you can see that much, there are many more time laps presented in the time control at the bottom. Now let’s toggle it back on.
So let’s talk about this gentleman here.
The model has before we do that, let’s just talk about the tree view here. Here we have the three classes.
No hard hats, no high visibility upper body, no safety vest. And for each of the leaf items, for whatever bit of description, we have this ID, which is simply a unique identifier for a violation.
So we can see that the model was able to assert that this gentleman is not wearing a hard hat.
It presents the confidence as well, but this is accurate. Obviously, he’s wearing a baseball cap.
It’s also able to identify that, he’s not wearing a safety vest, and he’s also not wearing high visibility on the upper body.
So we can toggle these off and on to to dive into the violations.
If we look at the gentleman to the left, we can see that the model, correctly infers that he’s wearing high visibility, a safety vest, as well as a hard hat.
If we pan all the way to the left, and turn the violations on, we can see that the model correctly identified he’s wearing a hard hat as well as high visibility on the upper body, but he is not wearing a safety vest.
So these are the the violations that we currently detect. In this case, it’s done it, very well. But in some cases, hopefully, the minority, and I believe the minority, it will get it wrong. And this is this is the feedback loop that Suya talked about and I talked about at the start.
And there’s two ways we get feedback and improve on the model. One is in the top right control. If you hover over either a violation class or an instance, you’ll see a small icon, garbage can icon, as well as a delete tooltip.
Now the reason we would use this is if the model correctly, incorrectly asserted no hard hat. So if we take this gentleman to the left, if there was a red box around his head that said no hard hat, this would essentially be a false positive. And we need to feedback to the model, the image in order to for it to get better over time.
So the end user can click delete, and this will have two effects.
One is the violation will be suppressed both in this view and the timeline view, but more importantly to me, the image will be routed for labeling and training. And ideally, in the next version of the model, it would better deal with, this image.
The other feedback feature is this thumbs down icon on the left, and this is for false negatives. And the use case here is that if someone presented without a hard hat, however, there wasn’t the red bounding box, the model incorrectly, didn’t identify no hard hat. And within here, the user in that case would click the thumbs down, describe something up, some pros to the effect that the gentleman far left doesn’t have a hard hat, the model missed it. Once they click save, again, the image will be routed for labeling and ultimately training. And in the next, monthly iteration of the model, it would be corrected.
I think that’s it.
Roger, back to you.
Awesome. So like I said, those are the smart guys, and they have built an amazing tool to help keep workers safe. The detections done by TrueLook PPE can become a powerful addition to your safety and compliance programs.
So, that’s a lot of great info, but now you’re probably wondering how do I get TrueAI PPE.
It’s actually very easy if you are a TrueLook user. Every single Trulik camera supports TrueAI PPE right now, and we are giving away free for the remainder of the year. You don’t need any new equipment or upgrades to support TrueLook PPE. It works on every TrueLook camera, past and present, in parallel with all of our standard features of live video, time lapsing, security, and so on.
We want as many people using TrueAI PPE as possible because this will help us improve accuracy, which ultimately will improve safety.
Our goal is to make sure nothing is missed and that you have all the information and tools needed to make sure you have a safe and compliant job site.
So that’s pretty much it. I wanna give a big thanks to all my co presenters here and everybody, who helped in the effort to get TrueLook ai created.
I appreciate all of their talents and time as well as all of yours, our attendees.
If there’s any questions, we’re happy to do our best to answer them. A lot of information is gonna be available on our website, TrueLook dot com.
You’ll find our phone number there as well as, a variety of ways to contact us. Info at true look dot com is another way to, reach out and get additional information about TrueAI PPE and all the additional services you can find with TrueLook.
And then, again, I do wanna say thanks once more to, AWS, for this amazing partnership to develop what we think is
something that is just, pretty neat for lack of a better word to say it. A great technical challenge overcome through teamwork and collaboration.
And, definitely wanna thank all of you, our attendees, for coming today, from learn for learning about, TrueAI PPE. And then, again, any of us would be happy to answer questions offline.
So, you know, please reach out. Don’t be shy. We love all this kind of stuff. So happy to hear from you. And, again, thank you.
