DeepSea Technologies published a new piece of research outlining a new way of verifying the accuracy of a ship’s AI-generated model in real-world conditions.
The new approach was developed by seven of DeepSea’s team of research scientists headed up by Dr. Antonis Nikitakis, and presented at the 2022 HullPic Conference in Tullamore, Ireland, in May 2022.
[smlsubform prepend=”GET THE SAFETY4SEA IN YOUR INBOX!” showname=false emailtxt=”” emailholder=”Enter your email address” showsubmit=true submittxt=”Submit” jsthanks=false thankyou=”Thank you for subscribing to our mailing list”]
Because the reality of ship-at-sea data is highly variable, most model accuracy figures reported in publications and marketing materials fail to bear relation to the actual utility of those models in real use cases.
DeepSea has researched approaches to solving the technical challenge of boosting models’ ability to understand unseen (out-of-domain) conditions for years. However, until today, there has been no benchmark for evaluating this sort of competence within a vessel model. This is a crucial step of AI modelling because the more accurate the virtual model, the more efficient a ship can be made, and vice-versa.
The few models that currently provide an estimation of their accuracy all do so based on testing with data obtained from the same distribution as the data used to train the model. For example, if the model is trained on data from the vessel’s historical behaviour, in a narrow range of well-experienced wind speeds or drafts, it is also tested on data with these speeds and drafts. Thus, the tests performed cannot tell if the model is reproducing the biases in the training data – and whether it will work as well in different, never-seen-before conditions.
Following the publication of this body of work on proving the real-world utility of AI approaches, DeepSea will focus on advocating for common and transparent standards for AI real-world utility in the industry.
Commenting on the research, Dr Nikitakis, DeepSea’s AI Research Director, said:
This research is an important step in helping our customers and the wider shipping community to understand the true power of an AI-based approach while alleviating its limitations. Coupled with the daily real-world impact we’re seeing on fuel consumption and CII ratings, we believe this sort of information is key to popularising this incredible technology throughout the industry
EXPLORE MORE AT DEEPSEA TECHNOLOGIES’ RESEARCH
