Due Diligence for Machine Learning in Mineral Exploration: A Technical Framework

Machine Learning and Artificial Intelligence (ML/AI) promises revolution across the mining industry. It is being integrated into mineral exploration workflows and operational procedures, adding a new dimension that leaves traditional due diligence approaches in unfamiliar territory.

As ML/AI is being increasingly used for predictive modelling in core scanning, enhancing resource estimation, geological mapping and even in basic exploratory data analysis (e.g. cluster analysis), it is increasingly difficult for a Competent Person/Qualified Person/Auditor to fully understand the technical details of the methods being used.

Rarely do they have the technical depth to understand the ML/AI that is going on under-the-hood of some technologies, or how they have been deployed inside a business, to conduct the algorithmic validation to that piece of drill core analysis.

The stakes are increasingly high in this area as the use of ML/AI begins to proliferate and the data structure, the feature engineering, the model hyperparameters and the validation, accuracy and uncertainty of the models are now a fundamental requirement alongside drilling, assays and geological validity.

So what questions should be asked? At Neo Geo Consulting Ltd, we provide the bridge between traditional due diligence and machine learning. Drawing on our unique experience we have developed a technical framework to help deliver clarity at the interface between ML/AI and the mining industry.

Key due diligence questions for machine learning in mineral exploration

How do I assess machine learning performance?

To state the obvious, if the model doesn’t perform, it isn’t worth anything. However, these tools are created by bright people, and there is a risk of throwing the baby out with the bathwater if you do not understand why the model isn’t performing.

Key areas where models can underperform lie in the input data and processing, the model selection, tuning and validation, and whether the model can meet regulatory compliance. Issues can arise in all sorts of places: poor data management and data integrity, feature engineering that does not align to the conceptual deposit model, or unbalanced, low quality training data. All this will fundamentally undermine the model - it is the old adage of ”garbage in, garbage out”.

The model selection is critical, too, choosing the wrong model can be like turning up to a cricket match with a tennis racket. Of course, if you cannot justify the use of the model, its outputs and how it fits into wider compliance regimes such as JORC and NI-43101 guidelines, then there is a world of trouble ahead when it comes to reporting, sign-off from a CP/QP and, ultimately, attracting investment.

For instance, we have previously reviewed a geological mapping project using various classification algorithms for classification. The project compared Random Forest, Support Vector Machine and Artificial Neural Network models comparing their accuracy, and crucially how uncertainty was distributed across the map area. Despite good agreement in Overall Accuracy metrics, the Random Forest was superior in its Precision and Recall (accuracy metrics relating to False Positives and False Negatives) and reduced the spatial uncertainty across the final map.

What is the value of a machine learning approach?

Perhaps of high importance are the factors around how to extract value from the ML/AI tool that is being assessed. There may be a genuine ”manual vs automated“ question to ask about whether this adds value, or how a “traditional vs machine learning approach” can give marginal improvements in accuracy but add unnecessary complexity to the model.

There are often some great tools and products available that leverage AI but they may have issues with privacy, or perhaps the plan is to scale the product but it has not been written with this in mind.

Crucial questions here surround the deployment and production risk of the tool in question. Machine learning can be conducted in a simple Python script or demonstrated in a Jupyter Notebook. However, if these have been designed and implement by a geologist “locally” on their desktop computer, it is unlikely to scale well.

Likewise, if it is built on a server with a technology stack that incorporates APIs, MCPs and more general security protocols, this has its own issues to consider. The value of this can be difficult to assess, and likely comes with a substantial degree of context to why the tool is being assessed in the first place. However, its integration into the business, or its architecture for use externally will be a key factor in how value can be realised and meet any return on investment.

How resilient is the machine learning approach being used?

The technology and tools in the ML/AI space is moving so quickly that redundancy can be a real risk to the value of a tool, workflow or product. It is important to understand this, through good documentation, being active in developing staff capabilities and also retaining the talent that builds the system.

One of the greatest risks to any business is losing people and despite the shift in technology and the potential for automation in the era of ML/AI, it still holds that companies can’t exist without people. Understanding the team capability and knowledge, the transferability of knowledge through a business and the inherent requirements of maintaining the codebase for a particular tool are essential in understanding the risk of investing in a tool or a business leaning heavily on ML/AI.

Furthermore, there is a retrospective cost to consider - known as technical debt - where it may be necessary to understand the cost of fixing problems created earlier in development. These may have been conscious or unconscious choices and links back to understanding the model architecture and ultimate end-goal. Often the goal changed with time, or the plan is to invest to repurpose the tool or codebase for something else. Sometimes this has been created inadvertently going for speed of deployment over product quality.

Small individual issues may not seem like a huge problem, but over time, this technical debt can stack up and cause substantial problems and may eventually require a complete rebuild of the codebase which can be expensive and cause delays.

The AI:CORE advantage

The above highlights the fundamentally different approach needed for due diligence involving machine learning, especially in cross-disciplinary areas where a geologist may have written some machine learning code, or a data scientist has applied an algorithm without understanding the geological phenomena being modelled.

Neo Geo Consulting Ltd positions itself to complement the traditional due diligence approach by providing domain expertise across both mining and machine learning and artificial intelligence through independent technical assessment of ML/AI methodologies.

Our AI:CORE and AI:CORE+ offerings provide detailed assessment of the risks involved through data audits, model reviews, and considering the context of geology or mining. This integrated approach provides a unique and balanced assessment of novel applications that provide an appropriate level of scrutiny to bold claims about revolutionary technology that will change the industry or assess the relevance and applicability of ML/AI use to enhance performance within a conventional mining company.

We provide a standardised report assessing 8 critical topics as part of the AI:CORE due diligence offering, or build on our basic offering through a bespoke assessment with AI:CORE+ that can include deep-dive on the codebase, model verification, and other specific needs for investors.

Contact Neo Geo Consulting Ltd for a consultation and find out how AI:CORE can help protect investment decisions with proper ML/AI due diligence.