This blog was co-authored by Emile Veliyev, Director, OSDU Delivery, EPAM.
The energy industry creates and consumes large amounts of highly complex data for key business decisions, like where to drill the next series of wells, how to optimize production, and where to lease the next big field. Despite good intentions, the industry is still plagued by large quantities of data that are inconsistent in location, quality, and format—much of which cannot reliably be found or used when needed. Even when the data is reliable, it can be locked into application-specific data stores that limit its use. The solution to this dilemma is multi-faceted and increasingly includes cloud technology, the OSDU™ Data Platform, modern applications, and data governance focused on people and their business processes.
Microsoft Energy Data Services is a data platform fully supported by Microsoft, that enables efficient data management, standardization, liberation, and consumption in energy exploration. The solution is a hyperscale data ecosystem that leverages the capabilities of the OSDU Data Platform, Microsoft’s secure and trustworthy cloud services with our partners’ extensive domain expertise.
Cloud and the OSDU Data Platform
Cloud-based computing is the future—scalable, reliable, secure storage and compute capabilities, all managed for you with many powerful add-on capabilities at your fingertips. For the energy industry, the Open Group® OSDU Data Platform is rapidly emerging as the standard—an open source, cloud-based data platform that unlocks data from applications and provides standard data schemas and access protocols, enabling both data governance and rapid innovation.
One of the things that EPAM discovered when delivering app developer boot camps and deploying the platform for ourselves and for clients is its high level of complexity. In those earlier days, platform deployment was a multi-step process, with each service being deployed and validated separately, taking up to a week. Before we could move on to solving business problems, a part of our work was to guide our clients through various technical deployment obstacles. In addition, it took another several days to ingest pre-formatted sample data in order to test the platform with real data. Not anymore.
Microsoft Energy Data Services
Microsoft has made the OSDU Data Platform enterprise-ready and pre-bundled with the capabilities needed to optimize Energy Company data value using the Microsoft Cloud. EPAM has seen its benefits. As an enterprise-grade platform, Microsoft Energy Data Services has nearly single-click deployment. Deployment time has reduced significantly—what previously took multiple days now takes about 45 minutes! Similarly, the time to ingest the sample data is drastically reduced from one week to around one hour! In addition, the management layer surrounding the platform provides the assured reliability, stability, security, tools, performance, and the SLAs needed by large enterprises such as major energy companies.
Data governance and modern applications
As noted before, excellent infrastructure alone does not magically solve all data and business problems. With Microsoft Energy Data Services providing a solid foundation with which to store data, process data, and build and host cloud-native apps aligned with the OSDU Technical Standard, what remains to empower a data-driven organization is modern applications and data governance.
It is a daunting task to manually track the manifold ways that data enters the company, the many places it is stored, and the many ways it is consumed, enriched, and duplicated. Improving this requires a team who can map out the detailed way in which all of this happens today. It also takes modern digital tools to automate the aggregation, parsing, quality assessment, and lineage-tracking of the data. It takes people with a broad and deep view to accomplish this for large organizations—people who understand the business, the data types, the technology, and how to provide the right data, in the right formats, in the right place, at the right time, to the right people. That includes application connectors and analytical applications themselves designed for the modern cloud environment so that liberated data can move back and forth to users seamlessly.
How to work with EPAM on Microsoft Energy Data Services
EPAM brings industry knowledge, technical expertise, tools, frameworks, relationships with software vendors, and world-class delivery built on the Microsoft Energy Data Services platform. EPAM has developed a document extraction and processing system (DEPS) accelerator, which provides capabilities to facilitate the development of customizable workflows for extracting and processing unstructured data in the form of scanned or digitalized document formats. DEPS is powered by Azure AI/machine learning and deep learning algorithms. It includes pluggable sub-systems for customization, uses machine learning pre/post processors, validation and extensions for UI review, automation machine learning models training, manual labeling, and analytics capabilities to improve classic optical character recognition (OCR) and text extraction accuracy. DEPS can be adapted to process numerous data types covering both image and text, PDF, XLS, ASCII, and other file formats. Ask more at OSDU@epam.com.
Microsoft Energy Data Services is an enterprise-grade, fully-managed, OSDU Data Platform for the energy industry that is efficient, standardized, easy to deploy, and scalable for data management—for ingesting, aggregating, storing, searching, and retrieving data. The platform will provide scale, security, privacy, and compliance expected by our enterprise customers. EPAM offers services providing the right data, in the right formats, in the right place, at the right time, to the right people, which includes application connectors and analytical applications, with data contained in Microsoft Energy Data Services.