Amazon Web Services Feed
Performant Access to Your Geoscience Data in the Cloud Using OSDU and IVAAP
By Steven Reynolds, Sr. Project Manager at INT
By Dhruv Vashisth, Sr. Partner Solutions Architect at AWS
By Dmitriy Tishechkin, Partner Technical Lead at AWS
 |
 |
 |
Many oil & gas operators now want—and need—near-instant access to their geoscience data. Unfortunately, most subsurface applications provide only a limited, fragmented view of data, with manual and time-consuming analysis, search, and visualization processes in place.
Considering the time it takes to move data and people, the most efficient way to optimize these processes is to allow remote access. The Open Subsurface Data Universe (OSDU) platform is expressly designed to allow users to be able to find the correct data, fast.
INT’s data visualization platform IVAAP uses OSDU on Amazon Web Services (AWS) to enable users who want to optimize this and other workflows. The IVAAP platform offers a single point of access to search, interact with, analyze, and visualize multiple petrotechnical data sources in a single web-based application.
In this post, we will show how to use IVAAP and OSDU on AWS to access geoscience data remotely and how these two systems use REST APIs to work together seamlessly.
Interactive Network Technologies (INT) is an AWS Advanced Technology Partner. INT software empowers the world’s largest oil & gas (IOC/NOC) and service companies to visualize their complex subsurface data.
INT’s powerful HTML5/JavaScript technology can be used for data aggregation, API services, and high-performance visualization of G&G and petrophysical data in the cloud.
Overview: Uniform Access to Data Remotely
To optimize data access, you first need to offer uniform access remotely. The combined system of IVAAP and OSDU provides two levels of REST APIs—both give uniform access to the data regardless of its location. Because the IVAAP client runs in your browser, you can access your data anywhere.
Figure 1 – IVAAP with connections to various data sources, including OSDU on AWS.
In addition to OSDU, the IVAAP backend offers many data connectors such as WITSML, OSIsoft PI, and databases like Peloton WellView, ProSource, and PPDM. This gives users many rich options for accessing their data remotely through IVAAP.
Explore and Visualize Data with IVAAP APIs
INT developed IVAAP as a powerful, extensible platform to enable users to access, interact with, and view domain data easily on the web. This means faster collaboration and better decision-making across teams, no matter the location.
To optimize this process, the IVAAP REST APIs are specific to each of the geoscience data types in use. For example, there are REST APIs that will get:
- Information about a well.
- Wellbores in a well.
- Logs in a wellbore.
- Curves in a log.
- Traces from a seismic dataset.
These APIs are tuned carefully to allow the IVAAP client to get just the information needed to build up displays without getting too much data. As a user, you can open these displays using nothing more than your browser and connect to your data wherever it is.
IVAAP pulls data from OSDU only when that specific data is needed. The end result is a nimble, responsive display that allows you to explore and visualize your data.
Search, Store, and Deliver with OSDU APIs
The OSDU data platform has three APIs used to access data: Search, Storage, and Delivery. These three form the strong foundation to build applications that can access data and for a data platform that can scale.
Robust Search
The Search API allows users to perform powerful searches to find their data. The most basic and essential search workflows are:
- Search for well data, seismic, horizons, or faults.
- Search for data based on user criteria.
In IVAAP, the search results are displayed conveniently on a map.
You can search for any data type in the platform and use any of the metadata in its schema as criteria. The Search API is implemented using Amazon ElasticSearch Service, which ensures the entry point to the OSDU data platform will scale to the amount of data used by the largest companies.
In this example, IVAAP uses the Search API to find some wells.
Figure 2 – Example of IVAAP using the OSDU Search API to find data.
To find well logs which have a GR curve, the OSDU Search endpoint is queried.
Storage
The OSDU Storage API is used to create the seismic metadata display in IVAAP. This API gives access to the metadata records for all the data in the platform.
One typical use for this is to enrich the metadata displays of data items. This storage API is a GET request to a URL like this example: <hostname>/records/opendes:doc:41d06fdfadbb45c286b183f838034898
Fast Delivery
The OSDU Delivery API gives read access to the source geoscience data files on Amazon Simple Storage Service (Amazon S3). In the case of well log data, the Delivery API gives the S3 URL of an ASCII (LAS) file. The IVAAP backend gets the LAS file and parses it to extract the curve data.
The resulting display looks like this:
Figure 3 – Example of IVAAP using the OSDU Delivery API to access and display a LAS file.
This call to the Delivery API returns access to the LAS files. At this point, the IVAAP backend pulls down and parses the LAS files.
For seismic data, the IVAAP backend uses various techniques to directly access the byte segments that are needed without needing to pull down the whole file.
- For SEG-Y files, IVAAP uses HTTP methods to access the file parts as they are needed.
- For OpenVDS, the VDS library streams the parts that are needed. OpenVDS is a set of tools and an open-source API from Bluware to read and write seismic data in Volume Data Store (VDS) format.
The IVAAP client pulls from the system only the specific seismic traces which are needed to paint the on-screen display pixels.
IVAAP, OSDU, and AWS: A Powerful Partnership
Together, IVAAP, OSDU, and AWS are a powerful combination for users to be able to quickly access, visualize, and display their data from anywhere. But how does it all work together?
In a typical example, a user runs the IVAAP frontend in their browser and accesses data which can be far away in the AWS Cloud near to the OSDU data platform.
Figure 4 – Detailed interactions of IVAAP and OSDU.
The user displays a map, searches for the data they want to see, and displays wells on the map. The user then creates a project, opens a well, creates a well log display, and drags a curve into the display.
Given this scenario, the interactions between IVAAP and OSDU are detailed below. The key REST APIs are shown for both the IVAAP backend and the OSDU platform.
Action: User searches for the data they want to see
- Key IVAAP APIs: IVAAP Search – ivaap/api/ds/opensdus3/v1/sources/9bf985c3-752d-4a1a-a149-3c670cd7a62e/search
- OSDU APIs: OSDU Search
Action: User opens a well in the left IVAAP data pane
- Key IVAAP APIs: IVAAP Wells – ivaap/api/ds/opensdus3/v1/sources/9bf985c3-752d-4a1a-a149-3c670cd7a62e/wells/srn%3Amaster-data%2FWell%3A7010%3A
- OSDU APIs: Search to get more info about the well and its children. OSDU Delivery to get the Amazon S3 credentials for the well and its children. OSDU Storage to get the well manifest. All children are prefetched.
. - Key IVAAP APIs: IVAAP Wellbores – ivaap/api/ds/opensdus3/v1/sources/9bf985c3-752d-4a1a-a149-3c670cd7a62e/wells/srn%3Amaster-data%2FWell%3A7010%3A/wellbores
- OSDU APIs: Uses cached data in the IVAAP backend.
. - Key IVAAP APIs: IVAAP Logs – ivaap/api/ds/opensdus3/v1/sources/9bf985c3-752d-4a1a-a149-3c670cd7a62e/wellbores/srn%3Amaster-data%2FWellbore%3A7010%3A/logs
- OSDU APIs: Uses cached data in the IVAAP backend.
Action: User clicks on the well log
- Key IVAAP APIs: IVAAP Well Log – ivaap/api/ds/opensdus3/v1/sources/9bf985c3-752d-4a1a-a149-3c670cd7a62e/logs/srn%3Awork-product-component%2FWellLog%3A61097454226119089765%3A
- OSDU APIs: Uses cached data in the IVAAP backend.
Action: User drags a curve into a track
- KeyIVAAP APIs: Curves – ivaap/api/ds/opensdus3/v1/sources/9bf985c3-752d-4a1a-a149-3c670cd7a62e/logs/srn%3Awork-product-component%2FWellLog%3A61097454226119089765%3A/curves?curveids=1&minindex=0.2&maxindex=3000&indextype=depth
- OSDU APIs: Uses Amazon S3 to get the LAS file.
All of this can be done in a single application, in a web browser. IVAAP’s partnership with OSDU and AWS makes it easy to work with data in one application that could previously only be accessed individually in specialized applications.
Conclusion
IVAAP is built to empower product owners, developers, and architects to accelerate the development of subsurface digital solutions for oil & gas. It offers robust capabilities to access, aggregate, search, and view geoscience data from anywhere in a single browser.
INT’s partnership with AWS and OSDU means INT can extend and scale those capabilities through a range of APIs.
For more information on IVAAP, visit int.com/ivaap, or contact INT at intinfo@int.com.
.
.
INT – AWS Partner Spotlight
INT is an AWS Advanced Technology Partner that empowers oil and gas companies to visualize their complex data. INT’s data visualization platform IVAAP uses OSDU on AWS to enable users who want to optimize this and other workflows.
Contact INT | Practice Overview
*Already worked with INT? Rate the Partner
*To review an AWS Partner, you must be a customer that has worked with them directly on a project.