CICS Screen-Scraping Persists (And It’s Still a Bad Idea)
How To | Strategy| by Jerry Rackley

The original inspiration for founding HostBridge and launching our integration software was to eliminate CICS screen-scraping. Over 20 years ago, the first HostBridge customer licensed our software specifically because it did not rely on screen-scraping. From the beginning, we warned enterprises of the perils and limitations of screen-scraping as an integration technique. Harping on a point for two decades can fool us into believing that everyone understands the problems of scraping screens to get data. We continue to find that enterprises still allow vendors to sell them integration solutions that scrape screens. Often the screen-scraping is embedded in solutions, such as Robotic Process Automation (RPA), where its presence isn’t obvious.

The level of inquiries the HostBridge team routinely gets from prospective customers and consultants indicate screen-scraping is still pervasive. These inquirers are looking for solutions to integrate or modernize terminal-based CICS applications. These are the most difficult CICS applications to integrate, because they output data to a screen. It’s easy to assume that scraping data from application screens is the only or best option, but neither is true. HostBridge pioneered the approach of integrating CICS applications written for human operators via an API or as callable web services with no screen-scraping (see U.S. Patent no. 6,981,257).

A recent dialogue with a customer provides an example. They are using an integration solution that leverages the Extended High-Level Language Application Programming Interface (EHLLAPI). This led them to conclude there was no screen-scraping in the mix. However, APIs like HLLAPI or EHLLAPI still scrape data from the 3270 data stream. Don’t be fooled! These terms describe what sounds like more advanced means of getting mainframe data. But if you look under the covers, you find these integration solutions and techniques process a TN3270 data stream. In other words, they doing screen-scraping.

What’s Wrong with Screen Scraping?

The problem with screen-scraping is, to get data from a mainframe application, the software must first understand the screen geometry. Using row and column coordinates, a screen scraper navigates to that screen location and “scrapes” what it finds there. There are a few problems with this approach:

  • It’s brittle. Any integration solution that predicates its success on the 3270 data stream will at some point, break. The well-meaning mainframe programmer that makes any change to the application that changes the screen layout by as little as one row or column, usually breaks the integration.
  • It performs poorly. Most integration solutions that scrape screens run in the middle-tier. The very fact that the integration layer exists outside of the mainframe introduces a lot of latency, negatively affecting response times.
  • It doesn’t scale well. Because of the latency issue, mainframe integrations that scrape screens usually can’t scale up to accommodate high volume transaction levels at acceptable levels of performance.

Enterprises with requirements for high-performing, scalable, flexible, standards-based CICS integrations reach the limits of screen-scraping quickly. The limitations of CICS screen-scraping drove the creation of HostBridge over 20 years ago, and continue to influence its evolution.

A Better Approach

HostBridge brought JavaScript to the mainframe to enable the integration and orchestration of CICS applications via an API or as callable web services. The HostBridge JavaScript Engine (HB.js) does not screen scrape because it runs on the mainframe, within CICS. Because of this, it is able to interact with terminal-oriented CICS applications without resorting to parsing 3270 data streams. Instead, HB.js interacts with CICS transactions on the basis of field name and value pairs. It does this by leveraging the 3270 Bridge Interface that IBM added to CICS Transaction Server 1.3. This style of interaction exploits the architecture; it is how CICS applications are designed to operate.

Because HB.js interacts with a CICS application in this way, a 3270 data stream is never generated. No screens are ever scraped, and the resulting integration is extremely robust, insensitive to and unaffected by changes in the CICS application. This approach makes it very simple for any mobile, cloud, or web application to get CICS data via an API or as a callable web service.

HB.js integration scripts run within CICS, which introduces virtually no latency into the transaction. Some might have concerns that running the integration script inside CICS increases mainframe cycles. In this era where enterprises are actively looking for ways to reduce MIPS consumption, they may be disinclined to investigate this approach. However, HB.js is zIIP-enabled, so the keepers of the mainframe can get the benefits of the approach without increasing overhead.

Ditching Screen-Scraping is Easy

Even for enterprises that are heavy users of middle-tier screen-scraping solutions, it’s easy to switch to this better approach. The productivity and economic benefits make a compelling case to switch to HB.js for CICS integration:

It’s very easy to find out how well HB.js can satisfy your CICS integration requirements. We provide free pilot software, help you install it, and we’ll even help you develop and test your initial integration scripts. Talk with us to find out how to start a HostBridge pilot. Just reach out to us using he contact information found at the bottom of this page.

I agree to receive commercial messages from Broadcom. I understand my personal data is processed according to Broadcom's Privacy Policy and I may unsubscribe from emailed communications at any time.

Categories