I was recently a guest during a YouTube Live broadcast with Anders Jensen to discuss UiPath mainframe automation. Anders has an excellent UiPath YouTube channel, and he is also a 2021 UiPath MVP. During the broadcast, I had the opportunity to demonstrate two methods of automating mainframe CICS applications using UiPath, which we documented in our Guide to UiPath Mainframe Integration:
- Via a Terminal Sessions object in UiPath.
- Via a RESTful service created with the HostBridge JavaScript Engine (HB.js) and invoked via HTTP.
Shortly after Anders posted our video to his channel, a UiPath VP posted a series of comments. These comments provide an excellent basis for further discussion on this topic of mainframe automation. I’ll address these comments in the balance of this post.
Comment: One should never do a measurement running in debug mode as a baseline.
During our demo, we ran in Debug mode so viewers could see what the bot was doing. We agree that Debug mode can bias the timings to a degree. However, that effect is minimal when using the Terminal Sessions (TS) object. When we “Run” that same bot/file (not in Debug mode) the time taken goes down to about 47 seconds: a 2 second reduction. Here’s why the difference is so small (and why we didn’t think it unfair to run the bot in Debug mode).
When using the Terminal Sessions object, by default it imposes delays before/after each mainframe interaction. This is understandable since the object is emulating a terminal and operating on content in a screen buffer. For example, when the object transmits data to the mainframe, it automatically imposes a 1 second delay before the data is sent (per the DelayMS attribute, below):
The WaitType attribute infers an additional delay, although the length of the delay is at the discretion of the TS object. Essentially, it’s waiting for the content of the screen buffer to “settle”.
We can easily determine the minimum total delay imposed by the Terminal Session object by summing all the DelayMS values in the XAML file:
The sum of all DelayMS attributes (explicit delays) is the MINUMUM delay the Terminal Sessions object imposes. It does not include the implicit delays due to the WaitType=READY attribute on each of the 50 object calls. I estimate each imposes a delay of about 250 MS: a reasonable amount of time for an emulator to wait for all data to be received from the mainframe app, and for the screen buffer to “settle”. Thus, 50 x 250 MS = 12.5 seconds. Combining the explicit waits (29 seconds) and the implicit waits (12.5 seconds) yields 41.5 seconds of total estimated wait time due to interaction with the mainframe via the Terminal Sessions object.
When we Run the bot (not in Debug mode) it takes 47 seconds. It’s a fair estimate that the remaining 5.5 seconds come from either:
- UiPath (apart from the Terminal Sessions object)
- Network latency
- Mainframe processing time.
Regarding network latency, that particular bot resulted in 21 interactions with the mainframe. Given the network we were using for the demo, we estimate total latency of no more than 2 seconds. Total mainframe response time accounted for about 1 second. We are then left with UiPath, apart for the TS object, accounting for about 2.5 seconds. Nice!
In summary, we can state the following:
- Debug mode is 2 seconds slower for that particular bot, but the delays due to Debug mode are inconsequential
- When using the Terminal Sessions object to interact with the mainframe, significant delays are imposed due to the nature of using emulation and screen scraping as the means of interaction
- Apart from the Terminal Sessions object (and all other external factors), UiPath is fast — which is why we really like it!
Comment: One should run the workflow unattended on the server connected right next to the Mainframe then you have minimal latency.
In an ideal world, we would agree! However, this is not how enterprise IT infrastructure is typically configured these days. The first customer who approached us 2 years ago for help optimizing their automations illustrates the current reality. Their mainframe is outsourced to a managed services provider whose data center is in Dallas. Their servers are outsourced to a different firm that maintains the hardware in Chicago. The customer doesn’t have control over where the hardware is located. All they can control is the speed of the network between them (in their case, it’s 1 gbps). As we see more movement to the hybrid cloud, we’ll continue to see this phenomenon of mainframes, servers, and users in different geographic locations. This is the norm in today’s hybrid IT world.
Considering the significant delays using emulation and screen scraping as the means of integration imposes, network latency is not the most significant factor (although all those discrete requests/responses really do clog up a corporate network). Running the HB.js demo using a PC on the same LAN segment as our mainframe would not change the outcome, because network latency is not imposing the most significant delay.
Comment: When you run a script on the mainframe to run the data then you may as well run the script via RPA.
Here we disagree, and physics provides the basis of our difference. We demonstrated a UiPath mainframe automation invoking a HB.js script running on the mainframe. That script interacted with the exact same mainframe transactions as did the Terminal Sessions object. However, since the script was running on the mainframe, the interactions between the script as the CICS transactions occurred at machine (memory-to-memory) speed. To illustrate the benefit, our script executed/interacted with the exact same transactions in about 275 MS – TOTAL! Running the automation on the mainframe (and avoiding all the discrete requests/responses) exploits this speed. While it’s obvious that network latency for each request/response is eliminated, there are other significant benefits.
When interacting with screen-oriented apps via terminal emulation and screen scraping, you must impose delays as the Terminal Session object does. Why? Because there is no general indicator in the 3270 data stream signaling that the mainframe app has finished sending output and is waiting for input. As a result, the Terminal Session object necessarily imposes delays to ensure that an orderly conversation occurs. We are not critical of the Terminal Session object for imposing these delays. This is simply the nature of performing application integration via emulation and screen scraping
By contrast, the HB.js script runs on the mainframe within CICS. It therefore has a much different relationship to application, with these benefits:
- The 3270 data stream is eliminated as the means of integration. CICS BMS apps don’t “think” in terms of 3270 data streams. Rather, they “think” in terms of field name/value pairs. And that is how a HB.js script interacts with a CICS BMS app: exchanging field name/value pairs. This doesn’t require ANY change to the app; it’s just the way they work. But you can only exploit this fact using code, like HB.js, that “talks” to the app in its “native tongue”.The implications of eliminating the 3270 data stream are significant. For example, what happens if the location and size of various fields change (controlled by the BMS map not the app)? When you use emulation and screen scraping, the integration breaks. This leads to what we call “application rigor mortis”: a state where the organization is afraid to change/evolve the mainframe apps for fear of what will break! However, when you use HB.js to interact with the exact same mainframe app, the integration continues to work. Why? Because the HB.js script doesn’t care one bit about the formatting of data on the screen.It’s no secret that application integration via emulation and screen scraping is brittle. Even the slightest change to the screen layout of the mainframe application usually breaks the integration. The RESTful service we demonstrated completely avoids the use of terminal emulation and screen scraping. This results in faster UiPath bots and more reliable automations.
- The interaction between the HB.js script and the mainframe app are stateful. Specifically, we know the exact nanosecond when the app has finished sending output and is ready to receive input. And, since there is absolutely NO ambiguity about this matter, a HB.js script does NOT have to impose delays to second guess the intentions of the app. The response time and performance implications are huge. The Terminal Sessions object imposes over 40 seconds of delay simply to ensure that it maintains an orderly conversation with the mainframe app. And, in its defense, it had no other choice. But we do – because the HB.js script runs in the same context as the mainframe app. And at that level, things are VERY deterministic! That’s why we can execute the exact same number of transactions in a fraction of the time (275 MS).
Comment: Because of the legacy application maintenance issue, RPA remains the fastest and reliable way to automate and capture all the data.
Full text of the comment: There are so many companies running legacy mainframe applications they can’t maintain because the people with the knowledge who did that have already retired. But since everyone still knows to operate these, RPA remains the fastest and reliable way to automate it and capture all the data right away into the UiPath Data Service or any other Cloud Database.
We agree and have seen firsthand the skills drain. However, it does not follow that an RPA platform must/should interact with legacy mainframe apps using terminal emulation and screen scraping. The RESTful services approach we demonstrated doesn’t require the integration developer to know anything more about the legacy application. Stated differently, the Terminal Sessions and RESTful services approach both live with the same assumption: the ability to operate the application is all that is required to create the automation. Either way, the integration developer simply needs to understand how to navigate the application.
Delegating the detailed mainframe interactions to HB.js is clearly faster and more resilient. We would argue that it’s actually easier to develop the automation than using the Terminal Sessions object. Why? Because the integration developer doesn’t have to specify all the details as to where fields are located on the screen.
Conclusion: RESTful services outperform Terminal Sessions for mainframe automation
At HostBridge, we are RPA advocates and have no argument with UiPath or other RPA vendors. Enterprises should embrace RPA and use it to automate business processes, including those that run on the mainframe. However, using terminal emulation and screen scraping as the means of bot interaction with the mainframe is a step backward. At scale, it’s an approach that is slow, expensive, and fragile. Even if the mainframe apps never change, using emulation and screen scraping as the means of integration is slow and costly.
Our demonstration is a very small example of what occurs on a large scale within some enterprises. Using our analytics tools, we recently discovered a bot that ran for 5 hours on each of 4 servers. Each instance of the bot orchestrated 225,000 CICS transactions for a total of 1 million transactions. The underlying transactions only took 25 minutes to execute. But because the bot orchestrated these interactions via terminal emulation and screen scraping, each instance of the bot took 4.5 hours longer to complete than necessary because of the delays we demonstrated. Using terminal emulation and screen scraping at this scale makes no sense whatsoever. As a result, the organization is actively working to remediate this automation. With this background, let me respond to your comments.
When an organization wants to use RPA to interact with screen-based applications at scale, the default method (Terminal Sessions) is slower, less efficient, and more expensive. The RESTful services approach we demonstrated is highly complementary to UiPath, performs better, uses mainframe cycles more efficiently, and lowers mainframe costs. RESTful services are easy to create using HB.js and easy to invoke via UiPath.
To learn more about using RESTful services with your automations, please reach out to us using the contact information found at the bottom of this page.