Real Time Data Replication

[Legacy User]

Hi everyone,

Let me describe the current situation first.

Extractions are scheduled to run from servers (Windows or LINUX) to get data from our IDMS databases. Our zOS is at 1.9 (soon to be 1.11), IDMS is at 16.0 going to 18 (not soon enough). They use ODBC or JDBC, it depends.

Those extractions are scheduled daily, weekly, or monthly. One of them is scheduled every 10 minutes (low volume, no problem).

I was asked if it was possible to reverse the process, so to speak, and have IDMS send updates to servers instead of scheduling extractions to get the latest data available.

That means every update to an IDMS record would be sent over or reflected in real time. At first, only a few record types would be included but then the process would later include a lot more.

Of course, the first thought is to have a database procedure do this, but then performance considerations explode in one's mind.

Is there anybody out there who has attempted to do this and how did you proceed ?

Thanks,
Claude Ferland .../

[John Abell]

Good morning Claude,

You may remember our old DARS product that you used when in Ottawa many years ago. DARS has evolved into DARSTRAN and now supports the capture and transformation of CA IDMS updates from the Active Journals. Some important points on this:

1. We have no Hooks, Exits, DBPROCs, in the CA IDMS CV, DARSTRAN runs as a separate started task and reads the CV Journals independently. In fact, you can run DARSTRAN on a separate LPAR as long as the Active and Archive Journals can be seen from that LPAR.
2. DARSTRAN transforms the CA IDMS updates to the appropriate SQL for the selected Target RDBMS(s). A single DARSTRAN instance can generate SQL for multiple target RDBMS.
3. We support AIX, HP-UX (both PA-RISC and Itanium), Linux (Wintel, zLinux, others as needed), Solaris (SPARC and Wintel) and Windows as target platforms.
4. We support DB2 (all platforms including z/OS), IDMS/SQL,MS SQLServer, MySQL, Oracle and Sybase as the major RDBMSs. Others on request.
5. You have the option of maintaining the DBKEY values in the target RDBMS for auditing and navigation or using a DARSTRAN-generated and maintained Unique Key value.
6. DARSTRAN comes as a complete Replication Suite for CA IDMS and there are no additional extra cost components.

I can provide many references for your target platforms. For our largest user, DARSTRAN generates and applies to the target RDBMS 60 million+ SQL statements per day from the Active CV Journals.

If you would contact me offline, I would be pleased to provide you with manuals and more information. ==> [e-mail]. 1-800-295-7608 Ext: 224

Yours truly,
John

[Legacy User]

John,

It's been a while. It's all coming back to me now.

I will contact you later.

Thanks.

[Anon Anon]

Do you have IBM's MQSeries on your mainframe? We use MQSeries to update Windows and UNIX servers application databases real time. We also pull certain updated records from the journals every 15 minutes automatically with a batch process to update a SAP UNIX servers application databases. The 1st solution uses OCA MQSeries for IDMS while the 2nd is in house written.

HTH,
Steve Harmeson

[Anon Anon]

Steve - you may have missed something? How do you get the data that is being replicated from IDMS on to the MQ Messaging Service?

Cheers - Gary

[Anon Anon]

Hi Gary,

I will clarify my earlier message. The OCA-MQSeries for IDMS runs as a subtask/listener in IDMS and connects to the mainframe MQSeries queue manager. That in turn can talk to any distributed platform (Windows, UNIX, AIX, etc..) running MQSeries using messages and queues that can read or update application databases and past info back to the mainframe.

Thanks,
Steve

[Anon Anon]

Now we're getting closer. The question was about "replication" - MQ provides a transport mechanism - what is the software or application or technique that creates the[b] "replication" data stream that is transported across MQ messaging services?

Thanks for your patience with my obtuseness - cheers - Gary

[Anon Anon]

Hi Gary

would ISP's DARS product not be good enough?

Steve Cannon (CA UK)

[Anon Anon]

Yes Steve - that's one option - there are others. The purpose of my questions to this part of the thread is to find out what is being used to put the replication data stream onto the MQ Messaging service - as[b] I am not aware of MQ being able to actually provide a replication service as such - it is just a transport mechanism!

Cheers - Gary

[Anon Anon]

Gary is correct on MQ.

John T. Abell

[Anon Anon]

Isn't there commercially available software, widely used around the world, that does exactly what's wanted here?

Jim Philips

[Gary Cherlet]

We use the International Software Product and it works great. We have used it for years. They also have very good support with a very reasonable price tag compared to others.

Tonya Hunley
Sr. DBA
Salliemae, Inc.

[Gary Cherlet]

Good to hear and to know.

To which vendors in particular are you referring to by "others"?

Thanks.

Groet, Jaap Houtman

[Gary Cherlet]

Jaap - should you just hang out a [color=#0e0ebf]"Gone Phishing"[color] sign? :tongue

I am no longer on the IUA Board nor am I President of Australian IUA - so as an "ordinary member" of the IDMS_L list server and of the IUA I wonder if I could ask the vendors to drop out of this thread - it's getting just a little too "cute" for my personal taste. I think that the names of your companies have been flashed across my screen enough for both of our purposes.

TIA - cheers - Gary Cherlet

[Anon Anon]

Informatica as an example.

Tonya Hunley

[Anon Anon]

Dear Tonya,

Thank you for replying to my previous message and sharing this information with me.

Best regards,
Jaap Houtman

[Anon Anon]

Dear Tonya,

Thank you for replying to my previous message and sharing this information with me.

Best regards,
Jaap Houtman

[Anon Anon]

Steve,

We do not have MQ series.

Can you give me an idea of how much work is involved to implement something like your in-house solution?

We don't have a huge number of updates, so perhaps that would be a good fit for us.

Thanks,
Claude Ferland.../

[Gary Cherlet]

Dear Claude,

It may be something you do not quite expect, but there is a software vendor in Europe that provides data integration in real-time including mainframe legacy systems like IDMS.

The technology includes:

* Bi-directional data exchange in real-time based upon changed data
* Changed Data Capture support for DB2, ADABAS, IMS/DB, VSAM, IDMS/DB, DATACOM/DB, DB2/ LUW, Oracle, SQL-Server and more
* Automatic Data Conversion
* Easy Data Mapping
* Integrated Technology for automatic Apply into target database
* Comprehensive Monitoring and Reporting provides widespread transparency for all exchanged data

The product is robust, stable, dependable, performance is something that goes without saying and affordable, in particular for companies with a limited amount of data to be replicated and as such does do not justify the development of an in-house solution.

The product does not require any additional expensive third party software that requires to be licensed, such as IBM's MQSeries.

The installation and implementation in a pre-production environment can be accomplished within one week.

I will send you a product brochure to your personal e-mail address that contains more detailed information.

Best regards,
Jaap Houtman

[Legacy User]

Hi Japp,

I have replied to you offline.

I thank all those who have contributed so far, directly to me or through this forum.. I now have 3 vendors who have communicated with me on this.

From my own research, it is obvious that an in-house solution would require resources we do not have.

At this point, we will consider our options with the products available on the market.

Thank you everybody,
Claude Ferland.../

[Gary Cherlet]

We've used ObjEx with MQ-Series for several years and it's been very reliable. I'd suggest you look at their website or contact me directly for more information.

Lonna Albertson
[email=[e-mail]]Albertson, Lonna J. [email]