Message flow between MQ and Datastage

Message flow between MQ and Datastage

INTRODUCTION

This document guides to establish connection between Datastage and Websphere MQ.
We will explain here step by step, how to configure Datastage and WebSphere MQ,
where in Datastage reads and writes message to/from a MQ queue. We use MQ
Connector stage (Datastage 8.5) for this purpose.

Prerequisites

We expect the following products installed and configured.

1. · IBM Websphere MQ V6 and above.
2. · Datastage server 8.5

If Websphere MQ and Datastage server are not installed in the same physical

machine, the server which has Datastage server must have MQ client installed.

Note: In the examples below, it assumed that the Websphere MQ and Datastage Server
are installed on different machines (nodes).

Content Task overview

•  Create MQ messages using IBM Websphere MQ
•  Websphere MQ connector Stage
•  Reading the MQ Message
•  Writing the MQ Messages

Create MQ messages

To create MQ messages, we require Websphere MQ explorer or similar tool.

After the Websphere MQ explorer got opened expand the queue manager’s folder

Provide the Queue manager name and click the next button till the below screen appears.

 
Give the “Listen on port number”. Give a new port number (should not be used by other
queues).

 
In the left pane of the Websphere MQ Explorer right click on folder
Queues -> New -> Local Queue

 
In the below screen give the Local Queue name and click Finish button.

A row entry with the Local queue name given is appeared in the right pane. Right click
on the row and click “Put Test Message…”

Give the data in the Message data and press “Put message” button for each entry.

To check the data in the Queue right click on the desired queue in the right pane and
press “Browse Message…”.

Websphere MQ connector Stage

The Websphere MQ connector Stage in Datastage read messages from and writes
messages to a Websphere MQ enterprise messaging system.
A queue manager manages one or more message queues. The MQ Connector stage
establishes a connection to a queue manager in order to read messages from or write
messages to a queue. To connect to the queue manager there are two types of mode.

Server mode: The queue manager resides on the same machine as the MQ Connector
stage.

Client mode: The queue manager resides on a remote machine Specify channel name,
transport type (TCP), and remote connection name or IP address.

Installation requirements must be met depending upon whether you use the Websphere
MQ Connector in server connection mode or in client connection mode.

This mode is determined by your selection in the Mode property.

Client connection mode

• The Websphere MQ client must be installed on the same node as the connector.
• The Websphere MQ server must be installed on a node in the same network in which the connector is installed. 
• The Websphere MQ server can be installed on the same node as the connector.
•  There must be a network connection between the client node and the server node.

Server connection mode

• The Websphere MQ server must be installed on the same node as the
• connector.

Reading the MQ Messages

Steps to read the data from the MQ to the sequential file in Datastage:
Create a new parallel job:
Draw the Websphere MQ Connector stage in the parallel job. This stage exists in the Real
Time section of the pallet box

Draw the output connector from the Websphere MQ Connector to a sequential file.

In the properties section of the Websphere MQ connector stage provide the following
details.

Mode: Client
Queue Manager: This name exists in the left pane of the Websphere MQ Explorer.
Username: Username for the server.
Password: Password for the server.
Channel name: Default Channel name
Transport Type: TCP
Connection name: Server IP (Listener port number given in the MQ explorer for the
Queue here 1416)
Queue Name: This name is found in the right pane of the MQ Explorer when the Queue
folder of the queue manager is clicked.

Click on the Columns Tab and enter the fields as below. Fields defined without specified
data elements are treated as payload fields.

If we want to display the MessageID as a column select WSMQ.MSGID from the Data
Element list.

After running the job, view the data of the sequential file. The test data given in the MQ

explorer appears. The fetched data is removed from the queue.

Writing the MQ Messages

Create new Local Queue (IBMTestQueue2) in the Queue manager (IBMTest)

In the Datastage parallel job, create an output link from a sequential file stage to a
Websphere MQ connector stage.

Provide some sample data in the sequential file.

In the properties table of the Websphere MQ connector (target) give the Queue manager
name, queue name and other information as below screenshot and press ok.

After running the job the data in the sequential file flows to MQ. To check the data in the
queue right click on the queue and press the “Browse message…” button.