Target node/queue unavailable in R12

Today, w.r.t Concurrent Managers and some Other Managers, status field was showing message "Target node/queue unavailable" and Node Field was appearing Blank. This was our Production Instance and not any Cloned Instance.

After Checking Several Metalink Note ID's, I decided to follow below steps,

OAM Generic Collection Service shows State: "The target node/queue unavailable". [ID 393706.1] (it's given for 11i, but still worked for me in R12)

After Cloning all the Concurrent Managers do not start for the cloned Instance [ID 555081.1]

Conflict Resolution Manager Shows Target Node/Queue Unavailable [ID 732709.1]

Concurrent Managers Do Not Start After Cloning Nodes Not Updated In Conc_queues [ID 466532.1]

Summary of Possible Reasons and Solutions for the Problem Where All Concurrent Requests Stuck in Pending Phase [ID 182154.1]

Output Post Processor is Down with Actual Process is 0 And Target Process is 1 [ID 858813.1]

Solution which worked for me

SQL> set lines 200

SQL> column CONTROL_CODE format A15

SQL> select CONCURRENT_QUEUE_NAME, CONTROL_CODE , TARGET_NODE, NODE_NAME   from FND_CONCURRENT_QUEUES where concurrent_queue_name like 'OAMGCS_%';

CONCURRENT_QUEUE_NAME          CONTROL_CODE    TARGET_NODE

------------------------------ --------------- ------------------------------

NODE_NAME

------------------------------

OAMGCS_CBOS                    E

CBOS

The standard codes used by Oracle Applications are as followings:

'A', -> 'Activate concurrent manager'
'D', -> 'Deactivate concurrent manager'
'E', -> 'Deactivated'
'N', -> 'Target node/queue unavailable'
'R', -> 'Restart concurrent manager'
'T', -> 'Terminate requests and deactivate manager'
'U', -> 'Update concurrent manager env inf.'
'V', -> 'Verify concurrent managers status'
'X', -> 'Terminated'

+++++++++++++++++++++++++++

STATUS_CODE Column:

A Waiting

B Resuming

C Normal

D Cancelled

E Error

F Scheduled

G Warning

H On Hold

I Normal

M No Manager

Q Standby

R Normal

S Suspended

T Terminating

U Disabled

W Paused

X Terminated

Z Waiting

PHASE_CODE column

C Completed

I Inactive

P Pending

R Running

SQL> update FND_CONCURRENT_QUEUES set control_code = null where concurrent_queue_name = 'OAMGCS_CBOS';

1 row updated.

SQL> update FND_CONCURRENT_QUEUES set TARGET_NODE='CBOS' where CONCURRENT_QUEUE_NAME='OAMGCS_CBOS';

1 row updated.

SQL> commit;

Commit complete.

SQL> set lines 200

SQL> col CONTROL_CODE format A15

SQL> select CONCURRENT_QUEUE_NAME, CONTROL_CODE , TARGET_NODE, NODE_NAME   from FND_CONCURRENT_QUEUES where concurrent_queue_name like 'OAMGCS_%';

CONCURRENT_QUEUE_NAME          CONTROL_CODE    TARGET_NODE                    NODE_NAME

------------------------------ --------------- ------------------------------ ------------------------------

OAMGCS_CBOS                                    CBOS                           CBOS

Stop and start Concurrent Managers using adcmctl.sh

The above Solution worked for me and Concurrent Managers came up without any issue.

################ Other Solutions (I've not tried) ######################

Solution A:

CMCLEAN.SQL - Non Destructive Script to Clean Concurrent Manager Tables [ID 134007.1]

Run the cmclean.sql on admin node  

Start the managers and re-test.

Solution B:

sqlplus apps/pwd 
1.       SQL> EXEC FND_CONC_CLONE.SETUP_CLEAN;
          SQL>COMMIT;

                SQL> EXIT;

Make sure following two sqls returns no rows

SQL> select node_name "Node Name", node_mode "Mode", support_cp "C",support_web "W", support_admin "A", support_forms "F" from FND_NODES;

SQL>  select * from FND_OAM_CONTEXT_FILES;

2. Run AutoConfig on the database tier.

3. Run AutoConfig on the apps tier.

          

4. Run cmclean.sql script .

SQL> sqlplus APPS/<Password>

SQL> select node_name "Node Name", node_mode "Mode", support_cp "C",support_web "W", support_admin "A", support_forms "F" from FND_NODES;

SQL> select CONCURRENT_QUEUE_NAME from FND_CONCURRENT_QUEUES where CONCURRENT_QUEUE_NAME like 'FNDSM%';

5. Start all application services and check whether managers are up & re-test.

Solution C:  

Apply this solution in last when the above one doesn’t work.

SQL> select node_name,target_node,control_code from fnd_concurrent_queues;

SQL> update apps.fnd_concurrent_queues set node_name = 'Node NAME' where node_name='Existing Node Name';

SQL> select NODE_NAME,NODE_MODE,STATUS from fnd_nodes;

SQL> select control_code,target_node,node_name,CONCURRENT_QUEUE_NAME from fnd_concurrent_queues;

SQL> UPDATE fnd_concurrent_queues set control_code = null;

SQL> select TARGET_NODE,NODE_NAME from fnd_concurrent_queues where node_name='<Existing Node Name>';

SQL> select TARGET_NODE,NODE_NAME from fnd_concurrent_queues where TARGET_NODE='<Existing Node Name>';

SQL> update fnd_concurrent_queues set NODE_NAME='<Node Name>' where NODE_NAME='<Source/Existing Node Name>';

SQL> update fnd_concurrent_queues set TARGET_NODE='<Node Name>' where TARGET_NODE='<Source/Exixting Node Name>';

SQL> UPDATE fnd_concurrent_queues set target_node = '<Node Name>';

SQL> UPDATE fnd_concurrent_queues set node_name = '<Node Name>';

SQL> Commit;

SQL> select control_code,target_node,node_name,CONCURRENT_QUEUE_NAME from fnd_concurrent_queues;

SQL>select TARGET_NODE,NODE_NAME from fnd_concurrent_queues where node_name='<Node Name>';

SQL>select TARGET_NODE,NODE_NAME from fnd_concurrent_queues where TARGET_NODE='<Node Name>';

Thanks,

Manish