Issue with NodeManager and Weblogic Server

Error :

<Jan 24, 2012 11:33:43 AM AST> <Error> <NodeManager> <BEA-300033> <Could not execute command "getVersion" on the node manager. Reason: "Access to domain 'Base_Domain' for user 'EZRXyZIP1P' denied".>

Debug Steps:

The default log level for node manager is : Info .
We need to set the log level to Finest.

Below are the available log level for Node Manager :

SEVERE             ----- (highest value)
WARNING
INFO                    ---- Default value .
CONFIG
FINE
FINER
FINEST             ----- (lowest value)

Then We will restart the Node Manager and check for the log file error message again.

Solution :

1 : Log on to the Admin Console of the domain for which we are getting the error ( http://hostname:port/console )

2 : Go to Domain Name (Base_Domain) in my case >>>>> Then to security >>>>

3 : Click on the Advance Option.

4 : Change the username and password values that you want to use with the node manager.

in my case : username is : weblogic

password is : weblogic123

5 : save the changes.

Then each remote host:

• Navigate to the folder %DOMAINHOME%\config\nodemanager


• Edit the file: nm_password.properties


• Content should be set to:


• username=weblogic


• password=weblogic123


• Save


• Restart the Weblogic Node Manager

6 : Then go to the Base_Domain/servers/mannaged_server/data/nodemanager/ dir.

7 : open the boot.properties file and enter the below values :

username=weblogic

password= weblogic123

the same values as set on Admin Console.

8: Restart the Admin Server.

9 : Now run the nmEnroll() wlst command for all the machines that has the Managed server.

wlst>connect('username','password','t3://admin_host:admin_port')

online>>nmEnroll('Domain_dir_path','NodeManager_Home_Path')

online>>exit()

Now you can check the node manager status from Admin Console .

left panel >> Machines >> Machine1 >> Monitoring >>

The Status should be reachable.

If the above steps did not resolve the issue the you can still post you issue in the comments below and I can try to help .

 

 

 Error :

Feb 2, 2012 2:55:02 AM weblogic.nodemanager.server.NMServer main SEVERE: Fatal error in node manager server java.net.BindException: Address already in use at java.net.PlainSocketImpl.socketBind(Native Method) at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:365) at java.net.ServerSocket.bind(ServerSocket.java:319) at java.net.ServerSocket.<init>(ServerSocket.java:185) at java.net.ServerSocket.<init>(ServerSocket.java:141) at weblogic.nodemanager.server.Listener.init(Listener.java:56) at weblogic.nodemanager.server.NMServer.start(NMServer.java:206) at weblogic.nodemanager.server.NMServer.main(NMServer.java:377) at weblogic.NodeManager.main(NodeManager.java:31)

Reason :

The error message : java.net.BindException: Address already in use at

Shows that the Port number used by the Node Manager process is being already  used by some other process .

Solution :

1 : First thing we need to check is whether Node Manager is already running on the machine :

For unix :

ps -ef | grep -v grep | grep -i weblogic.NodeManager

For windows :

netstat -a0 | findstr <NM_PORT_NO>

tasklist | findstr nodemanager

2 : If the process is not running already then, check which process is using the using the port assigned to node manager .

By default Weblogic Node Manager runs on port : 5556

So, to check which process is using this port use the below command .

For unix :

netstat -an | grep 5556

For Windows :

netstat -ao | findstr 5556

3 : If this port is already used by some other process then try to change the Node Manager Port number from file

%Middleware_Home%/wlserver_10.3/common/nodemanager/nodemanager.properties file :

ListenPort = XXXX

4 : Now try to start the Node manager from :

%Middleware_Home%/wlserver_10.3/server/bin/startNodemanager.sh or

%Middleware_Home%/wlserver_10.3/server/bin/startNodemanager.cmd

 

 Error :

NodeManager not Reachable: java.io.IOException: Invalid State File Format

Due to above error, Managed Servers cannot be started as the nodemanager is not reachable to Admin Console.

Stack trace from nodemanager.log file :

    

java.io.IOException: Invalid state file format. State file contents: at weblogic.nodemanager.common.StateInfo.load(StateInfo.java:135) at weblogic.nodemanager.server.ServerMonitor.loadStateInfo(ServerMonitor.java:475) at weblogic.nodemanager.server.ServerMonitor.isCleanupAfterCrashNeeded(ServerMonitor.java:139) at weblogic.nodemanager.server.ServerManager.recoverServer(ServerManager.java:255) at weblogic.nodemanager.server.DomainManager.initialize(DomainManager.java:103) at weblogic.nodemanager.server.DomainManager.(DomainManager.java:55) at weblogic.nodemanager.server.NMServer.getDomainManager(NMServer.java:257) at weblogic.nodemanager.server.Handler.handleDomain(Handler.java:224) at weblogic.nodemanager.server.Handler.handleCommand(Handler.java:108) at weblogic.nodemanager.server.Handler.run(Handler.java:70) at java.lang.Thread.run(Thread.java:619) 

 

 

 Reason :

The state file of the managed server is in an invalid state.
Under each managed server directory, there is a NodeManager directory containing a state file <managed_server_name>.state. If this file is empty or corrupt, then the described errors occur.

For example, underE:\oracle\middleware\user_projects\domains\base_domain\servers\soa_server1\data\nodemanager, the soa_server1.state file is empty.
The major cause of the corruption of this file is unexpected shutdown of the Machine.

Solution :

1. Stop the managed server if it is RUNNING.


2. Stop the Admin Server.


3. Stop Node Manager.


4. Delete the following files:

E:\oracle\middleware\user_projects\domains\base_domain\servers\soa_server1\data\nodemanager\soa_server1.state
E:\oracle\middleware\user_projects\domains\base_domain\servers\soa_server1\data\nodemanager\soa_server1.lck
E:\oracle\middleware\user_projects\domains\base_domain\servers\soa_server1\data\nodemanager\soa_server1.pid

1. Start Node Manager again.


2. Start the Admin Server again.


3. Start the managed server using the Admin Console.