- 1 console tools
- 2 web tools
- 3 IPMIView
- 4 iKVM
- 5 Virtual Media
- 6 changing password – IPMICFG
- 7 changing password – bmc-config
- 8 resetting an IPMI password
- 9 resetting the BMC
- 10 Accessing health data
Some of our machines have IPMI.
The older Supermicro machines just have IPMI 1.1 and it’s pretty limited and not that usable.
The new (2010) Supermicro machines have IPMI 2.0 and you can do a lot with it.
Install freeipmi-tools. There are a number of different ipmi tools, each of them implements slightly different things, we’ve been using freeipmi-tools, and they seem to work reasonably well.
All tools support some common flags:
-h or --hostname hostname or IP of the IPMI processor -u or --username username (ADMIN is the IPMI default) -p or --password password (ADMIN is the IPMI default) -P prompt for password
remote power: ipmipower
To get an interactive prompt you can run various commands from:
/usr/sbin/ipmipower --hostname=foo-mp --username=ADMIN -P
or you can run commands directly:
To query the status
/usr/sbin/ipmipower --hostname=foo-mp --username=ADMIN -P --stat
To turn the machine off and then on
/usr/sbin/ipmipower --hostname=foo-mp --username=ADMIN -P --off sleep 10 /usr/sbin/ipmipower --hostname=foo-mp --username=ADMIN -P --on
etc., see the man page for more options.
remote serial console: ipmiconsole
/usr/sbin/ipmiconsole --hostname=foo-mp --username=ADMIN -P
The escape sequence for exiting is
During the bios splash screen, you might need to hit a function key to get at things like the BIOS, or the BBS popup menu. I could get those to work by hitting esc-#, rather than the function keys.
point your web browser at the IPMI management processor’s hostname/IP.
If the IPMI is on a private network you don’t have direct access to, you can use an ssh tunnel to access it. Run something like
ssh -L4443:ipmihost:443 jumphost.example.com
Then in your browser go to https://localhost:4443/
You can do a few things with just a browser, but for the remote KVM and some other features you must have Sun java installed, currently free java alternatives aren’t enough to make it work.
Here’s what used to work in the squeeze and older days:
sun-java6-bin sun-java6-jre sun-java6-plugin
Currently attempting to use icedtea6-plugin or icedtea7-plugin doesn’t work and you get this error
net.sourceforge.jnlp.LaunchException: Fatal: Initialization Error: Could not initialize application. at net.sourceforge.jnlp.Launcher.createApplication(Launcher.java:778) at net.sourceforge.jnlp.Launcher.launchApplication(Launcher.java:552) at net.sourceforge.jnlp.Launcher$TgThread.run(Launcher.java:889) Caused by: net.sourceforge.jnlp.LaunchException: Fatal: Application Error: Cannot grant permissions to unsigned jars. Application requested security permissions, but jars are not signed. at net.sourceforge.jnlp.runtime.JNLPClassLoader.setSecurity(JNLPClassLoader.java:312) at net.sourceforge.jnlp.runtime.JNLPClassLoader.<init>(JNLPClassLoader.java:232) at net.sourceforge.jnlp.runtime.JNLPClassLoader.getInstance(JNLPClassLoader.java:357) at net.sourceforge.jnlp.runtime.JNLPClassLoader.getInstance(JNLPClassLoader.java:330) at net.sourceforge.jnlp.Launcher.createApplication(Launcher.java:770)
Searching with google finds lots of people having problems getting the web based KVM working, it seems that only particular combinations of browser/OS/java version work. That path leads to madness which is why most people recommend using….
Supermicro provides an application called IPMIView, you can download it here. It is also java but appears to work, even on Linux. I’m sure it’s very nice, but it doesn’t allow setting the ports, so that means we need to be running it somewhere with direct network access (either locally or have some sort of VPN to the BMCs). Using an ssh tunnel won’t work since 623 is a privileged port (which would require root to forward it, we don’t want to do that.
Anyway it looks nice but we can’t use it easily….
Searching around I found this page complaining about how older versions of IPMIView tried to install their own java on the system and how to run the iKVM java app by hand. I noticed that the ports are set on the command line! So we do this:
- On the IPMI web interface, go to the configuration→network settings, and change the RMCP
- In one window start the tunnels for the KVM
$ ssh -L2623:bnc:623 -L5900:bnc:5900 jumphost.example.com
- Then from the directory you unpacked the IPMIView application in run
$ java -Djava.library.path=. -jar iKVM.jar 127.0.0.1 ADMIN ADMIN null 5900 2623 2 0
Where “ADMIN ADMIN” are your ipmi login/password. This connects to the localhost side of the tunnel using ports 5900 and 2623, which then connect to the BNC ports 5900 and 623.
The virtual media is limited to a 1.44MB floppy image and using a Windows Share to provide an ISO image.
changing password – IPMICFG¶
Supermicro provides a command line tool for configuring the IPMI stuff. You can download it here
Check and make sure there is only the ADMIN user and that’s it’s id 2 and then change the password.
root@motmot-pn:~# /tmp/IPMICFG-Linux.x86_64 -user list Maximum number of Users : 10 Count of currently enabled Users : 2 User ID | User Name | Privilege Level | Enable ------- | --------- | --------------- | ------ 2 | ADMIN | Administrator | Yes root@motmot-pn:~# /tmp/IPMICFG-Linux.x86_64 -user setpwd 2 <passwd>
The PDF that comes with the tool describes how to use it to do almost everything the web interface can do. (including sensors stuff)
changing password – bmc-config¶
To change the password of the IPMI device, you first check out its config to a file:
/usr/sbin/bmc-config -h foo-mp -u ADMIN -P --checkout -n foo-ipmi
Now edit the file foo-ipmi with your favorite editor. I found this to be very problematic because the version of the file that is spit out cannot be loaded back in. You have to make a number of adjustments to the file before you can:
For User1, make sure:
- Enable_User No
- Lan_Privilege_Limit No_Access
- SOL_Payload_Access No
- Lan_Enable_IPMI_Msgs Yes
- Lan_Enable_Link_Auth No
- Lan_Privilege_Limit Administrator
- SOL_Payload_Access Yes
On all the remaining User blocks, make sure:
- Enable_User No
- Lan_Privilege_Limit No_Access
- SOL_Payload_Access No
In Section SOL_Conf make sure:
- Enable_SOL Yes
- SOL_Privilege_Level Administrator
- Force_SOL_Payload_Encryption Yes
- Character_Accumulate_Interval 5
- Character_Send_Threshold 50
- SOL_Retry_Count 5
- SOL_Retry_Interval 10
WARNING: I’ve only included the values above that you need to make sure are set properly, the rest will be fine. I found that many of these were wrong and if you do not set them, you will fail to load the config. For example, the Character_Accumulate_Interval is written as ‘0’ but when you try to load the config it says it is wrong. I found the values from an example config provided by the freeipmi project.
WARNING: with ipmi2 you are supposed to be able to set a 20 character password in the Password20 field. I did that, and loaded the config, and it no longer accepted any password. I believe that you need to use the Password field (not Password20) to set the password. See “Resetting the IPMI password” below for information about how to reset things back to the factory default if you manage to screw it up like I did.
Load that changed file back to the IPMI:
/usr/sbin/bmc-config -h foo-mp -u ADMIN -P --commit --filename=foo-ipmi
resetting an IPMI password¶
To reset the IPMI password, you need to be on the machine itself that has the IPMI card in it (ie. not from the network).
I am sure there is a way to do this using the packaged tools, but the way I tried first, which succeeded was to do this:
# wget ftp://ftp.supermicro.com/utility/IPMICFG/Linux/IPMICFG-Linux_v1.20.zip # unzip # ./ipmicfg-linux.x86.static -fd;./ipmicfg-linux.x86.static -I open lan set 1 password NEWPASSWORD
The ‘set 1’ above is saying that channel 1 is the ethernet interface, on yours it might not be ‘1’, you can find out by doing: ‘ipmitool -I open channel info 1’ which should show something like: Channel Medium Type : 802.3 LAN if it shows ‘Serial/Modem’, try channel 2
resetting the BMC¶
Sometimes the BMC can hang. You can reset it from the host system using ipmitool (ipmitool package). There are some bugs (#365896, #506934,LP#110992, LP#908112) with the IPMI drivers not automatically loading, so you need to load them first.
modprobe ipmi_devintf modprobe ipmi_si # might need to apt-get install ipmitool ipmitool bmc reset cold
Accessing health data¶
You can get some basic health information by using one of the two commands:
$ ipmi-sensors -h ip.of.ipmi.device -u ADMIN -P 4: System Temp (Temperature): 31.00 C (-7.00/77.00): [OK] 71: CPU Temp (OEM Reserved): [OEM State = 0000h] 138: FAN 1 (Fan): NA (NA/NA): [Unknown] 205: FAN 2 (Fan): 5025.00 RPM (585.00/29815.00): [OK] 272: FAN 3 (Fan): 5210.00 RPM (585.00/29815.00): [OK] 339: FAN 4 (Fan): 5025.00 RPM (585.00/29815.00): [OK] 406: FAN 5 (Fan): NA (NA/NA): [Unknown] 473: CPU Vcore (Voltage): 0.95 V (0.66/1.41): [OK] 540: +3.3VCC (Voltage): 3.28 V (2.88/3.65): [OK] 607: +12 V (Voltage): 12.35 V (10.60/13.20): [OK] 674: CPU DIMM (Voltage): 1.52 V (1.22/1.78): [OK] 741: +5 V (Voltage): 4.96 V (4.32/5.60): [OK] 808: -12 V (Voltage): -12.29 V (-13.65/-11.51): [OK] 875: VBAT (Voltage): 3.15 V (2.88/3.65): [OK] 942: +3.3VSB (Voltage): 3.26 V (2.88/3.65): [OK] 1009: AVCC (Voltage): 3.28 V (2.88/3.65): [OK] 1076: Chassis Intru (Physical Security): [OK] 1143: PS Status (Power Supply): [Presence detected][Unrecognized State][Unrecognized State][Unrecognized State][Unrecognized State][Unrecognized State][Unrecognized State][Unrecognized State]
The ipmimonitoring command pulls out the sensor data and instead of reporting them, interprets them to indicate if they are NOMINAL, WARNING or CRITICAL states:
$ ipmimonitoring -h ip.of.ipmi.device -u ADMIN -P Record_ID | Sensor Name | Sensor Group | Monitoring Status| Sensor Units | Sensor Reading 4 | System Temp | Temperature | Nominal | C | 31.000000 205 | FAN 2 | Fan | Nominal | RPM | 5025.000000 272 | FAN 3 | Fan | Nominal | RPM | 5210.000000 339 | FAN 4 | Fan | Nominal | RPM | 4840.000000 473 | CPU Vcore | Voltage | Nominal | V | 0.944000 540 | +3.3VCC | Voltage | Nominal | V | 3.280000 607 | +12 V | Voltage | Nominal | V | 12.349000 674 | CPU DIMM | Voltage | Nominal | V | 1.520000 741 | +5 V | Voltage | Nominal | V | 4.960000 808 | -12 V | Voltage | Nominal | V | -12.292000 875 | VBAT | Voltage | Nominal | V | 3.152000 942 | +3.3VSB | Voltage | Nominal | V | 3.264000 1009 | AVCC | Voltage | Nominal | V | 3.280000 1076 | Chassis Intru | Physical Security | Nominal | N/A | 'OK' 1143 | PS Status | Power Supply | Nominal | N/A | 'Presence detected' 'Unrecognized State' 'Unrecognized State' 'Unrecognized State' 'Unrecognized State' 'Unrecognized State' 'Unrecognized State' 'Unrecognized State'