op-test User Guide¶

Command Line Options¶

All configuration options can be specified via the command line or in a configuration file (see Configuration Files and Command Line Options).

usage: op-test [-h] [-c FILE] [--list-suites] [--list-tests]
               [--list-all-tests] [--run-suite RUN_SUITE] [--run RUN] [-f]
               [--quiet]
               [--machine-state {UNKNOWN,UNKNOWN_BAD,OFF,PETITBOOT,PETITBOOT_SHELL,OS}]
               [-o OUTPUT] [-l LOGDIR] [--suffix SUFFIX]
               [--hostlocker HOST_NAME | --aes ENV_NAME|Q|L|U [ENV_NAME|Q|L|U ...]]
               [--hostlocker-user HOSTLOCKER_USER]
               [--hostlocker-server HOSTLOCKER_SERVER]
               [--hostlocker-base-url HOSTLOCKER_BASE_URL]
               [--hostlocker-locktime HOSTLOCKER_LOCKTIME]
               [--hostlocker-keep-lock HOSTLOCKER_KEEP_LOCK]
               [--hostlocker-proxy HOSTLOCKER_PROXY]
               [--hostlocker-no-proxy-ips HOSTLOCKER_NO_PROXY_IPS]
               [--aes-search-args AES_SEARCH_ARGS [AES_SEARCH_ARGS ...]]
               [--aes-user AES_USER] [--aes-server AES_SERVER]
               [--aes-base-url AES_BASE_URL]
               [--aes-rel-on-expire AES_REL_ON_EXPIRE]
               [--aes-keep-lock AES_KEEP_LOCK] [--locker-wait LOCKER_WAIT]
               [--aes-add-locktime AES_ADD_LOCKTIME] [--aes-proxy AES_PROXY]
               [--aes-no-proxy-ips AES_NO_PROXY_IPS]
               [--bmc-type {AMI,SMC,FSP,FSP_PHYP,OpenBMC,qemu,mambo}]
               [--bmc-ip BMC_IP] [--bmc-mac BMC_MAC]
               [--bmc-username BMC_USERNAME] [--bmc-password BMC_PASSWORD]
               [--bmc-usernameipmi BMC_USERNAMEIPMI]
               [--bmc-passwordipmi BMC_PASSWORDIPMI] [--bmc-prompt BMC_PROMPT]
               [--smc-presshipmicmd SMC_PRESSHIPMICMD]
               [--qemu-binary QEMU_BINARY] [--mambo-binary MAMBO_BINARY]
               [--mambo-initial-run-script MAMBO_INITIAL_RUN_SCRIPT]
               [--mambo-autorun MAMBO_AUTORUN]
               [--mambo-timeout-factor MAMBO_TIMEOUT_FACTOR]
               [--host-ip HOST_IP] [--host-user HOST_USER]
               [--host-password HOST_PASSWORD]
               [--host-serial-console-command HOST_SERIAL_CONSOLE_COMMAND]
               [--host-lspci HOST_LSPCI]
               [--host-scratch-disk HOST_SCRATCH_DISK]
               [--qemu-scratch-disk QEMU_SCRATCH_DISK]
               [--host-prompt HOST_PROMPT] [--host-name HOST_NAME]
               [--host-gateway HOST_GATEWAY] [--host-submask HOST_SUBMASK]
               [--host-mac HOST_MAC] [--host-dns HOST_DNS] [--proxy PROXY]
               [--host-cmd HOST_CMD] [--host-cmd-file HOST_CMD_FILE]
               [--host-cmd-timeout HOST_CMD_TIMEOUT]
               [--host-cmd-resultpath HOST_CMD_RESULTPATH]
               [--platform {unknown,habanero,firestone,garrison,firenze,p9dsu,witherspoon,mihawk}]
               [--os-cdrom OS_CDROM] [--os-repo OS_REPO] [--no-os-reinstall]
               [--git-repo GIT_REPO] [--git-repoconfigpath GIT_REPOCONFIGPATH]
               [--git-repoconfig GIT_REPOCONFIG] [--git-branch GIT_BRANCH]
               [--git-home GIT_HOME] [--git-patch GIT_PATCH] [--use-kexec]
               [--append-kernel-cmdline APPEND_KERNEL_CMDLINE]
               [--bmc-image BMC_IMAGE] [--host-pnor HOST_PNOR]
               [--host-hpm HOST_HPM] [--host-img-url HOST_IMG_URL]
               [--flash-skiboot FLASH_SKIBOOT] [--flash-kernel FLASH_KERNEL]
               [--flash-initramfs FLASH_INITRAMFS]
               [--flash-part PART name bin file] [--noflash] [--only-flash]
               [--pflash PFLASH] [--pupdate PUPDATE] [--pdbg PDBG]
               [--un-signed-pnor UN_SIGNED_PNOR] [--signed-pnor SIGNED_PNOR]
               [--signed-to-pnor SIGNED_TO_PNOR]
               [--key-transition-pnor KEY_TRANSITION_PNOR]
               [--test-container PART name bin file] [--secure-mode]
               [--trusted-mode] [--add-kernel-args ADD_KERNEL_ARGS]
               [--remove-kernel-args REMOVE_KERNEL_ARGS]
               [--cronus-release CRONUS_RELEASE]
               [--cronus-product CRONUS_PRODUCT]
               [--cronus-system-type CRONUS_SYSTEM_TYPE]
               [--cronus-code-level CRONUS_CODE_LEVEL]
               [--cronus-hdct CRONUS_HDCT]
               [--cronus-dump-directory CRONUS_DUMP_DIRECTORY]
               [--cronus-dump-suffix CRONUS_DUMP_SUFFIX] [--cronus-smart-path]
               [--hmc-ip HMC_IP] [--hmc-username HMC_USERNAME]
               [--hmc-password HMC_PASSWORD] [--system-name SYSTEM_NAME]
               [--lpar-name LPAR_NAME] [--lpar-prof LPAR_PROF]
               [--lpar-vios LPAR_VIOS] [--check-ssh-keys]
               [--known-hosts-file KNOWN_HOSTS_FILE] [--accept-unknown-args]
               [--secvar-payload-url SECVAR_PAYLOAD_URL]

Named Arguments¶

-c, --config-file

Configuration File

--machine-state

Possible choices: UNKNOWN, UNKNOWN_BAD, OFF, PETITBOOT, PETITBOOT_SHELL, OS

Current machine state

-o, --output

Output directory for test reports. Can also be set via OP_TEST_OUTPUT env variable.

-l, --logdir

Output directory for log files. Can also be set via OP_TEST_LOGDIR env variable.

--suffix

Suffix to add to all reports. Default is current time.

--hostlocker

Hostlocker host name to checkout, see HOSTLOCKER GROUP below for more options

--aes

AES environment name to checkout or Q|L|U for query|lock|unlock of AES environment, refine by adding –aes-search-args, see AES GROUP below for more options

Test¶

Tests to run

--list-suites

List available suites to run

Default: False

--list-tests

List each test that would have been run

Default: False

--list-all-tests

List all defined tests

Default: False

--run-suite

Run a test suite(s)

--run

Run individual tests

-f, --failfast

Stop on first failure

Default: False

--quiet

Don’t splat lots of things to the console

Default: False

HOSTLOCKER GROUP¶

Options for HostLocker (see above optional arguments –hostlocker, mutually exclusive with –aes)

--hostlocker-user

UID login for HostLocker, uses OS UID if not specified, you must have logged in at least once via the web prior to running

--hostlocker-server

Override URL for HostLocker Server

--hostlocker-base-url

Override Base URL for HostLocker

--hostlocker-locktime

Time duration (see web for formats) to lock the host, never is the default, it will unlock post test

--hostlocker-keep-lock

Release the lock once the test finishes, defaults to False to always release the lock post test

Default: False

--hostlocker-proxy

socks5 proxy server setup, defaults to use localhost port 1080, you must have the SSH tunnel open during tests

--hostlocker-no-proxy-ips

Allows dynamic determination if you are on proxy network then no proxy will be used

AES GROUP¶

Options for AES (see above optional arguments –aes, mutually exclusive with –hostlocker)

--aes-search-args

AES allowable, match done by regex like –aes-search-args Environment_Name=wl2, run –aes Q for more info

--aes-user

UID login for AES, uses OS UID if not specified, you must have logged in at least once via the web prior to running

--aes-server

Override URL for AES Server

--aes-base-url

Override Base URL for AES

--aes-rel-on-expire

AES setting related to aes-add-locktime when making the initial reservation, defaults to True, does not affect already existing reservations

Default: True

--aes-keep-lock

Release the AES reservation once the test finishes, defaults to False to always release the reservation post test

Default: False

--locker-wait

Time in minutes to try for the lock, default does not retry

Default: 0

--aes-add-locktime

Time in hours (float value) of how long to reserve the environment, reservation defaults to never expire but will release the environment post test, if a reservation already exists for UID then extra time will be attempted to be added, this does NOT work on NEVER expiring reservations, be sure to add –aes-keep-lock or else the reservation will be given up after the test, use –aes L option to manage directly and –aes U option to manage directly without running a test

Default: 0

--aes-proxy

socks5 proxy server setup, defaults to use localhost port 1080, you must have the SSH tunnel open during tests

--aes-no-proxy-ips

Allows dynamic determination if you are on proxy network then no proxy will be used

BMC¶

Options for Service Processor

--bmc-type

Possible choices: AMI, SMC, FSP, FSP_PHYP, OpenBMC, qemu, mambo

Type of service processor

--bmc-ip

BMC address

--bmc-mac

BMC MAC address

--bmc-username

SSH username for BMC

--bmc-password

SSH password for BMC

--bmc-usernameipmi

IPMI username for BMC

--bmc-passwordipmi

IPMI password for BMC

--bmc-prompt

Prompt for BMC ssh session

Default: “#”

--smc-presshipmicmd

--qemu-binary

[QEMU Only] qemu simulator binary

Default: “qemu-system-ppc64”

--mambo-binary

[Mambo Only] mambo simulator binary, defaults to /opt/ibm/systemsim-p9/run/p9/power9

Default: “/opt/ibm/systemsim-p9/run/p9/power9”

--mambo-initial-run-script

[Mambo Only] mambo simulator initial run script, defaults to skiboot.tcl

Default: “skiboot.tcl”

--mambo-autorun

[Mambo Only] mambo autorun, defaults to ‘1’ to autorun

Default: “1”

--mambo-timeout-factor

[Mambo Only] factor to multiply all timeouts by, defaults to 2

Default: 2

Host¶

Installed OS information

--host-ip

Host address

--host-user

SSH username for Host

--host-password

SSH password for Host

--host-serial-console-command

Command to get serial console for host.Used instead of IPMI SoL. Useful for buggy BMCs.

--host-lspci

Known ‘lspci -n -m’ for host

--host-scratch-disk

A block device we can erase

Default: “”

--qemu-scratch-disk

A block device for qemu

--host-prompt

Prompt for Host SSH session

Default: “#”

--platform

Possible choices: unknown, habanero, firestone, garrison, firenze, p9dsu, witherspoon, mihawk

Platform (used for EnergyScale tests)

Host OS Install¶

Options for installing an OS on the Host

--host-name

Host name

Default: “localhost”

--host-gateway

Host Gateway

Default: “”

--host-submask

Host Subnet Mask

Default: “255.255.255.0”

--host-mac

Host Mac address (used by OS installer to set up OS on the host)

Default: “”

--host-dns

Host DNS Servers (used by OS installer to set up OS on the host)

Default: “”

--proxy

proxy for the Host to access the internet. Only needed for tests that install an OS

Default: “”

Host Run Commands¶

Options for Running custom commands on the Host

--host-cmd

Command to run

Default: “”

--host-cmd-file

Commands to run from file

Default: “”

--host-cmd-timeout

Timeout for command

Default: 1000

--host-cmd-resultpath

Result path from host

Default: “”

OS Images¶

OS Images to boot/install

--os-cdrom

OS CD/DVD install image

--os-repo

OS repo

Default: “”

--no-os-reinstall

If set, don’t run OS Install test

Default: False

git repo¶

Git repository details for upstream kernel install/boot

--git-repo

Kernel git repository

--git-repoconfigpath

Kernel config file to be used

--git-repoconfig

Kernel config to be used

Default: “ppc64le_defconfig”

--git-branch

git branch to be used

Default: “master”

--git-home

home path for git repository

Default: “/home/ci”

--git-patch

patch to be applied on top of the git repository

--use-kexec

Use kexec to boot to new kernel

Default: False

--append-kernel-cmdline

Append kernel commandline while booting with kexec

Images¶

Firmware LIDs/images to flash

--bmc-image

BMC image to flash(*.tar in OpenBMC, *.bin in SMC)

--host-pnor

PNOR image to flash

--host-hpm

HPM image to flash

--host-img-url

URL to Host Firmware image to flash on FSP systems (Must be URL accessible petitboot shell on the host)

--flash-skiboot

skiboot to use/flash. Depending on platform, may need to be xz compressed

--flash-kernel

petitboot zImage.epapr to use/flash.

--flash-initramfs

petitboot rootfs to use/flash. Not all platforms support this option

--flash-part

PNOR partition to flash, Ex: –flash-part OCC occ.bin

--noflash, --no-flash

Even if images are specified, don’t flash them

Default: False

--only-flash

Only flash, don’t run any tests (even if specified)

Default: False

--pflash

pflash to copy to BMC (if needed)

--pupdate

pupdate to flash PNOR for Supermicro systems

--pdbg

pdbg binary to be executed on BMC

STB¶

Secure and Trusted boot parameters

--un-signed-pnor

Unsigned or improperly signed PNOR

--signed-pnor

Properly signed PNOR image(imprint)

--signed-to-pnor

Properly signed PNOR image(imprint or production)

--key-transition-pnor

Key transition PNOR image

--test-container

PNOR partition container to flash, Ex: –test-container CAPP capp_unsigned.bin

--secure-mode

Secureboot mode

Default: False

--trusted-mode

Trustedboot mode

Default: False

Kernel cmdline options¶

add/remove kernel commandline arguments

--add-kernel-args

Kernel commandline option to be added

Default: “”

--remove-kernel-args

Kernel commandline option to be removed

Default: “”

Cronus¶

Cronus Config options

--cronus-release

Cronus Release

Default: “auto”

--cronus-product

Cronus Product

Default: “p9”

--cronus-system-type

Cronus System Type

Default: “witherspoon”

--cronus-code-level

Cronus Code Level

Default: “dev”

--cronus-hdct

Cronus Hardware Dump Content Table file

Default: “HDCT.txt”

--cronus-dump-directory

Cronus dump file directory

--cronus-dump-suffix

Cronus dump file suffix

Default: “optest”

--cronus-smart-path

Cronus path added after /usr/bin

Default: False

HMC¶

HMC CLI commands

--hmc-ip: HMC address
--hmc-username: SSH username for HMC
--hmc-password: SSH password for HMC
--system-name: Managed system/server name in HMC
--lpar-name: Lpar name as provided in HMC
--lpar-prof: Lpar profile provided in HMC
--lpar-vios: Lpar VIOS to boot before other LPARS

Misc¶

--check-ssh-keys

Check remote host keys when using SSH (auto-yes on new)

Default: False

--known-hosts-file

Specify a custom known_hosts file

--accept-unknown-args

Don’t exit if we find unknown command line arguments

Default: False

--secvar-payload-url

Specify a URL for the secvar test data payload

Configuration Files and Command Line Options¶

When working with op-test, you can specify every configuration option on the command line, or you can save some in a configuration file. Typically, you may keep a configuration file per-machine you test against, to save on typing out all the IP addresses and login credentials.

For example, this witherspoon.conf file will connect to a Witherspoon machine:

[op-test]
bmc_type=OpenBMC
bmc_ip=10.0.0.1
bmc_username=root
bmc_password=0penBmc
host_ip=10.0.0.2
host_user=root
host_password=abc123

It can be used as such:

./op-test --config-file witherspoon.conf

Other options can also be specified on the commandline, such as --host-pnor which is documented in Flashing Firmware with op-test.

There is also a per user configuration file, at ~/.op-test-framework.conf where you can store global options, for example:

[op-test]
smc_presshipmicmd=foo
pupdate=~/pupdate
pflash=~/pflash
ubuntu-cdrom=~/ubuntu-18.04-ppc64el.iso

This per user configuration file will be loaded in addition to the config file provided on the command line.

Flashing Firmware with op-test¶

In the future, there may be some standard interface for flashing firmware onto OpenPOWER machines. We are not yet in that future, so the method of flashing firmware varies somewhat between platforms.

There is code in op-test to hide these differences from you, and just provide easy ways to say “flash this firmware and run the test suite”. This functionality is primarily focused towards firmware developers.

By default, op-test will flash any firmware provided before running any tests. You can change this behaviour with the --no-flash or -only-flash` command line options (which should be fairly self-explanatory).

To flash a full PNOR image, you will need one in the correct format:

AMI BMC based systems (e.g. Palmetto, Habanero, Firestone and Garrison): A raw PNOR file (palmetto.pnor, habanero.pnor etc) as produced by op-build should be provided to the --host-pnor command line option. op-test will use this with pflash on the BMC to write the whole PNOR image to flash. It will NOT preserve anything from what was previously flashed (e.g. NVRAM, GUARD records). Since most AMI BMCs do not ship with pflash you will need to tell op-test where to find a pflash binary compiled for the BMC with the --pflash option
IBM FSP based systems (e.g. Tuleta, ZZ): The only current method that op-test supports for flashing a full FSP image is in-band, fetching the image over the network. You need to specify the URL to the full firmware image to the --host-image-url command line option.
OpenBMC based systems (e.g. Witherspoon, Romulus): There are two ways for OpenBMC systems to store PNOR, either using the full NOR chip for the PNOR, or using the vPNOR (Virtual PNOR) method. The normal PNOR method needs a .pnor file, e.g. romulus.pnor. The vPNOR method needs a .squashfs.tar file, e.g. witherspoon.pnor.squashfs.tar. If you provide the incorrect format, op-test will error out.

For BMC firmware, you also need the machine specific BMC firmware image format to supply to the --bmc-image command line option. For some BMCs, you may need an external utility. For example, Supermicro (SMC) BMCs need the pUpdate utility, which you can point op-test at with the --pupdate command line option.

OpenBMC/Witherspoon Examples¶

These examples are for the Witherspoon system, which uses OpenBMC as a BMC and the vPNOR implementation. For Romulus (which does not use vPNOR), instead of witherspoon.pnor.squashfs.tar you may need to use witherspoon.pnor.

FIXME Confirm details on Romulus.

For example, this command will use the witherspoon.conf configuration file (see Configuration Files and Command Line Options) for login credentials to a Witherspoon machine, and will flash host firmware before running the default test suite:

./op-test --config-file witherspoon.conf \
--host-pnor ~/op-build/output/images/witherspoon.pnor.squashfs.tar

In this example we’ve provided the full path to a witherspoon firmware image that we’ve built using op-build.

If you also want to flash BMC firmware, you can do that with the addition of the --bmc-image command line option:

./op-test --config-file witherspoon.conf \
--bmc-image obmc-phosphor-image-witherspoon.ubi.mtd.tar \
--host-pnor ~/op-build/output/images/witherspoon.pnor.squashfs.tar

In this example, op-test will first update the BMC firmware, then update the host firmware and then run the test suite.

If you’re a skiboot/OPAL developer and wanting to test your latest code when applied on top of a known-good BMC and PNOR image, you can use the --flash-skiboot command line option to instruct op-test to, as a final step, overwrite the PAYLOAD partition with your skiboot:

./op-test --config-file witherspoon.conf \
--bmc-image obmc-phosphor-image-witherspoon.ubi.mtd.tar \
--host-pnor ~/op-build/output/images/witherspoon.pnor.squashfs.tar \
--flash-skiboot ~/skiboot/skiboot.lid.xz.stb

In this case, if “field mode” is enabled on the BMC, op-test will disable it for you to allow for overriding host firmware with the skiboot image you aksed it to use.

Since the Witherspoon platform has Secure Boot enabled, you will need the .stb variant of skiboot (i.e. with the Secure and Trusted Boot header), and since we’re an OpenPOWER system, we need the .xz compressed version, and this is why we provide skiboot.lid.xz.stb to op-test for this system.

Note that with Secure Boot enabled, by default we only sign with imprint keys.

AMI BMC/POWER8 OpenPOWER sytems examples¶

For machines such as Palmetto, Habanero, Firestone and Garrison.

TODO Document BMC flashing.

These systems have an AMI BMC and op-test will use pflash on the BMC to write host firmware. You will need to point op-test towards a pflash binary compiled for the BMC for op-test to copy over and use to flash firmware.

TODO Document HPM flashing.

An example of flashing a full habanero.pnor image and running the default test suite is:

./op-test --config-file hab4.conf \
--host-pnor ~/op-build/output/images/habanero.pnor

Just like on other systems, if you’re an OPAL/skiboot developer and you want to test your changes along with a known-good full PNOR image, you’d do that the same way, using the --flash-skiboot parameter:

./op-test –config-file hab4.conf –host-pnor ~/op-build/output/images/habanero.pnor –flash-skiboot ~/skiboot/skiboot.lid.xz

We need to provide the skiboot.lid.xz file as all POWER8 OpenPOWER systems need the compressed payload in order to fit in flash. It is only very old Hostboots that do not support this and require the raw skiboot.lid.

IBM FSP System examples¶

For machines such as Tuleta and ZZ (firenze class).

Your FSP must have an NFS mount and be configured correctly for this operation.

Currently, flashing a full FSP image is only supported by doing it from the host. In future, we may support out of band methods.

The primary use op-test on Tuleta/ZZ is for flashing new OPAL LIDs onto an existing FSP image. Unlike OpenPOWER machines, the kernel and initramfs are split up into two separate LIDs, and must be pointed to separately.

This example will run the stest suite against our ZZ machine after flashing our skiboot, kernel and initramfs built fresh from op-build (with the configuration zz_defconfig).

./op-test --config-file zz.conf \
--flash-skiboot ~/op-build/output/images/skiboot.lid \
--flash-kernel ~/op-build/output/images/zImage.epapr \
--flash-initramfs ~/op-build/output/images/rootfs.cpio.xz

For FSP based systems, the uncompressed skiboot.lid is needed, as the FSP will load this image directly into memory and start executing it.

op-test and Qemu¶

You can use the ‘qemu’ BMC type to run many tests using the qemu simulator. This can be useful for test development/debug as well as testing the qemu simulator itself.

It may be useful to keep a configuration file with your qemu configuration in it for running tests. An example of such a configuration file is below:

[op-test]
bmc_type=qemu
qemu_binary=~/qemu/ppc64-softmmu/qemu-system-ppc64
host_pnor=palmetto.pnor
flash_skiboot=~/skiboot/skiboot.lid
flash_kernel=zImage.epapr
flash_initramfs=rootfs.cpio
host_user=ubuntu
host_password=abc123
ubuntu_cdrom=osimages/ubuntu-17.10-server-ppc64el.iso

Note that for qemu we want the uncompressed skiboot.lid for qemu to load, and while it’s not required, using the uncompressed rootfs.cpio does significantly improve boot time to Petitboot.

In this configuration file example, we point to a qemu development tree rather than using the system default qemu-system-ppc64 binary. We also specify a --host-pnor file so that qemu can mount an NVRAM partition.

To run the “boot to petitboot” test in qemu with the above configuration file, you can do so like this:

./op-test --config-file qemu.conf \
--run testcases.BasicIPL.BootToPetitbootShell

Not all tests currently pass in qemu, and running tests in qemu should be considered somewhat experimental.

op-test User Guide¶

Command Line Options¶

Named Arguments¶

Test¶

HOSTLOCKER GROUP¶

AES GROUP¶

BMC¶

Host¶

Host OS Install¶

Host Run Commands¶

OS Images¶

git repo¶

Images¶

STB¶

Kernel cmdline options¶

Cronus¶

HMC¶

Misc¶

Configuration Files and Command Line Options¶

Flashing Firmware with op-test¶

OpenBMC/Witherspoon Examples¶

AMI BMC/POWER8 OpenPOWER sytems examples¶

IBM FSP System examples¶

op-test and Qemu¶

op-test-framework

Navigation

Related Topics