.. _user-guide: op-test User Guide ================== .. _commandline: Command Line Options -------------------- All configuration options can be specified via the command line or in a configuration file (see :ref:`config-file`). .. argparse:: :module: OpTestConfiguration :func: get_parser :prog: op-test .. _config-file: Configuration Files and Command Line Options -------------------------------------------- When working with `op-test`, you can specify every configuration option on the command line, or you can save some in a configuration file. Typically, you may keep a configuration file per-machine you test against, to save on typing out all the IP addresses and login credentials. For example, this ``witherspoon.conf`` file will connect to a Witherspoon machine: .. code-block:: ini [op-test] bmc_type=OpenBMC bmc_ip=10.0.0.1 bmc_username=root bmc_password=0penBmc host_ip=10.0.0.2 host_user=root host_password=abc123 It can be used as such: .. code-block:: bash ./op-test --config-file witherspoon.conf Other options can also be specified on the commandline, such as ``--host-pnor`` which is documented in :ref:`flashing-firmware`. There is also a *per user* configuration file, at ``~/.op-test-framework.conf`` where you can store global options, for example: .. code-block:: ini [op-test] smc_presshipmicmd=foo pupdate=~/pupdate pflash=~/pflash ubuntu-cdrom=~/ubuntu-18.04-ppc64el.iso This per user configuration file will be loaded in *addition* to the config file provided on the command line. .. _flashing-firmware: Flashing Firmware with op-test ------------------------------ In the future, there may be some standard interface for flashing firmware onto OpenPOWER machines. We are not yet in that future, so the method of flashing firmware varies somewhat between platforms. There is code in `op-test` to hide these differences from you, and just provide easy ways to say "flash this firmware and run the test suite". This functionality is *primarily* focused towards firmware developers. By default, `op-test` will flash any firmware provided *before* running any tests. You can change this behaviour with the ``--no-flash`` or `-only-flash`` command line options (which should be fairly self-explanatory). To flash a full PNOR image, you will need one in the correct format: AMI BMC based systems (e.g. Palmetto, Habanero, Firestone and Garrison) A raw PNOR file (`palmetto.pnor`, `habanero.pnor` etc) as produced by `op-build` should be provided to the ``--host-pnor`` command line option. `op-test` will use this with `pflash` on the BMC to write the whole PNOR image to flash. It will **NOT** preserve *anything* from what was previously flashed (e.g. NVRAM, GUARD records). Since most AMI BMCs do not ship with `pflash` you will need to tell `op-test` where to find a pflash binary compiled for the BMC with the ``--pflash`` option IBM FSP based systems (e.g. Tuleta, ZZ) The only current method that `op-test` supports for flashing a full FSP image is in-band, fetching the image over the network. You need to specify the URL to the full firmware image to the ``--host-image-url`` command line option. OpenBMC based systems (e.g. Witherspoon, Romulus) There are two ways for OpenBMC systems to store PNOR, either using the full NOR chip for the PNOR, or using the vPNOR (Virtual PNOR) method. The normal PNOR method needs a ``.pnor`` file, e.g. ``romulus.pnor``. The vPNOR method needs a ``.squashfs.tar`` file, e.g. ``witherspoon.pnor.squashfs.tar``. If you provide the incorrect format, `op-test` will error out. For *BMC* firmware, you also need the machine specific BMC firmware image format to supply to the ``--bmc-image`` command line option. For some BMCs, you may need an external utility. For example, Supermicro (SMC) BMCs need the ``pUpdate`` utility, which you can point `op-test` at with the ``--pupdate`` command line option. OpenBMC/Witherspoon Examples ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ These examples are for the Witherspoon system, which uses OpenBMC as a BMC and the vPNOR implementation. For Romulus (which does not use vPNOR), instead of `witherspoon.pnor.squashfs.tar` you may need to use `witherspoon.pnor`. **FIXME** Confirm details on Romulus. For example, this command will use the ``witherspoon.conf`` configuration file (see :ref:`config-file`) for login credentials to a Witherspoon machine, and will flash *host* firmware before running the default test suite: .. code-block:: bash ./op-test --config-file witherspoon.conf \ --host-pnor ~/op-build/output/images/witherspoon.pnor.squashfs.tar In this example we've provided the *full* path to a witherspoon firmware image that we've built using `op-build`. If you *also* want to flash BMC firmware, you can do that with the addition of the ``--bmc-image`` command line option: .. code-block:: bash ./op-test --config-file witherspoon.conf \ --bmc-image obmc-phosphor-image-witherspoon.ubi.mtd.tar \ --host-pnor ~/op-build/output/images/witherspoon.pnor.squashfs.tar In this example, `op-test` will first update the BMC firmware, then update the host firmware and *then* run the test suite. If you're a skiboot/OPAL developer and wanting to test your latest code when applied on top of a known-good BMC and PNOR image, you can use the ``--flash-skiboot`` command line option to instruct `op-test` to, as a final step, overwrite the `PAYLOAD` partition with your skiboot: .. code-block:: bash ./op-test --config-file witherspoon.conf \ --bmc-image obmc-phosphor-image-witherspoon.ubi.mtd.tar \ --host-pnor ~/op-build/output/images/witherspoon.pnor.squashfs.tar \ --flash-skiboot ~/skiboot/skiboot.lid.xz.stb In this case, if "field mode" is enabled on the BMC, `op-test` will disable it for you to allow for overriding host firmware with the skiboot image you aksed it to use. Since the Witherspoon platform has Secure Boot enabled, you will need the `.stb` variant of skiboot (i.e. with the Secure and Trusted Boot header), and since we're an OpenPOWER system, we need the `.xz` compressed version, and this is why we provide `skiboot.lid.xz.stb` to `op-test` for this system. **Note** that with Secure Boot enabled, by default we only sign with *imprint* keys. AMI BMC/POWER8 OpenPOWER sytems examples ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ For machines such as Palmetto, Habanero, Firestone and Garrison. **TODO** Document BMC flashing. These systems have an AMI BMC and `op-test` will use `pflash` on the BMC to write host firmware. You will need to point `op-test` towards a `pflash` binary compiled for the BMC for `op-test` to copy over and use to flash firmware. **TODO** Document HPM flashing. An example of flashing a full `habanero.pnor` image and running the default test suite is: .. code-block:: bash ./op-test --config-file hab4.conf \ --host-pnor ~/op-build/output/images/habanero.pnor Just like on other systems, if you're an OPAL/skiboot developer and you want to test your changes along with a known-good full PNOR image, you'd do that the same way, using the ``--flash-skiboot`` parameter: .. code-block:: bash ./op-test --config-file hab4.conf \ --host-pnor ~/op-build/output/images/habanero.pnor \ --flash-skiboot ~/skiboot/skiboot.lid.xz We need to provide the `skiboot.lid.xz` file as all POWER8 OpenPOWER systems need the compressed payload in order to fit in flash. It is only *very* old Hostboots that do not support this and require the raw `skiboot.lid`. IBM FSP System examples ^^^^^^^^^^^^^^^^^^^^^^^ For machines such as Tuleta and ZZ (firenze class). Your FSP must have an NFS mount and be configured correctly for this operation. Currently, flashing a full FSP image is only supported by doing it from the host. In future, we may support out of band methods. The primary use `op-test` on Tuleta/ZZ is for flashing new OPAL LIDs onto an existing FSP image. Unlike OpenPOWER machines, the kernel and initramfs are split up into two separate LIDs, and must be pointed to separately. This example will run the stest suite against our ZZ machine *after* flashing our skiboot, kernel and initramfs built fresh from `op-build` (with the configuration `zz_defconfig`). .. code-block:: bash ./op-test --config-file zz.conf \ --flash-skiboot ~/op-build/output/images/skiboot.lid \ --flash-kernel ~/op-build/output/images/zImage.epapr \ --flash-initramfs ~/op-build/output/images/rootfs.cpio.xz For FSP based systems, the *uncompressed* `skiboot.lid` is needed, as the FSP will load this image directly into memory and start executing it. op-test and Qemu ---------------- You can use the 'qemu' BMC type to run many tests using the qemu simulator. This can be useful for test development/debug as well as testing the qemu simulator itself. It may be useful to keep a configuration file with your qemu configuration in it for running tests. An example of such a configuration file is below: .. code-block:: ini [op-test] bmc_type=qemu qemu_binary=~/qemu/ppc64-softmmu/qemu-system-ppc64 host_pnor=palmetto.pnor flash_skiboot=~/skiboot/skiboot.lid flash_kernel=zImage.epapr flash_initramfs=rootfs.cpio host_user=ubuntu host_password=abc123 ubuntu_cdrom=osimages/ubuntu-17.10-server-ppc64el.iso Note that for `qemu` we want the *uncompressed* `skiboot.lid` for `qemu` to load, and while it's not *required*, using the uncompressed `rootfs.cpio` does *significantly* improve boot time to Petitboot. In this configuration file example, we point to a `qemu` development tree rather than using the system default `qemu-system-ppc64` binary. We also specify a ``--host-pnor`` file so that `qemu` can mount an NVRAM partition. To run the "boot to petitboot" test in qemu with the above configuration file, you can do so like this: .. code-block:: bash ./op-test --config-file qemu.conf \ --run testcases.BasicIPL.BootToPetitbootShell Not all tests currently pass in `qemu`, and running tests in `qemu` should be considered somewhat experimental.