Running commands in parallel

Use command.start() and await_done() to run multiple commands concurrently across one or more devices.

Moler command objects implement the Future pattern: you can start a command in the background, do other work, and collect the result later. This lets you run operations on different devices simultaneously without managing threads directly.

The Future pattern

Every Moler command object is both:

• A callable — call it like a function to block until it finishes and returns a result.
• A future — call .start() to launch it in the background, then .await_done(timeout) to collect the result when you’re ready.

# Blocking (foreground) — simple but sequential
result = ping_cmd()

# Non-blocking (background) — start now, collect later
ping_cmd.start()
# ... do other work ...
result = ping_cmd.await_done(timeout=10)

await_done(timeout) requires an explicit timeout. Moler enforces this by design: every test must know its worst-case wait time. There is no way to await indefinitely.

Complete example: ping and ls in parallel

The following example — taken directly from the Moler README — pings a public host while simultaneously listing files on a remote machine over SSH:

from moler.config import load_config
from moler.device import DeviceFactory

load_config(config='my_devices.yml')

my_unix = DeviceFactory.get_device(name='MyMachine')
host = 'www.google.com'
ping_cmd = my_unix.get_cmd(cmd_name="ping",
                           cmd_params={"destination": host, "options": "-w 6"})

remote_unix = DeviceFactory.get_device(name='RebexTestMachine')
remote_unix.goto_state(state="UNIX_REMOTE")
ls_cmd = remote_unix.get_cmd(cmd_name="ls", cmd_params={"options": "-l"})

print("Start pinging {} ...".format(host))
ping_cmd.start()                                # run command in background
print("Let's check readme.txt at {} while pinging {} ...".format(remote_unix.name, host))

remote_files = ls_cmd()                         # foreground "run in the meantime"
file_info = remote_files['files']['readme.txt']
print("readme.txt file: owner={fi[owner]}, size={fi[size_bytes]}".format(fi=file_info))

ping_stats = ping_cmd.await_done(timeout=6)     # await background command
print("ping {}: {}={}, {}={} [{}]".format(host, 'packet_loss',
                                          ping_stats['packet_loss'],
                                          'time_avg',
                                          ping_stats['time_avg'],
                                          ping_stats['time_unit']))

Output:

Start pinging www.google.com ...
Let's check readme.txt at RebexTestMachine while pinging www.google.com ...
readme.txt file: owner=demo, size=403
ping www.google.com: packet_loss=0, time_avg=35.251 [ms]

The key point is the ordering: ping_cmd.start() fires the ping in the background immediately, then ls_cmd() runs and completes in the foreground, and finally ping_cmd.await_done(timeout=6) blocks until the ping finishes (or times out).

How the main log shows parallel activity

With logging configured, moler.log records activity across all devices:

22:30:20.919  MyMachine        |Command 'moler.cmd.unix.ping.Ping':'ping www.google.com -w 6' started.
22:30:20.920  MyMachine        |ping www.google.com -w 6
22:30:20.920  RebexTestMachine |Command 'moler.cmd.unix.ls.Ls':'ls -l' started.
22:30:20.922  RebexTestMachine |ls -l
22:30:20.985  RebexTestMachine |Command 'moler.cmd.unix.ls.Ls' finished.
22:30:26.968  MyMachine        |Command 'moler.cmd.unix.ping.Ping' finished.

The timestamps confirm that ls started at 20.920 and finished at 20.985, while ping was still running until 26.968 — they overlapped.

Step-by-step: the start → work → await pattern

1. Get the command object

   Obtain a command from a device. The command is not yet running.

   ping_cmd = my_unix.get_cmd(cmd_name="ping",
                              cmd_params={"destination": "8.8.8.8", "options": "-w 10"})
   
2. Start in background

   Call .start() to send the command over the connection and begin parsing output in a background thread. The call returns immediately.

   ping_cmd.start()
   
3. Do other work

   While the background command is running, execute other commands (on the same or different devices).

   # This blocks until ls completes — fine because ping is running in background
   files = ls_cmd()
   
4. Await the background command

   Call .await_done(timeout) to wait for the background command to complete and retrieve its result. If the command doesn’t finish within timeout seconds, MolerTimeout is raised.

   ping_stats = ping_cmd.await_done(timeout=10)
   print("avg RTT:", ping_stats['time_avg'], ping_stats['time_unit'])
   

Running on multiple devices simultaneously

You can start commands on several devices and await them all:

from moler.config import load_config
from moler.device import DeviceFactory

load_config(config='my_devices.yml')

# Get command objects for three different devices
devices = [DeviceFactory.get_device(name=n) for n in ['Host1', 'Host2', 'Host3']]
cmds = [
    dev.get_cmd(cmd_name="ps", cmd_params={"options": "-ef"})
    for dev in devices
]

# Fire all ps commands in parallel
for cmd in cmds:
    cmd.start()

# Collect results from each
results = [cmd.await_done(timeout=15) for cmd in cmds]
for device, result in zip(devices, results):
    python_procs = [p for p in result if 'python' in p.get('CMD', '')]
    print(f"{device.name}: {len(python_procs)} python processes")

Using a direct connection (without device)

You can also use commands directly with a connection object, bypassing the device abstraction entirely:

import time
from moler.cmd.unix.ping import Ping
from moler.connection_factory import get_connection

host = 'www.google.com'
terminal = get_connection(io_type='terminal', variant='threaded')
with terminal.open():
    ping_cmd = Ping(connection=terminal.moler_connection,
                    destination=host, options="-w 6")
    print("Start pinging {} ...".format(host))
    ping_cmd.start()
    print("Doing other stuff while pinging {} ...".format(host))
    time.sleep(3)
    ping_stats = ping_cmd.await_done(timeout=4)
    print("ping {}: {}={}, {}={} [{}]".format(host, 'packet_loss',
                                              ping_stats['packet_loss'],
                                              'time_avg',
                                              ping_stats['time_avg'],
                                              ping_stats['time_unit']))

Start pinging www.google.com ...
Doing other stuff while pinging www.google.com ...
ping www.google.com: packet_loss=0, time_avg=50.000 [ms]

The terminal connection is a context manager — it opens on entry and closes on exit.

Timeout parameter importance

Always choose timeout values that reflect the realistic worst-case duration of the command. Setting a timeout that is too short causes MolerTimeout and aborts the command (Moler sends Ctrl+C to the device). Setting it too long means your test suite will hang for a long time if something goes wrong.

For ping -w 6 (which runs for up to 6 seconds), a timeout of 6 seconds is the minimum. A small buffer like timeout=8 is safer:

# ping runs for up to 6 seconds; give it a 2-second buffer
ping_stats = ping_cmd.await_done(timeout=8)

The timeout clock starts from when .start() is called. If the device is slow to respond, that counts against the timeout.