Tasks

Submit a job to a remote server and handle logging.

This module can be used either with Celery, in which case it will run in a background thread, or as a normal function call, in which case it will block the current execution thread.

class django_remote_submission.tasks.LogPolicy[source]

Specify how logging should be done when running a job.

LOG_NONE = 0

Don’t log anything from the running job.

LOG_LIVE = 1

Create Log objects immediately when they are received.

LOG_TOTAL = 2

Combine all of stdout and stderr at the end of the job.

django_remote_submission.tasks.is_matching(filename, patterns=None)[source]

Check if a filename matches the list of positive and negative patterns.

Positive patterns are strings like "1.txt", "[23].txt", or "*.txt".

Negative patterns are strings like "!1.txt", "![23].txt", or "!*.txt".

Each pattern is checked in turn, so the list of patterns ["!*.txt", "1.txt"] will still match "1.txt".

>>> from django_remote_submission.tasks import is_matching
>>> is_matching("1.txt", patterns=["1.txt"])
True
>>> is_matching("1.txt", patterns=["[12].txt"])
True
>>> is_matching("1.txt", patterns=["*.txt"])
True
>>> is_matching("1.txt", patterns=["1.txt", "!*.txt"])
False
>>> is_matching("1.txt", patterns=["!*.txt", "[12].txt"])
True
class django_remote_submission.tasks.LogContainer(job, log_policy)[source]

Manage logs sent by a job according to the log policy.

>>> from django_remote_submission.tasks import LogContainer, LogPolicy
>>> from datetime import datetime
>>> now = datetime(year=2017, month=1, day=2, hour=3, minute=4, second=5)
>>> logs = LogContainer(job, LogPolicy.LOG_LIVE)
>>> logs.write_stdout(now, 'hello world')  # doctest: +SKIP
>>> Log.objects.get()  # doctest: +SKIP
<Log: 2017-01-02 03:04:05 My Job>
__init__(job, log_policy)[source]

Instantiate a log container.

Parameters:
  • job (models.Job) – the job these logs are coming from
  • log_policy (LogPolicy) – the policy to use for logging
class LogLine(now, output)
now

Alias for field number 0

output

Alias for field number 1

job = None

The job that these logs are coming from.

log_policy = None

The policy to use when logging.

write_stdout(now, output)[source]

Write some output from a job’s stdout stream.

Parameters:
  • now (datetime.datetime) – the time this output was produced
  • output (str) – the output that was produced
write_stderr(now, output)[source]

Write some output from a job’s stderr stream.

Parameters:
  • now (datetime.datetime) – the time this output was produced
  • output (str) – the output that was produced
flush()[source]

Flush the stdout and stderr lists to Django models.

If the log_policy is LogPolicy.LOG_TOTAL, this method will need to be called at the end of the job to ensure all the data gets written out.

There is no penalty for calling this method multiple times, so it can be called at the end of the job regardless of which log policy is used.

django_remote_submission.tasks.submit_job_to_server(*a, **kw)[source]

Submit a job to the remote server.

This can be used as a Celery task, if the library is installed and running.

Parameters:
  • job_pk (int) – the primary key of the models.Job to submit
  • password (str) – the password of the user submitting the job
  • public_key_filename – the path where it is.
  • username (str) – the username of the user submitting, if it is different from the owner of the job
  • timeout (datetime.timedelta) – the timeout for running the job
  • log_policy (LogPolicy) – the policy to use for logging
  • store_results (list(str)) – the patterns to use for the results to store
  • remote (bool) – Either runs this task locally on the host or in a remote server.
django_remote_submission.tasks.copy_key_to_server(*a, **kw)[source]

Copy the client key to the remote server so the next connections do not need the password any further

This can be used as a Celery task, if the library is installed and running.

Parameters:
  • username (str) – the username of the user submitting
  • password (str) – the password of the user submitting the job
  • hostname (str) – The hostname used to connect to the server
  • port (int) – The port to connect to for SSH (usually 22)
  • public_key_filename – the path where it is.
  • remote (bool) – Either runs this task locally on the host or in a remote server.
django_remote_submission.tasks.delete_key_from_server(*a, **kw)[source]

Delete the client key from the remote server so the next connections will need password. This can be used at the logout of the session.

This can be used as a Celery task, if the library is installed and running.

Parameters:
  • username (str) – the username of the user submitting
  • password (str) – the password of the user submitting the job
  • hostname (str) – The hostname used to connect to the server
  • port (int) – The port to connect to for SSH (usually 22)
  • public_key_filename – the path where it is.
  • remote (bool) – Either runs this task locally on the host or in a remote server.