How can I adapt the scheduling to my workflow?

Projects (but not individual users) can to some extend adapt the job scheduling to their needs.

If you want maximum throughput for your project:

• Make sure there are always jobs from your project waiting in the queue, that way you will be able to run a lot during periods of low demand from other projects.
• If possible, structure your jobs so that the time limit is 24 hours or less. Such jobs can be run as “bonus” jobs during periods of low demand even if your project has already hit the 1.5x limit. This can often let a project run a lot more than its allocated time.
• If possible, structure your jobs so that the time limit is between 30 minutes and 24 hours. Such jobs can often be started using “backfill” (using nodes that would otherwise be idle). Making your jobs shorter than 30 minutes does not help you much, and risks overloading the job scheduler.
• Use boost-tools when you need to get a few jobs started quickly despite the low priority that high throughput will give your project.

If you want short and predictable queue times:

• Keep track of how much your project runs (e.g using projinfo) and make sure you are always using less than your allocated time. You should then see short (sometimes none) and predictable queue times for your jobs.
• You can use boost-tools if your jobs still won’t start quickly (e.g when an even higher-priority project has jobs in the queue)
• Ensure that all project members work like this!

Please note that the job scheduler responds fairly slowly to changes in behaviour. If you change your behaviour (e.g start running fewer jobs) it will still take days or weeks until the full effect of this change is seen in your queue times.

If you want more throughput and short queue times

• Get a larger project allocation!

SNAC Small projects can be enlarged up to the maximum size for Small projects by emailing NSC Support, explaining why you need more time, and how much.

SNAC Medium projects can be enlarged up to the maximum size for Medium projects by emailing NSC Support, explaining why you need more time, and how much.