acs_docs
  • Linux Fundamentals
    • Introduction to Linux
      • Navigating the Linux Shell
        • Check Your Current Shell with echo
        • List All Environment Variables with env
        • Check Your Current Directory with pwd
        • List Directory Contents with ls
        • Move Between Directories with cd
        • Visualize Directory Structure with tree
        • Locate Commands with which
      • Interacting with Files
        • Creating Directories with mkdir
        • Removing Directories with rmdir
        • Removing Files and Directories with rm
        • Viewing File Contents with cat
        • Creating Files with cat and Redirection
        • Viewing Files with head and tail
        • Renaming and Moving with mv
        • Copying with cp
        • Editing Files with nano, emacs, vi
        • Viewing File Contents by Page with more and less
      • File Permissions in Linux
        • Understanding File Permission Categories
          • File and Directory Permission Examples
          • Decimal representation
        • Changing File Permissions in Linux
          • Examples with chmod
        • Changing Group Ownership in Linux
          • Examples with chgrp
      • Access Control Lists – ACL
        • Getting ACL information
          • Get ACL information with getfacl
        • Setting ACL information
          • Set ACL with setfacl -m (modify)
        • Removing ACL information
          • Remove ACL permissions with setfacl -x
          • Remove all ACL rules with setfacl -b
    • Linux FAQs
  • Services
    • Connecting to IDSC Systems from offsite
      • How to access Pegasus from home
      • What if I’m at home with no VPN / no UMIT laptop?
      • Connecting to Advanced Computing Systems from UM networks
    • Windows
    • Mac and Linux
    • Using VS Code
    • Forwarding the display with x11
    • Connecting to IDSC Systems from offsite
    • Storage
      • GPFS storage
      • CES storage
    • File Transfers
      • Use the Pegasus Data Transfer Node for large workflows
      • Using command line utilities
        • scp
        • rsync
        • rclone
      • Using FileZilla
      • Using VS Code
    • IDSC Onboarding Training Videos
    • IDSC Adanced Computing Services
  • User Guide
    • Quickstart
      • Before You Begin
      • 1. Connect to a Cluster
      • 2. Create a Scratch Working Directory
      • 3. Prepare the Example Files Locally
        • Create the Dataset
        • Create the Python Script
        • Create the LSF Job Script
      • 4. Transfer the Files to the Cluster
        • Transfer with SCP
        • Transfer with an SFTP Application
      • 5. Verify the Uploaded Files
      • 6. Submit the LSF Job
      • 7. Monitor the Job
      • 8. Inspect the Output
      • 9. Download the Result
      • 10. Disconnect
      • Next Steps
    • Important Guidelines
      • Accounts, Projects, and Policies
      • Network Access
      • Use Login Nodes Responsibly
      • Use Scratch Storage for Active Jobs
      • Request Appropriate Job Resources
      • Manage Software with Modules and Environments
      • Use the Data Transfer Node for I/O-Intensive Work
      • Getting Help
    • Projects & Resources
    • Job Scheduling
      • LSF Overview
      • Queues
        • Pegasus Queues
        • Triton Queues
      • Commands
      • Submitting Jobs
      • Monitoring Jobs
      • Examining Jobs
      • Interactive Jobs
      • Serial/Parallel Job Scripts
      • How to Request GPUs
      • Memory Management
        • Cluster Hardware and Queue Model
          • Standard compute nodes
          • Big-memory nodes (bigmem)
          • Queues and host spanning rules
        • Units and what is enforced
          • Memory units
          • Reservation vs. limit
        • How rusage[mem=…] behaves on Pegasus
          • Per-host reservation (the most important rule)
        • Controlling placement (single host vs multiple hosts)
          • Single-host placement
          • Multi-host placement (parallel jobs)
        • A simple formula to apply (parallel queue)
        • Guidance by queue
          • Queue: normal / general (single host only)
          • Queue: parallel (multi-host allowed)
          • Queue: bigmem (96 cores / 2 TB RAM)
        • Verifying what you asked for and what happened
          • Show what resources were requested
          • Confirm placement and packing
          • Diagnose pending jobs
        • Recommended starting points (Pegasus)
          • Standard node jobs (single host)
          • Parallel jobs
          • Bigmem jobs
        • Summary
    • Software and Modules
      • Software Modules
        • Basic Commands
        • Find Software
        • Load Software
        • Modules in LSF Jobs
        • System Notes
          • Pegasus
          • Triton/Summit
        • Recommended Practice
      • Python
        • Find and Load Python
        • User Package Installs
        • Virtual Environments
        • Run Python in LSF
        • Using a Virtual Environment in LSF
        • System Notes
          • Pegasus
          • Triton/Summit
        • Recommended Practice
      • Conda and Miniforge
        • Find Conda or Miniforge
        • Create an Environment
        • Use Project or Scratch Storage
        • Use Conda in LSF
        • Export an Environment
        • Pegasus Notes
        • Triton/Summit Notes
        • Recommended Practice
      • R
        • Find and Load R
        • User R Libraries
        • Run R in LSF
        • Recommended Practice
      • JupyterHub
        • IDSC-Managed JupyterHub
          • Typical Workflow
        • Personal JupyterLab Through LSF
          • Step 1: Connect to Pegasus
          • Step 2: Start an Interactive LSF Job
          • Step 3: Create the Startup Script
          • Step 4: Start JupyterLab
          • Step 5: Create the SSH Tunnel
          • Step 6: Open JupyterLab
          • Step 7: Stop the Personal JupyterLab Session
        • GPU JupyterLab Through LSF
          • Start a GPU Job
          • Start JupyterLab
          • Check GPU Access
          • PyTorch Check
          • TensorFlow Check
        • Troubleshooting
        • When to Contact IDSC
      • MATLAB
        • Find and Load MATLAB
        • Interactive MATLAB
        • Graphical MATLAB
        • Batch MATLAB Jobs
        • Running a MATLAB Function
        • Recommended Practice
      • Machine Learning Frameworks
        • Availability
        • Start a GPU Session
        • Architecture Notes
        • Check PyTorch
        • Check TensorFlow
        • Pegasus Notes
        • Triton/Summit Notes
        • Recommended Practice
      • Workflow Tools
        • Nextflow
          • Load Nextflow
          • Complete Pegasus Example
          • Storage Requirement
          • Resuming a Workflow
        • Recommended Practice
      • Debugging and Profiling Tools
        • Allinea Forge
        • DDT
        • MAP
        • Performance Reports
        • Graphical Tools
        • Recommended Practice
      • Specialized Scientific Software
        • Gaussian
          • Triton
          • Pegasus
        • GROMACS
          • Triton
          • Pegasus
        • LAMMPS
          • Triton
          • Pegasus
        • MAPSplice
          • Pegasus
        • SAS
          • Pegasus
        • SimVascular
          • Pegasus
    • Apptainer Containers
      • Quick Start
        • Load Apptainer
        • Cluster Module Quick Reference
        • View Apptainer Help
        • Download a Container Image
        • Shell: Interactive Use
        • Exec: Run a Command
        • Run: Use the Container Default Command
        • Choosing Between Shell, Exec, and Run
        • Command Order
        • Quick Validation Checklist
      • Container Images and Builds
        • Container Image Types
        • SIF Images
        • Image Cache
        • Temporary Writable Containers
        • Sandbox Directory
        • Persistent Overlay
        • Definition Files
        • Common Definition File Sections
        • Advanced Definition File Sections
        • Example Definition File
        • Build the Container
          • Fakeroot Shell Example
        • Skip Build-Time Tests
        • Test the Container
        • Run the Container
        • View Container Help
        • Recommended Usage
      • Inspecting and Verifying Images
        • Inspect an Image
        • View Container Help
        • Verify an Image
        • Sign an Image
        • List SIF Objects
        • Fingerprints Header
        • When to Inspect or Verify
        • Quick Command Summary
      • Working with Files and Bind Mounts
        • Default File Access
        • Run a Host Script Inside a Container
        • Writing Output Files
        • Custom Bind Mounts
          • Read-Only Bind Mounts
          • Using APPTAINER_BIND
          • Project/Scratch Bind Mount Example
        • Set the Working Directory
        • Pipes and Redirects
        • More Isolated Runs
        • Common Problems
        • Recommended Practice
      • Running Jobs with LSF
        • Interactive LSF Session
        • Direct Interactive Command
        • Batch Job Example
        • Use Full Paths in Job Scripts
        • Binding Data Directories in Jobs
        • Using --cleanenv
        • Avoid Downloading Images Inside Jobs
        • GPU Jobs
        • Recommended Job Pattern
        • Common LSF Commands
        • Common Problems
      • GPU Containers
        • Workflow
        • Prepare a Basic Image
        • Start a GPU Session
        • Check GPU Access on the Host
        • Run with --nv
        • Container Compatibility
        • Framework Images
          • Triton/Summit
          • Pegasus
        • Batch Jobs
        • Common Problems
        • Recommended Practice
      • Troubleshooting Common Issues
        • Apptainer Command Not Found
        • Image Is Read-Only
        • Wrong Architecture
        • File Not Found Inside Container
        • Permission Denied for Host Files
        • Output File Is Missing
        • Fakeroot Not Available
        • Package Installation Fails During Build
        • Container Works Interactively but Fails in LSF
        • GPU Not Visible Inside Container
        • TensorFlow Shows an Empty GPU List
        • PyTorch Reports CUDA Is Not Available
        • Environment Conflicts
        • Cache Uses Too Much Space
        • Remote Image Pull Fails
        • Jupyter or RStudio Left Running
        • When to Contact IDSC
      • Practical Examples
        • Example 1: Quick Container Check
        • Example 2: Run a Python Script
        • Example 3: Use Input and Output Directories
        • Example 4: CPU Batch Job with LSF
        • Example 5: Build a Small Container from a Definition File
        • Example 6: GPU Container Check
        • Example 7: More Isolated Run with Explicit Binds
        • Example 8: Reuse Bind Mounts with APPTAINER_BIND
        • Recommended Pattern
      • Advanced Topics
        • JupyterLab
          • One-Time Setup
          • Start JupyterLab
          • Request Different Resources
          • Create the SSH Tunnel
          • Open JupyterLab
          • Verify the Session
          • Manage JupyterLab Sessions
          • Stop the Session
          • Troubleshooting
        • RStudio Server
          • One-Time Setup
          • Start RStudio
          • Request Different Resources
          • Create the SSH Tunnel
          • Open RStudio Server
          • Verify the Session
          • Manage RStudio Sessions
          • Stop the Session
          • Troubleshooting
        • Interactive R
          • Pull the R container
          • Start an interactive LSF session
          • Run R inside the container
          • Example R session
          • End the interactive job
          • Using project data
          • Troubleshooting
        • Running MCMICRO
          • Load the Modules
          • Prepare the Directories
          • Download MCMICRO and the Exemplar Dataset
          • Create the Pegasus Configuration
          • Pre-pull the ASHLAR Image
          • Run the Registration Test
          • Troubleshooting
        • Advanced Image Security
          • Build-time Verification with Fingerprints
          • Fingerprint Notes
          • Creating Local Signing Keys
          • Signing an Image
          • SIF Object IDs and Groups
          • Object-level Verify Example
          • Security Practice
    • Parallel Computing
      • Loading the Appropriate MPI Module
      • Sample parallel programs:
        • C++ source code and compilation
        • C source code and compilation:
        • Fortran 90 source code and compilation:
        • Fortran 77 source code and compilation:
      • The LSF script to run parallel jobs
    • Sylvester HPC QuickStart
      • How do I get started?
      • Sylvester Utilization
      • Sylvester Dedicated HPC Queues
        • HPC Storage
        • Tier 1 Development (sccc-dev)
        • Tier 2 General (sccc)
        • Tier 3 Big Memory (sccc-bigmem)
        • Tier 3 GPU Queue (sccc-gpu)
        • Tier 4 Premium (sccc-premium, sccc-bigmem-premium, sccc-gpu-premium)
      • What are specs of available HPC Nodes?
        • Triton (96 nodes)
        • Pegasus Compute (350 nodes)
        • Pegasus Sylvester Dedicated Compute (16 nodes)
        • Pegasus Sylvester Dedicated Big Memory GPU (2 nodes)
      • How do I reset my IDSC password?
      • How do I access the Secure Network remotely?
      • How do I run Nextflow on Sylvester HPC resources?
  • IDSC ACS Policies & FAQs
    • Policies
      • Accounts
      • IDSC Links
      • Supercomputers
      • Allocations
      • Software
      • Support
    • FAQs
      • Projects and Resources
        • How do I join a project?
        • How do I request a new project?
        • When will my project be created?
        • How can I manage my Projects and Groups?
      • Software Suites
        • What software is available?
        • How do I view my currently loaded modules?
        • How do I use software modules?
        • May I install software?
        • How do I request global software installation?
        • When will my global software request be approved/installed?
        • How can I increase Java memory?
      • Job Scheduling
        • May I run resource-intensive jobs on login nodes?
        • How do I submit jobs?
        • How do I check on my submitted jobs?
        • How do I monitor job progress?
        • Is there a limit on how many jobs I can run?
        • How can I see pending and running job counts for queues?
        • Why is my job still pending?
        • Are other users’ pending jobs slowing my job?
        • How do I submit jobs to my Project?
        • How do I submit an interactive job?
        • How do I submit an interactive X11 job?
        • Why was my job killed?
        • What about jobs in UNKWN state?
        • GPU Resource Requests on Pegasus or Triton
        • Containerization Support on Pegasus
        • Disk Quota and Storage Expansion
    • Terms and Conditions
      • Secure Storage
acs_docs
  • IDSC ACS User Guide
  • Job Scheduling
  • View page source

Job Scheduling

  • LSF Overview
  • Queues
    • Pegasus Queues
    • Triton Queues
  • Commands
  • Submitting Jobs
  • Monitoring Jobs
  • Examining Jobs
  • Interactive Jobs
  • Serial/Parallel Job Scripts
  • How to Request GPUs
  • Memory Management
    • Cluster Hardware and Queue Model
    • Units and what is enforced
    • How rusage[mem=…] behaves on Pegasus
    • Controlling placement (single host vs multiple hosts)
    • A simple formula to apply (parallel queue)
    • Guidance by queue
    • Verifying what you asked for and what happened
    • Recommended starting points (Pegasus)
    • Summary
Previous Next

© Copyright 2024, UM IDSC ACS.

Built with Sphinx using a theme provided by Read the Docs.