FAQs - Frequently Asked Questions
Detailed information for FAQ topics is available here and in IDSC ACS Policies
Note: IDSC ACS does not install, provide support for, or provide documentation on how to code in your preferred software. ACS documentation contains information on using software in a Linux cluster environment.
Projects and Resources
Checkout the documentation on Projects & Resources for more information.
How do I join a project?
Contact the project owner. See <Projects & Resources> for more info.
How do I request a new project?
Any PI or faculty member may request a new project : IDSC Project Registration Form
When will my project be created?
When the allocations committee has reviewed and approved it. Scratch requests over 2TB can take a month for the allocations committee to review as availability is limited.
How can I manage my Projects and Groups?
Contact IDSC ACS by submitting a ticket `here https://idsc.miami.edu/hpc-help`_
Software Suites
Checkout the documentation on Pegasus Software Suites & Triton Software Suites for more information.
What software is available?
Software Modules from the command line: $ module avail.
How do I view my currently loaded modules?
$ module list
How do I use software modules?
May I install software?
Yes! Users are free to compile and install software in their respective home directories by following the software’s source code or local installation instructions. See our Software Installation on Pegasus guide for more information.
Note: IDSC ACS does not install user software. For global installations on Pegasus, submit a Software Request `here <https://idsc.miami.edu/systems`_.
How do I request global software installation?
Submit your request here. We only globally install software when we receive multiple requests for the software.
When will my global software request be approved/installed?
When a minimum of 20 users require it, software requests will be approved. Software requests are reviewed and installed quarterly.
How can I increase Java memory?
Load the java module, then change the value of _JAVA_OPTIONS.
[username@pegasus ~]$ module load java
[username@pegasus ~]$ echo $_JAVA_OPTIONS
-Xmx512m
[username@pegasus ~]$ export _JAVA_OPTIONS="-Xmx4g"
Job Scheduling
See our LSF Overview guide for more information.
May I run resource-intensive jobs on login nodes?
No. Resource-intensive jobs must be submitted to LSF.
How do I submit jobs?
With bsub (LSF Job Commands)
How do I check on my submitted jobs?
With bjobs (LSF Job Commands)
How do I monitor job progress?
With bpeek (LSF Job Commands)
Is there a limit on how many jobs I can run?
No. Users are limited by number of simultaneous CPUs used. Individual users can run on up to 512 CPUs at a time, projects on up to 1000 CPUs at a time.
How can I see pending and running job counts for queues?
With bqueues (LSF Job Commands)
Why is my job still pending?
Jobs wait for enough resources to satisfy requirements. When the cluster
is under heavy user load, jobs will wait longer. Use
$ bjobs -l jobID to see PENDING REASONS. Check your resource
requirements for accuracy and feasibility.
The job scheduler operates under Fairshare scheduling. Fairshare scheduling divides the processing power of the cluster among users and queues to provide fair access to resources, so that no user or queue can monopolize the resources of the cluster and no queue will be starved.
If your job has been pending for more than 24 hours and is not requesting exclusive access or all cores on a node, you may request help `here <https://idsc.miami.edu/hpc-help`_ for assistance.
Are other users’ pending jobs slowing my job?
No. The number of pending jobs is irrelevant to job performance in LSF. The scheduler can handle hundreds of thousands of jobs.
How do I submit jobs to my Project?
With the -P flag (LSF Overview)
How do I submit an interactive job?
With the -Is -q interactive (Interactive Jobs)
How do I submit an interactive X11 job?
With the -Is -q interactive -XF flags (Interactive Jobs)
Why was my job killed?
Jobs are killed to protect the cluster and preserve system performance. Common reasons include:
running on a login node
using more memory than reserved
using all the memory on a compute node
using more CPUs than reserved
needing more time to complete than reserved
using more
/tmpspace than available on compute nodes
See LSF Overview for assistance with appropriate resource reservations and Pegasus and Triton Queues for default wall times.
What about jobs in UNKWN state?
Re-queue your job in LSF :
$ bkill -r jobID$ bkill -r jobID (a second time)$ brequeue -e jobIDGPU Resource Requests on Pegasus or Triton
If GPU resources are not accessible to you on cluster nodes, please contact the IDSC support team `here <https://idsc.miami.edu/hpc-help`_ with the following information:
Project details
Number of GPUs required
Names and profile details of users who need access
Cluster name (Pegasus or Triton)
Principal Investigator (PI)
User email ID
If GPU resources are accessible to you on cluster nodes, then you can request GPU resources by editing your LSF job script to submit to a GPU queue with the following parameters:
#BSUB -q gpu_queue_name # Example: gpu_h100
#BSUB -gpu "num=1"
Containerization Support on Pegasus
Docker containers are not allowed on shared resource clusters like Pegasus or Triton because they require root privileges. Pegasus supports containerization through:
Apptainer (formerly Singularity) (Check here)
Specific containerized software like RStudio (Check here)
If you need a specific application containerized, please contact the IDSC support team `here <https://idsc.miami.edu/hpc-help`_ with your current project and detailed requirements.
Disk Quota and Storage Expansion
Home storage cannot be expanded.
Scratch storage may be expanded on a case-by-case basis.
If 10TB or more of storage is required, users will need to purchase either GPFS or CES storage. Check out the details on our storage services page: s_storage and for pricing details, refer to the following link: IDSC Fee Schedule
To request additional disk space allocation, contact the IDSC support team `here <https://idsc.miami.edu/systems`_ with the following information:
Current allocation details
Required additional space
Project details
PI
User email ID