Child pages
  • Lumerical on the FAS Odyssey cluster (SEAS users only)
Skip to end of metadata
Go to start of metadata

Lumerical is a suite of software for simulating photonic components, circuits and systems (https://www.lumerical.com/tcad-products/). 

Lumerical FDTD is a finite-difference-time-domain solver for time-dependent Maxwell's equations. The solver is parallelized using MPI. In order to use FDTD, one needs to first build the model using Lumerical CAD software. Currently SEAS Computing has limited CAD licenses. If you need to use CAD on a regular basis, we encourage you to buy a license for CAD directly from Lumerical. Once you have the CAD based model available, you can use FAS RC resources for solving the model with FDTD.

Lumerical Mode consists of an eigenmode solver and a propagator (describes propagation of light in planar integrated optical systems). 

Lumerical has also introduced a multiphysics suite with finite element capabilities (charge, heat, DGTD etc.). Please refer to the above Lumerical page for details.

With the introduction of a convenient virtual desktop (VDI) on the FASRC Cluster (see below), the only supported mode for working with Lumerical is now on FASRC.

Getting started

Obtain a FAS RC account by visiting:

https://portal.rc.fas.harvard.edu/request/account/new

Please consult the documentation available via the above link for connecting to FAS systems and familiarizing yourself with the SLURM queue management system. For any help, contact RC using the information available on:

https://rc.fas.harvard.edu/about/contact/

To use Lumerical, you need to be in the SEAS group. You may want to request FASRC to be added to SEAS group (rchelp@rc.fas.harvard.edu).

OnDemand Virtual Desktop (VDI)

You can now work conveniently via a browser on the FASRC cluster using the virtual desktop infrastructure (VDI) known as OnDemand. For more details, please visit:

https://www.rc.fas.harvard.edu/resources/documentation/virtual-desktop/

You may have to adjust the number of cores and memory when you request an OnDemand session. Start with the default and change options after you have a better idea of resource requirements. In a terminal, type:

Lumerical versions

module-query lumerical

Lumerical – Interactive Simulations – Newer versions

Lumerical can be used interactively on the FASRC cluster via the new VDI. Exact way of invoking the relevant graphical user interface (GUI) depends on the version of Lumerical you are using. For the latest version (2019b_r2_e2360c6-fasrc01), you can do:

module load lumerical-seas/2019b_r2_e2360c6-fasrc01
launcher

This should open a GUI. You should be able to choose the software component (FDTD, MODE, etc.) by clicking on "Products". Start a new project by clicking on "New Project". You should also be able to download examples via the "Examples" item on the main page. You need to have (or create) an account on Lumerical to be able to do this.

Lumerical – Interactive Simulations – Older versions

Again, you need to be on FASRC cluster via VDI for this. For older versions (eg. lumerical-seas/8.20.1634-fasrc01), you can do:

module load lumerical-seas/8.20.1634-fasrc01

For FDTD, type in a terminal,

CAD

For MODE, type in a terminal,

mode-solutions

For DEVICE, type in a terminal,

device


Running Lumerical FDTD in batch using SLURM

The following SLURM script 'runscript_lumerical.sh' will load the necessary SW using modules and run the model using parallel (OpenMPI) version of FDTD. The script requests 2 CPU cores (as indicated by '#SBATCH -n 2', and solves the problem using 2 MPI processes (as indicated by 'mpirun -np  ...'):

#!/bin/bash
#
#SBATCH -n 2        # Number of cores 
#SBATCH -t 0-00:05  # Runtime in D-HH:MM format 
#SBATCH -p general  # Partition to submit to 
# SLURM requires memory specification. For lumerical, this may be found from
# the GUI for FDTD interactively via the "Check" button at the top.
#SBATCH --mem=500 

module load intel/17.0.4-fasrc01 impi/2017.2.174-fasrc01 lumerical-seas/2019b_r2_e2360c6-fasrc01

#For older version, comment above and uncomment below.
#module load intel/17.0.4-fasrc01 impi/2017.2.174-fasrc01 lumerical-seas/8.20.1634-fasrc01

# use $SLURM_NTASKS env set by #SBATCH -n value
#nanowire.fsp is an example input file. Just google for "Lumerical nanowire.fsp" and it will
#lead you to the Lumerical Knowledge Base with the file. You may need to open the file in the
#GUI for FDTD and save it before you can run it via batch.

mpirun -np $SLURM_NTASKS fdtd-engine-impi-lcl nanowire.fsp

date

exit 

You can copy and paste the above into a file named runscript_lumerical.sh  and submit it to SLURM as follows:

sbatch ./runscript_lumerical.sh


Running Lumerical Mode in batch using SLURM

This example illustrates solving the 'ring_resonator.lms' model that is available in the Lumerical Knowledge Base. The following SLURM script will load the necessary modules and run the model using parallel (OpenMPI) version of FDTD. The script requests 2 CPU cores (as indicated by '#SBATCH -n 2', and solves the problem using 2 MPI processes (as indicated by 'mpirun -np  ...'):

#!/bin/bash
#
#SBATCH -n 2        # Number of cores 
#SBATCH -t 0-00:05  # Runtime in D-HH:MM format 
#SBATCH -p general  # Partition to submit to 
# SLURM requires memory specification.
#SBATCH --mem=500 
 
module load intel/17.0.4-fasrc01 impi/2017.2.174-fasrc01 lumerical-seas/2019b_r2_e2360c6-fasrc01


#For older version, comment above and uncomment below.
#module load intel/17.0.4-fasrc01 impi/2017.2.174-fasrc01 lumerical-seas/8.20.1634-fasrc01

# use $SLURM_NTASKS env set by #SBATCH -n value
#ring_resonator.lms is an example input file. Just google for "Lumerical ring_resonator.lms" and it will
#lead you to the Lumerical Knowledge Base with the file. You may need to open the file in the
#GUI and save it before you can run it via batch.

mpirun -np $SLURM_NTASKS varfdtd-engine-mpich2nem -logall ring_resonator.lms 

date

exit 

You can copy and paste the above into a file named runscript_mode.sh  and submit it to SLURM as follows:

sbatch ./runscript_mode.sh

 

For further details on using Odyssey and  SLURM (such as monitoring or canceling your jobs) please visit https://rc.fas.harvard.edu or the list of convenient SLURM commands at https://rc.fas.harvard.edu/resources/documentation/convenient-slurm-commands/.

Running Lumerical Device in batch using SLURM

The input file for this simulation and the associated documentation is at:

https://kb.lumerical.com/en/solar_cells_device_getting_started.html

You need to download the input file "solar_cell_simple.ldev" from the above site.

Important: As this file was produced with a different version of Device than the one we would be using, you need to open a Device GUI session (using the same Lumerical version as the one you would be using in the batch script), load the input file into GUI and save it (we have saved it as 'solar_cell_simple.ldev').  You can then use the following batch script:

#!/bin/bash
#
#SBATCH -N 1        # Number of nodes
#SBATCH -c 8        # Number of CPUs
#SBATCH -t 0-00:05  # Runtime in D-HH:MM format
#SBATCH -p general  # Partition to submit to 
# SLURM requires memory specification. For lumerical, this may be found from
# CAD via the "Check" button at the top.
#SBATCH --mem=500


module load intel/17.0.4-fasrc01 impi/2017.2.174-fasrc01 lumerical-seas/2019b_r2_e2360c6-fasrc01

#For older version, comment above and uncomment below.
#module load intel/17.0.4-fasrc01 impi/2017.2.174-fasrc01 lumerical-seas/8.20.1634-fasrc01

echo $SLURM_CPUS_PER_TASK

#Note: device-engine cannot use processors across nodes. It can use
#processors (or cores) on the same node.

export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK

#This assumes you have saved the example file as solar_cell_simple.ldev
device-engine solar_cell_simple.ldev

date

exit                                                                                                                                               

Documentation and help

  • Lumerical Online Help
  • Please note that Lumerical does not provide support for individual users under current licensing (starting November, 2016). 
    You can get help by visiting Lumerical Knowledge Base and via the Knowledge Exchange link available on that page.
  • No labels