Skip to end of metadata
Go to start of metadata

What is CESM / CAM / SPCAM?

(Please add more)

CESM is earth system model.

CAM is the atmospheric part of CESM.

CCSM is an older version of CESM.

SPCAM is super-parameterized CAM.

POP is cesm's ocean model.

Running CESM on Harvard cluster

Latest release on official GitHub:

CESM2 with SPCAM (Jun 2018 or later): instructions to download yourself
CESM1.1.1 with SPCAM (Feb 2013) is downloaded in ~pchan/spcam2_0-cesm1_1_1/
CESM1.2.2 (Jun 2014) is downloaded in /n/holystore01/INTERNAL_REPOS/CLIMATE_MODELS/cesm1_2_2/
SPCAM3.5.0 (Feb 2009) is downloaded in ~kuang/Model/origSPCAM/

1. One-time setup

Register in

If you do not yet have "huce file access", request it from RC. If you see groups=...10047(huce)... when you run id, then you already have the access.

Edit ~/.bashrc to include following lines (May 2018, also known as CentOS7):

module load intel/17.0.4-fasrc01 impi/2017.2.174-fasrc01 netcdf/4.1.3-fasrc02

module load perl-modules/5.10.1-fasrc13

If you never use cpan (package manager for Perl) before, after loading perl-modules, enter cpan, respond: yes, blank, yes.., then exit cpan. (takes several minutes)

If you need POP2 (e.g. compset B), enter cpan, key in: install Switch, then exit cpan.

cp /n/home05/pchan/spcam2_0-cesm1_1_1/scripts/ ~/  #please copy a newer version after May 2018 (the CentOS7 update)

2. Edit

Edit ~/  # look at TODO’s. For details, see "about".

Partition is default to be huce_amd. (huce_amd decomissioned, 2/3/2020. Instead, you can use huce_intel) For more about HUCE's partitions, see Harvard cluster job queues.

3. (Advanced user): prepare namelist and SourceMods

If you already know what namelist (user_nl_*) or SourceMods you are going to use, cd to the directory with user_nl_* and/or SourceMods. should be run in this directory.

4. Run (~12 mins)

Make sure you have modules loaded as step 1.


E.g. ~/ c1test f19_f19 F_2000 64

5. Final check

You can modify SourceMods here, but if you do so, you will have to do $CASE.clean_build and rebuild the model with

CASE=`./xmlquery -valonly -silent CASE`
srun -p huce_intel,test -c 4 -t 100 --mem=4000 ./${CASE}.build

You can modify namelist here. Often you don't have to rebuild the model. should give you a sentence like this:

2018-06-15T13:30:20 TEST-ONLY-sbatch: Job 46319755 to start at 2018-06-15T13:30:23 using 64 processors on holy2c[12315-12316]

If you are not satisfied with the queue time, or any sbatch setting, you can modify $, and run this command to test the setting:

echo -n `date +%FT%T` TEST-ONLY-; sbatch --test-only ${CASE}.run

6. Submit the model

Make sure you have modules loaded as step 1.

Follow instructions from printout of, for example:

cd /n/home05/pchan/cesm_caseroot/c1test



This script combines create_newcase, cesm_setup and build. It is originally written by Andy Rhines, and further developed by Wanying Kang, Minmin Fu and Packard Chan. You do not have to use this script if you want to do each step separately.

How to modify:

OUTPUT is the path of large file storage where all Build/Run and Short Term Archive will be located. Default is kuanglfs, which works only for Kuang's group. "$SCRATCH/`id -gn`/$USER/cesm_output/${CASE}" will work well if you want to use $SCRATCH.

Partition is default to be huce_amd. To use huce_intel, uncomment the 3rd-4th lines.

Email notification is default off. Turn on by uncommenting the relevant line.

DOUT_S is now default true. This switch turns on the short term archive procedure, which moves things out of RUNDIR when job completes, i.e. (1)-(3) in the following figure. If you want to leave everything in RUNDIR, set to false.

STOP_N and STOP_OPTION set the length of the model run.

Setting GET_REFCASE to false may cause problem for some compset (e.g. B_1850_CN). Now using default from compset.

REST_N determines how often restart files are saved. Default is only once, after the run completes. For a long run, uncomment this line and change to something like 1 (nyears) so you can restart your run if the job fails.

DOUT_S_SAVE_INT_REST_FILES is a switch that can stop "short term archive" from deleting your intermediate restart files. However, those restart files will not be put in one folder for restart, but distributed across components.

BRANCH is one way of restarting a model from some restart files, disabled by default.

Not much speed up can be obtained when GMAKE_J >4, based on some tests with F_2000. This number should be same as srun -c argument for build.

In general, the above parameters and more can be changed after you initially build the model by using ./xmlchange <VARIABLE NAME>=<new value>. Often, the variables you want to change are listed in env_run.xml. You will need to rebuild the model if you make any changes here - see step 5.

What to expect:

Right after create_newcase, copying user_nl_* and SourceMods might results in "cp: No match". These errors can be safely ignored.

Right after build, there are two lines that print out all non-empty, uncommented lines in user_nl_*; and list files in SourceMods. You might see "ls: No match" if no files are in SourceMods, please ignore.

Test-only sbatch is submitted to test sbatch settings. See step 5.

In case directory, a link named "scratch" will point you to the Build/Run/Short-term Archive directory.

The current is copied to the case directory. (xxx: bugs identified, to be fixed)

More about CESM

Directories and files in CESM

The above figure is adapted from p.48.

There is no need to copy the CESM download folder. (for CESM1, which someone already downloaded for you)

The inputdata directory is located in /n/holystore01/INTERNAL_REPOS/CLIMATE_MODELS/cesm_2_1/inputdata. Seb has opened write access so everyone should be able to download inputdata when needed. Please only download data that you need, as the whole data set is >20T in size.

CASE directory is default to be under ~/cesm_caseroot/$CASE

Build/Run directory and Short Term Archive is changed to use the same directory. That is, atm, lnd, etc. from short term archive, and bld and run, will now locate under the same folder in the same level.

Short term archive is procedure when the job completes, move files out of run directory, which are (1)-(3) described in the above figure.

Long-term archive is not needed on Harvard cluster, as there is no HPSS disk.

What to expect?

When the job is running, or the job got error and didn't reach the short-term archive, expect to see log files and output (e.g. history) files in run directory.

Naming conventions of nc files:

*.h* files are history files. Similar for r (restart), i (initial), d (diagnostic).

Default for CAM is to only output monthly history files. If your run is less than one month, you might get no history files. (Jan has 31 days.) To increase output, see namelist.

Namelist in CESM

xxx: migrating from

What is namelist?

Namelist in this page refers to text files read in by Fortran NAMELIST statements. In many Fortran numerical models (including CESM), namelist are used to specify runtime options, without having to rebuild the model. Sample namelist looks like:

 char_variable = "a char variable",
 logical_variable = .true.,
 real_variable = 12.3,
 int_array = 1, 2, 3, 4,

Read more about Fortran namelist in: link1, link2, link3.

How namelists are built in CESM?

$RUNDIR/*_in are namelists directly read by Fortran program. They are recommended NOT to be edited directly, but instead, indirectly through user_nl_* and env_*.xml. The processes of indirectly creating namelists are as follows:

Before submitting job:

  • calls cesm_setup and $
  • Both cesm_setup and $ call preview_namelists
  • preview_namelists 
    • calls Buildconf/$comp.buildnml.csh
    • copies *_in to CaseDocs/
  • Buildconf/$comp.buildnml.csh 
    • call $CCSMROOT/models/**/bld/build-namelist, which create namelist Buildconf/*conf/*_in
    • copy *_in to $RUNDIR (overwrite)
  • build-namelist 
    • takes “infile” from user_nl_*
    • writes out Buildconf/*conf/*_in and Buildconf/*.input_data_list

When job is submitted:

  • $CASE.submit submits $
  • $ calls preview_namelists
  • Same things happen again.


How to modify namelists in CESM?

If possible, always use user_nl_* and env_*.xml.

E.g. Change the path to an input data file: ./xmlchange SSTICE_DATA_FILENAME=/path/to/ (this can also be listed as a line in user_nl_cam).

E.g. To customize CAM output, edit user_nl_cam according to userguide, and master field list: v1.1.1, v1.2.2.

If impossible, manually change $RUNDIR/*_in, and disable copying (overwriting) in Buildconf/$comp.buildnml.csh

After modifying the namelist, run ./preview_namelists to submit the changes.


  1. grep -Ev '^\s*$|^\s*!' user_nl_*
    can display all non-empty uncommented lines in user_nl_*

  2. ./preview_namelists can update namelists (and input_data_list)

  3. grep '' Buildconf/*.input_data_list
    can list out all input data.
  4. DIN_LOC_ROOT=`./xmlquery -valonly -silent DIN_LOC_ROOT`
    ./check_input_data -inputdata $DIN_LOC_ROOT -check
    can check input data

  5. Section &seq_infodata_inparm in drv_in can be checked in $RUNDIR/cpl.log.$LID

  6. Always compare default namelist and new namelist

  7. Sometimes, user_nl_* are not interpreted literally. E.g. specifying prescribed_ozone_file could remove lines for prescribed_ozone_cycle_yr, prescribed_ozone_name, etc.

  8. Never edit namelist in CaseDocs/ or Buildconf/, they are overwritten every time and NOT read by Fortran program.



Compset is a short form for "component set", which specifies component models, forcing scenarios and physics options for those models.

Commonly used compset are: F_2000 (atmosphere and land only for 2000), F_2000_SPCAM_sam1mom (SPCAM), FC4AQUAP (aquaplanet, only CESM1.2+)

List all available compset by ~pchan/spcam2_0-cesm1_1_1/scripts/create_newcase -list, or browse

You can view technical details about compset in under case root directory, after you have created a case with that compset.

To create a new compset (e.g. Aquaplanet in SP-CESM), use the -compset_file argument in create_newcase:

     -compset_file <name> Full pathname of compset file to use. (optional)
                          See sample_compset_file.xml for an example.

(from ~pchan/spcam2_0-cesm1_1_1/scripts/create_newcase -help)

Make a copy of ~pchan/spcam2_0-cesm1_1_1/scripts/sample_compset_file.xml to your directory and edit it. And edit to add '-compset_file' argument when it calls create_newcase.

Details coming

SourceMod related

Technical Note (283 pp.):

Reference Manual:


CAM uses dry static energy as prognostic variable (instead of temperature). Detailed discretization is implemented in geopotential.F90:geopotential_dse, and described in 4.11.5 in cam4_desc.pdf.

Parallelization and indexing


Restart and looping

Based on CESM1.1.1. Following link is now public (9/27/2019).

Bugs and technical notes

/n/holylfs & /n/kuanglfs are near end of life. Both disks store inputdata. Please transition to /n/holystore01/INTERNAL_REPOS/CLIMATE_MODELS/cesm_2_1.
For CESM1, the command is ./xmlchange DIN_LOC_ROOT="/n/holystore01/INTERNAL_REPOS/CLIMATE_MODELS/cesm_2_1/inputdata"
For CESM2 (who follows our GoogleDoc instructions), the command is ln -sfT /n/holystore01/INTERNAL_REPOS/CLIMATE_MODELS/cesm_2_1/inputdata $CESMDATAROOT/inputdata
A temporary link has been created pointing from /n/holylfs/INTERNAL_REPOS/CLIMATE_MODELS to /n/holystore01/INTERNAL_REPOS/CLIMATE_MODELS. This temporary link will be removed when /n/holylfs decommission. Please run the above commands before the decommission.
The new place is limited to huce group. Updates (if any) will be posted here.
(last update: 7/30/2020)

Technical notes on how the model is ported to Harvard cluster:

Starting CESM1.2.2, cesm_postrun_setup is problematic.

CESM1.2.2 obtained from NCAR is missing a few directories since google apps no longer works.  If you are running in to this problem:

1. Missing pio directory in /models/utils/pio
svn checkout ./cesm1_2_2/models/utils/pio

2. Missing file
svn checkout ./cesm1_2_2/tools/cprnc/genf90

Unconfirmed: aquaplanet - src.aquap do not take effect.

References (look for practical sessions)


Transition from holylfs & kuanglfs to holystore01. Read more. (last update: 7/30/2020)

  • No labels