inciter::Partitioner class

Partitioner Charm++ chare nodegroup class

Instantiations of Partitioner comprise a processor aware Charm++ chare node group. When instantiated, a new object is created on each compute node and not more (as opposed to individual chares or chare array object elements). See also the Charm++ interface file partitioner.ci.

Public static functions

static void registerReducers()
Configure Charm++ reduction types.

Constructors, destructors, conversion operators

Partitioner(std::size_t meshid, const std::string& filename, const tk::PartitionerCallback& cbp, const tk::RefinerCallback& cbr, const tk::SorterCallback& cbs, const CProxy_Transporter& host, const CProxy_Refiner& refiner, const CProxy_Sorter& sorter, const tk::CProxy_MeshWriter& meshwriter, const CProxy_Discretization& discretization, const CProxy_RieCG& riecg, const CProxy_LaxCG& laxcg, const CProxy_ZalCG& zalcg, const CProxy_KozCG& kozcg, const CProxy_ChoCG& chocg, const tk::CProxy_ConjugateGradients& cgpre, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>& faces, const std::map<int, std::vector<std::size_t>>& bnode)
Constructor.
Partitioner(CkMigrateMessage* m) explicit
Migrate constructor.

Public functions

void graph(CkReductionMsg* msg)
Reduction target to aggregate mesh graphs.
void parts(CkReductionMsg* msg)
Reduction target to aggregate mesh partition assginments.
void partition(int nchare)
Partition the computational mesh into a number of chares.
void addMesh(int fromnode, const std::unordered_map<int, std::tuple<std::vector<std::size_t>, tk::UnsMesh::CoordMap, std::unordered_map<int, std::vector<std::size_t>>, std::unordered_map<int, std::vector<std::size_t>>>>& chmesh)
Receive mesh associated to chares we own after refinement.
void recvMesh()
Acknowledge received mesh after initial mesh refinement.
void refine()
Optionally start refining the mesh.

Charm++ pack/unpack serializer member functions

std::size_t m_meshid
Mesh ID.
tk::PartitionerCallback m_cbp
Charm++ callbacks associated to compile-time tags for partitioner.
tk::RefinerCallback m_cbr
Charm++ callbacks associated to compile-time tags for refiner.
tk::SorterCallback m_cbs
Charm++ callbacks associated to compile-time tags for sorter.
CProxy_Transporter m_host
Host proxy.
CProxy_Refiner m_refiner
Mesh refiner proxy.
CProxy_Sorter m_sorter
Mesh sorter proxy.
tk::CProxy_MeshWriter m_meshwriter
Mesh writer proxy.
CProxy_Discretization m_discretization
Discretization base proxy.
CProxy_RieCG m_riecg
Discretization scheme proxy.
CProxy_LaxCG m_laxcg
Discretization scheme proxy.
CProxy_ZalCG m_zalcg
Discretization scheme proxy.
CProxy_KozCG m_kozcg
Discretization scheme proxy.
CProxy_ChoCG m_chocg
Discretization scheme proxy.
tk::CProxy_ConjugateGradients m_cgpre
Conjugate Gradients Charm++ proxy for pressure solve.
std::vector<std::size_t> m_ginpoel
Element connectivity of this compute node's mesh chunk (global ids)
std::unordered_map<std::size_t, std::vector<std::size_t>> m_graph
Aggregated mesh graph of owned nodes if graph-based partitioner is used.
tk::UnsMesh::Coords m_coord
Coordinates of mesh nodes of this compute node's mesh chunk.
std::vector<std::size_t> m_inpoel
Element connectivity with local node IDs of this compute node's mesh chunk.
std::unordered_map<std::size_t, std::size_t> m_lid
std::size_t m_ndist
Counter during mesh distribution.
int m_nchare
Total number of chares across all compute nodes.
std::unordered_map<int, std::size_t> m_nface
Counters (for each chare owned) for assigning face ids in parallel.
std::unordered_map<std::size_t, std::vector<int>> m_nodech
std::unordered_map<std::size_t, std::size_t> m_linnodes
Map associating new node IDs (as in producing contiguous-row-id linear system contributions) as map-values to old node IDs (as in file) as map-keys.
std::unordered_map<int, std::vector<std::size_t>> m_chinpoel
Mesh connectivity using global node IDs associated to chares owned.
std::unordered_map<int, tk::UnsMesh::CoordMap> m_chcoordmap
Coordinates associated to global node IDs of our mesh chunk for chares.
std::unordered_map<int, std::map<int, std::vector<std::size_t>>> m_chbface
Side set id + boundary face id for each chare.
std::map<int, std::vector<std::size_t>> m_chtriinpoel
Boundary face connectivity for each chare.
std::unordered_map<int, std::map<int, std::vector<std::size_t>>> m_chbnode
Side set id + boundary nodes for each chare.
std::unordered_map<std::size_t, std::vector<int>> m_bnodechares
Map associating a list of chare IDs to old (as in file) global mesh node IDs on the chare boundaries.
std::map<int, std::vector<std::size_t>> m_bface
Boundary face IDs associated associated to side set IDs.
std::vector<std::size_t> m_triinpoel
Boundary face-node connectivity.
std::map<int, std::vector<std::size_t>> m_bnode
List of boundary nodes associated to side-set IDs.
void pup(PUP::er& p) override
Pack/Unpack serialize member function.
void operator|(PUP::er& p, Partitioner& i)
Pack/Unpack serialize operator|.

Function documentation

static void inciter::Partitioner::registerReducers()

Configure Charm++ reduction types.

Since this is a [initnode] routine, see the .ci file, the Charm++ runtime system executes the routine exactly once on every logical node early on in the Charm++ init sequence. Must be static as it is called without an object. See also: Section "Initializations at Program Startup" at in the Charm++ manual http://charm.cs.illinois.edu/manuals/html/charm++/manual.html.

inciter::Partitioner::Partitioner(std::size_t meshid, const std::string& filename, const tk::PartitionerCallback& cbp, const tk::RefinerCallback& cbr, const tk::SorterCallback& cbs, const CProxy_Transporter& host, const CProxy_Refiner& refiner, const CProxy_Sorter& sorter, const tk::CProxy_MeshWriter& meshwriter, const CProxy_Discretization& discretization, const CProxy_RieCG& riecg, const CProxy_LaxCG& laxcg, const CProxy_ZalCG& zalcg, const CProxy_KozCG& kozcg, const CProxy_ChoCG& chocg, const tk::CProxy_ConjugateGradients& cgpre, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>& faces, const std::map<int, std::vector<std::size_t>>& bnode)

Constructor.

Parameters
meshid in Mesh ID
filename in Input mesh filename to read from
cbp in Charm++ callbacks for Partitioner
cbr in Charm++ callbacks for Refiner
cbs in Charm++ callbacks for Sorter
host in Host Charm++ proxy we are being called from
refiner in Mesh refiner proxy
sorter in Mesh reordering (sorter) proxy
meshwriter in Mesh writer proxy
discretization in Discretization base
riecg in Discretization scheme
laxcg in Discretization scheme
zalcg in Discretization scheme
kozcg in Discretization scheme
chocg in Discretization scheme
cgpre in ConjugateGradients Charm++ proxy for pressure solve
bface in File-internal elem ids of side sets (whole mesh)
faces in Elem-relative face ids of side sets (whole mesh)
bnode in Node lists of side sets (whole mesh)

void inciter::Partitioner::graph(CkReductionMsg* msg)

Reduction target to aggregate mesh graphs.

Parameters
msg in Serialized aggregated mesh graph

void inciter::Partitioner::parts(CkReductionMsg* msg)

Reduction target to aggregate mesh partition assginments.

Parameters
msg in Serialized aggregated mesh nodes partition assignments

void inciter::Partitioner::partition(int nchare)

Partition the computational mesh into a number of chares.

Parameters
nchare in Number of parts the mesh will be partitioned into

This function calls the mesh partitioner to partition the mesh. The number of partitions equals the number nchare argument which must be no lower than the number of compute nodes.

void inciter::Partitioner::addMesh(int fromnode, const std::unordered_map<int, std::tuple<std::vector<std::size_t>, tk::UnsMesh::CoordMap, std::unordered_map<int, std::vector<std::size_t>>, std::unordered_map<int, std::vector<std::size_t>>>>& chmesh)

Receive mesh associated to chares we own after refinement.

Parameters
fromnode in Compute node call coming from
chmesh in Map associating mesh connectivities to global node ids and node coordinates for mesh chunks we are assigned by the partitioner

void inciter::Partitioner::pup(PUP::er& p) override

Pack/Unpack serialize member function.

Parameters
in/out Charm++'s PUP::er serializer object reference

void inciter::Partitioner::operator|(PUP::er& p, Partitioner& i)

Pack/Unpack serialize operator|.

Parameters
in/out Charm++'s PUP::er serializer object reference
in/out Partitioner object reference

Variable documentation

std::unordered_map<std::size_t, std::size_t> inciter::Partitioner::m_lid

Global->local node IDs of elements of this compute node's mesh chunk

Key: global node id, value: local node id

std::unordered_map<std::size_t, std::vector<int>> inciter::Partitioner::m_nodech

Chare IDs (value) associated to global mesh node IDs (key)

Multiple chares can contribute to a single node, hence vector for map value.

std::unordered_map<std::size_t, std::vector<int>> inciter::Partitioner::m_bnodechares

Map associating a list of chare IDs to old (as in file) global mesh node IDs on the chare boundaries.

Note that a single global mesh node ID can be associated to multiple chare IDs as multiple chares can contribute to a single node.