inciter::Partitioner class
#include <src/Inciter/Partitioner.hpp>

Partitioner Charm++ chare nodegroup class

Instantiations of Partitioner comprise a processor aware Charm++ chare node group. When instantiated, a new object is created on each compute node and not more (as opposed to individual chares or chare array object elements). See also the Charm++ interface file partitioner.ci.

Public static functions

static void registerReducers(): Configure Charm++ reduction types.

Constructors, destructors, conversion operators

Partitioner(std::size_t meshid, const std::string& filename, const tk::PartitionerCallback& cbp, const tk::RefinerCallback& cbr, const tk::SorterCallback& cbs, const CProxy_Transporter& host, const CProxy_Refiner& refiner, const CProxy_Sorter& sorter, const tk::CProxy_MeshWriter& meshwriter, const CProxy_Discretization& discretization, const CProxy_RieCG& riecg, const CProxy_LaxCG& laxcg, const CProxy_ZalCG& zalcg, const CProxy_KozCG& kozcg, const CProxy_ChoCG& chocg, const CProxy_LohCG& lohcg, const tk::CProxy_ConjugateGradients& cgpre, const tk::CProxy_ConjugateGradients& cgmom, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>& faces, const std::map<int, std::vector<std::size_t>>& bnode): Constructor.
Partitioner(CkMigrateMessage* m) explicit: Migrate constructor.

Public functions

void graph(CkReductionMsg* msg): Reduction target to aggregate mesh graphs.
void parts(CkReductionMsg* msg): Reduction target to aggregate mesh partition assginments.
void partition(int nchare): Partition the computational mesh into a number of chares.
void addMesh(int fromnode, const std::unordered_map<int, std::tuple<std::vector<std::size_t>, tk::UnsMesh::CoordMap, std::unordered_map<int, std::vector<std::size_t>>, std::unordered_map<int, std::vector<std::size_t>>>>& chmesh): Receive mesh associated to chares we own after refinement.
void recvMesh(): Acknowledge received mesh after initial mesh refinement.
void refine(): Optionally start refining the mesh.

Charm++ pack/unpack serializer member functions

std::size_t m_meshid: Mesh ID.
tk::PartitionerCallback m_cbp: Charm++ callbacks associated to compile-time tags for partitioner.
tk::RefinerCallback m_cbr: Charm++ callbacks associated to compile-time tags for refiner.
tk::SorterCallback m_cbs: Charm++ callbacks associated to compile-time tags for sorter.
CProxy_Transporter m_host: Host proxy.
CProxy_Refiner m_refiner: Mesh refiner proxy.
CProxy_Sorter m_sorter: Mesh sorter proxy.
tk::CProxy_MeshWriter m_meshwriter: Mesh writer proxy.
CProxy_Discretization m_discretization: Discretization base proxy.
CProxy_RieCG m_riecg: Discretization scheme proxy.
CProxy_LaxCG m_laxcg: Discretization scheme proxy.
CProxy_ZalCG m_zalcg: Discretization scheme proxy.
CProxy_KozCG m_kozcg: Discretization scheme proxy.
CProxy_ChoCG m_chocg: Discretization scheme proxy.
CProxy_LohCG m_lohcg: Discretization scheme proxy.
tk::CProxy_ConjugateGradients m_cgpre: Conjugate Gradients Charm++ proxy for pressure solve.
tk::CProxy_ConjugateGradients m_cgmom: Conjugate Gradients Charm++ proxy for momentum solve.
std::vector<std::size_t> m_ginpoel: Element connectivity of this compute node's mesh chunk (global ids)
std::unordered_map<std::size_t, std::vector<std::size_t>> m_graph: Aggregated mesh graph of owned nodes if graph-based partitioner is used.
tk::UnsMesh::Coords m_coord: Coordinates of mesh nodes of this compute node's mesh chunk.
std::vector<std::size_t> m_inpoel: Element connectivity with local node IDs of this compute node's mesh chunk.
std::unordered_map<std::size_t, std::size_t> m_lid
std::size_t m_ndist: Counter during mesh distribution.
int m_nchare: Total number of chares across all compute nodes.
std::unordered_map<int, std::size_t> m_nface: Counters (for each chare owned) for assigning face ids in parallel.
std::unordered_map<std::size_t, std::vector<int>> m_nodech
std::unordered_map<std::size_t, std::size_t> m_linnodes: Map associating new node IDs (as in producing contiguous-row-id linear system contributions) as map-values to old node IDs (as in file) as map-keys.
std::unordered_map<int, std::vector<std::size_t>> m_chinpoel: Mesh connectivity using global node IDs associated to chares owned.
std::unordered_map<int, tk::UnsMesh::CoordMap> m_chcoordmap: Coordinates associated to global node IDs of our mesh chunk for chares.
std::unordered_map<int, std::map<int, std::vector<std::size_t>>> m_chbface: Side set id + boundary face id for each chare.
std::map<int, std::vector<std::size_t>> m_chtriinpoel: Boundary face connectivity for each chare.
std::unordered_map<int, std::map<int, std::vector<std::size_t>>> m_chbnode: Side set id + boundary nodes for each chare.
std::unordered_map<std::size_t, std::vector<int>> m_bnodechares: Map associating a list of chare IDs to old (as in file) global mesh node IDs on the chare boundaries.
std::map<int, std::vector<std::size_t>> m_bface: Boundary face IDs associated associated to side set IDs.
std::vector<std::size_t> m_triinpoel: Boundary face-node connectivity.
std::map<int, std::vector<std::size_t>> m_bnode: List of boundary nodes associated to side-set IDs.
void pup(PUP::er& p) override: Pack/Unpack serialize member function.
void operator|(PUP::er& p, Partitioner& i): Pack/Unpack serialize operator|.

Function documentation

static void inciter::Partitioner::registerReducers()

Configure Charm++ reduction types.

Since this is a [initnode] routine, see the .ci file, the Charm++ runtime system executes the routine exactly once on every logical node early on in the Charm++ init sequence. Must be static as it is called without an object. See also: Section "Initializations at Program Startup" at in the Charm++ manual http://charm.cs.illinois.edu/manuals/html/charm++/manual.html.

inciter::Partitioner::Partitioner(std::size_t meshid, const std::string& filename, const tk::PartitionerCallback& cbp, const tk::RefinerCallback& cbr, const tk::SorterCallback& cbs, const CProxy_Transporter& host, const CProxy_Refiner& refiner, const CProxy_Sorter& sorter, const tk::CProxy_MeshWriter& meshwriter, const CProxy_Discretization& discretization, const CProxy_RieCG& riecg, const CProxy_LaxCG& laxcg, const CProxy_ZalCG& zalcg, const CProxy_KozCG& kozcg, const CProxy_ChoCG& chocg, const CProxy_LohCG& lohcg, const tk::CProxy_ConjugateGradients& cgpre, const tk::CProxy_ConjugateGradients& cgmom, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>& faces, const std::map<int, std::vector<std::size_t>>& bnode)

Constructor.

Parameters
meshid in	Mesh ID
filename in	Input mesh filename to read from
cbp in	Charm++ callbacks for Partitioner
cbr in	Charm++ callbacks for Refiner
cbs in	Charm++ callbacks for Sorter
host in	Host Charm++ proxy we are being called from
refiner in	Mesh refiner proxy
sorter in	Mesh reordering (sorter) proxy
meshwriter in	Mesh writer proxy
discretization in	Discretization base
riecg in	Discretization scheme
laxcg in	Discretization scheme
zalcg in	Discretization scheme
kozcg in	Discretization scheme
chocg in	Discretization scheme
lohcg in	Discretization scheme
cgpre in	ConjugateGradients Charm++ proxy for pressure solve
cgmom in	ConjugateGradients Charm++ proxy for momentum solve
bface in	File-internal elem ids of side sets (whole mesh)
faces in	Elem-relative face ids of side sets (whole mesh)
bnode in	Node lists of side sets (whole mesh)

void inciter::Partitioner::graph(CkReductionMsg* msg)

Reduction target to aggregate mesh graphs.

Parameters
msg in	Serialized aggregated mesh graph

void inciter::Partitioner::parts(CkReductionMsg* msg)

Reduction target to aggregate mesh partition assginments.

Parameters
msg in	Serialized aggregated mesh nodes partition assignments

void inciter::Partitioner::partition(int nchare)

Partition the computational mesh into a number of chares.

Parameters
nchare in	Number of parts the mesh will be partitioned into

This function calls the mesh partitioner to partition the mesh. The number of partitions equals the number nchare argument which must be no lower than the number of compute nodes.

void inciter::Partitioner::addMesh(int fromnode, const std::unordered_map<int, std::tuple<std::vector<std::size_t>, tk::UnsMesh::CoordMap, std::unordered_map<int, std::vector<std::size_t>>, std::unordered_map<int, std::vector<std::size_t>>>>& chmesh)

Receive mesh associated to chares we own after refinement.

Parameters
fromnode in	Compute node call coming from
chmesh in	Map associating mesh connectivities to global node ids and node coordinates for mesh chunks we are assigned by the partitioner

void inciter::Partitioner::pup(PUP::er& p) override

Pack/Unpack serialize member function.

Parameters
p in/out	Charm++'s PUP::er serializer object reference

void inciter::Partitioner::operator|(PUP::er& p, Partitioner& i)

Pack/Unpack serialize operator|.

Parameters
p in/out	Charm++'s PUP::er serializer object reference
i in/out	Partitioner object reference

Variable documentation

std::unordered_map<std::size_t, std::size_t> inciter::Partitioner::m_lid

Global->local node IDs of elements of this compute node's mesh chunk

Key: global node id, value: local node id

std::unordered_map<std::size_t, std::vector<int>> inciter::Partitioner::m_nodech

Chare IDs (value) associated to global mesh node IDs (key)

Multiple chares can contribute to a single node, hence vector for map value.

std::unordered_map<std::size_t, std::vector<int>> inciter::Partitioner::m_bnodechares

Map associating a list of chare IDs to old (as in file) global mesh node IDs on the chare boundaries.

Note that a single global mesh node ID can be associated to multiple chare IDs as multiple chares can contribute to a single node.